Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects

Claim 2

A man-machine interface method for permitting a user to act on thumbnails, each thumbnail being a scaled image of an associated object containing information, for use with a computer-implemented system having a video display device and a user input device, the method comprising the steps of: a) depicting a virtual three-dimensional environment, having a three-dimensional surface, on the video display device; b) determining a virtual location of each of the thumbnails in the environment so as to define a plurality of determined locations; c) rendering each of the thumbnails, within the environment, at a corresponding one of the determined locations so as to define a plurality of rendered thumbnails; d) if a cursor, as rendered on the display device, is positioned, by the user, on one of the rendered thumbnails, designating said one thumbnail as being an active thumbnail; and e) if the user instructs the system, through a move input provided through the user input device, to move the active thumbnail in the environment, in response to the move input: e1) updating a corresponding one of the plurality of determined locations associated with the active thumbnail so as to define an updated location; e2) rendering the active thumbnail at the updated location; and e3) if the updated location of the active thumbnail lies within a predetermined distance of a corresponding one of the plurality of determined locations associated with another one of the thumbnails: e3a) defining the other thumbnail as a pushed thumbnail; e3b) changing the corresponding one of the determined locations associated with the pushed thumbnail so as to define a changed location, wherein the changed location maintains a predetermined distance between the other thumbnail and the active thumbnail so as to prevent the visual representations of both the pushed and active thumbnails from being totally occluded by each other; and e3c) rendering the pushed thumbnail at the changed location in the environment.

Claim 3

The man-machine interface method of claim 1 wherein the three-dimensional surface comprises an inclined plane.

Claim 4

The man-machine interface method of claim 2 wherein the inclined plane includes landmarks.

Claim 5

The man-machine interface method of claim 3 wherein the landmarks are depicted as colored circles on the inclined plane.

Claim 6

The man-machine interface method of claim 1 further comprising the steps of: f) accepting inputs from the user input device so as to define accepted inputs; g) determining the cursor location based on the accepted inputs so as to define a determined cursor location; h) generating the cursor at the determined cursor location, to be rendered on the video display device.

Claim 7

The man-machine interface method of claim 5 wherein the step of determining the cursor location includes the sub-steps of: f1) determining a cursor position in a two-dimensional plane; and f2) mapping the cursor position in the two-dimensional plane to the cursor location in the environment.

Claim 8

The man-machine interface method of claim 5 further comprising a step of: i) generating a pop-up information bar located over the active thumbnail, to be rendered on the video display device.

Claim 9

The man-machine interface method of claim 5 further comprising a step of: i) if the user input provides a selection input and if the active thumbnail exists, then generating a higher resolution visual representation of the object represented by and associated with the active thumbnail, at a preferred viewing location at a foreground of the three dimensional environment, to be rendered on the video display device.

Claim 10

The man-machine interface method of claim 8 wherein the selection input comprises a mouse click.

Claim 11

The man-machine interface method of claim 8 wherein the step of generating the higher resolution visual representation of the object represented by and associated with the active thumbnail includes a sub-step of: generating an animation which moves the higher resolution visual representation of the object represented by and associated with the active thumbnail from the location of the active thumbnail to the preferred viewing location at the foreground of the three dimensional environment, to be rendered on the video display device.

Claim 12

The man-machine interface method of claim 10 further comprising a step of: i) if the user input provides a deselection input and if a selected thumbnail exists, then generating a video output for moving the high resolution visual representation of the object represented by and associated with the active thumbnail to the location of the selected thumbnail, to be rendered on the video display device.

Claim 13

The man-machine interface method of claim 11 wherein the deselection input comprises a mouse click.

Claim 14

The man-machine interface method of claim 5 further comprising a step of: i) if the user input provides a selection input and if the active thumbnail exists, then i) invoking an application related to the object represented by and associated with the active thumbnail, ii) loading the object represented by and associated with the active thumbnail into the application, and iii) generating a video output of the application with the loaded object represented by and associated with the active thumbnail at a preferred viewing location, to be rendered on the video display device.

Claim 15

The man-machine interface method of claim 13 wherein the selection input is a mouse click.

Claim 16

The man-machine interface method of claim 5 wherein the move input comprises a left button mouse drag.

Claim 17

The man-machine interface method of claim 1 wherein the three-dimensional surface defines a foreground and a background, and wherein the step of generating thumbnails, within the three-dimensional environment, at the determined locations, to be rendered on the video display device, includes a sub-step of: i) using perspective views so that any thumbnails in the foreground defined by the three-dimensional surface appear larger than any thumbnails in the background defined by the three-dimensional surface.

Claim 18

The man-machine interface method of claim 16 wherein a non-active one of the thumbnails partially occludes any other non-active ones of the thumbnails located behind it, based on a viewing point.

Claim 19

The man-machine interface method of claim 1 further comprising steps of: f) accepting inputs from the user input device; g) determining a head location of the user relative to the video display device based on the accepted inputs; h) determining a virtual viewing point, in the three-dimensional environment, based on the head location determined in step (g); and i) generating the three-dimensional environment and the thumbnails with simulated parallax based on the virtual viewing point determined in step (h), to be rendered on the video display device.

Claim 20

The man-machine interface method of claim 1 wherein the thumbnails comprise low resolution bit maps.

Claim 21

The man-machine interface method of claim 19 wherein the low resolution bit maps comprise 64 pixels by 64 pixels and have 24 bit color.

Claim 22

The man-machine interface method of claim 1 wherein the objects are selected from a group consisting of (a) an HTML document, (b) a database record, (c) a word processor document, or (d) a contact.

Claim 23

A computer readable medium, having computer executable instructions stored therein, for performing the steps of claim 1.

Claim 24

A computer-implemented system which permits a user to act on thumbnails, each thumbnail being a scaled image of an associated object containing information, wherein the system comprises: a) a processor; b) a memory, connected to the processor and storing computer executable instructions therein; c) a user input device which provides user input for use by the processor; and d) a video display device which displays information provided by the processor; e) wherein the processor, in response to the executable instructions: e1) depicts a virtual three-dimensional environment, having a three-dimensional surface, on the video display device; e2) determines a virtual location of each of the thumbnails in the environment so as to define a plurality of determined locations; e3) renders each of the thumbnails, within the environment, at a corresponding one of the determined locations so as to define a plurality of rendered thumbnails; e4) if a cursor, as rendered on the display device, is positioned, by the user, on one of the rendered thumbnails, designates said one thumbnail as being an active thumbnail; and e5) if the user instructs the system, through a move input provided through the user input device, to move the active thumbnail in the environment, in response to the move input: e5a) updates a corresponding one of the plurality of determined locations associated with the active thumbnail so as to define an updated location; e5b) renders the active thumbnail at the updated location; and e5c) if the updated location of the active thumbnail lies within a predetermined distance of a corresponding one of the plurality of determined locations associated with another one of the thumbnails: e5c(i)) defines the other thumbnail as a pushed thumbnail; e5c(ii)) changes the corresponding one of the determined locations associated with the pushed thumbnail so as to define a changed location, wherein the changed location maintains a predetermined distance between the other thumbnail and the active thumbnail so as to prevent the visual representations of both the pushed and active thumbnails from being totally occluded by each other; and e5c(iii)) renders the pushed thumbnail at the changed location in the environment.

Claim 25

A computer-implemented system which permits a user to act on thumbnails, each thumbnail being a scaled image of an associated object containing information, wherein the system comprises: a) a processor; b) a memory, connected to the processor and storing computer executable instructions therein; c) a user input device which provides user input for use by the processor; and d) a video display device which displays information provided by the processor; e) wherein the processor, in response to the executable instructions: e1) depicts a virtual three-dimensional environment, having a three-dimensional surface, on the video display device; e2) determines a virtual location of each of the thumbnails in the environment so as to define a plurality of determined locations; e3) renders each of the thumbnails, within the environment, at a corresponding one of the determined locations so as to define a plurality of rendered thumbnails; e4) if a cursor, as rendered on the display device, is positioned, by the user, on one of the rendered thumbnails, designates said one thumbnail as being an active thumbnail; and e5) if the user instructs the system, through a move input provided through the user input device, to move the active thumbnail in the environment, in response to the move input: e5a) updates a corresponding one of the plurality of determined locations associated with the active thumbnail so as to define an updated location; e5b) renders the active thumbnail at the updated location; and e5c) if the updated location of the active thumbnail lies within a predetermined distance of a corresponding one of the plurality of determined locations associated with another one of the thumbnails: e5c(i)) defines the other thumbnail as a pushed thumbnail; e5c(ii)) changes the corresponding one of the determined locations associated with the pushed thumbnail so as to define a changed location, wherein the changed location maintains a predetermined distance between the other thumbnail and the active thumbnail so as to prevent the visual representations of both the pushed and active thumbnails from being totally occluded by each other; and e5c(iii)) renders the pushed thumbnail at the changed location in the environment.

Claim 26

The system of claim 23 wherein the memory stores state information for each of the thumbnails, the state information comprising an indication of whether or not said each thumbnail is active, and wherein the processor, in response to the stored instructions, determines that one of the thumbnails is active if the cursor is located on said one thumbnail based on a cursor location and a current location of the one thumbnail.

Claim 27

The system of claim 24 wherein the processor, in response to the stored instructions: generates a pop-up bar, based on descriptive textual information, for the active thumbnail; and renders, through the video display device, the pop-up bar in the vicinity of the active thumbnail.

Claim 28

The system of claim 24 wherein the processor, in response to the stored instructions: updates a state of the active thumbnail based on a selection input provided by the user through the user input device; accesses, from the memory, a second, higher resolution, visual representation of the object represented by and associated with the active thumbnail; generates a video output based on the higher resolution, visual representation of the object represented by and associated with the active thumbnail situated at a preferred viewing location in the environment; and renders, through the video display device, the video output.

Claim 29

The system of claim 26 further comprising an audio output device, wherein the memory stores a first audio cue; and wherein, when one of the thumbnails is selected by the user, the processor, in response to the stored instructions, provides the first audio cue to the audio output device.

Claim 30

The system of claim 26 wherein each of the plurality of thumbnails comprises a 64 pixel by 64 pixel bit map having 24 bit color and wherein each higher resolution, visual representation of the objects comprises a 512 pixel by 512 pixel bit map having 24 bit color.

Claim 31

The system of claim 26 wherein the processor, in response to the stored instructions, effects a video output based on an animation of the higher resolution, visual representation of the object represented by and associated with the active thumbnail moving from the corresponding one of the determined locations associated with the active thumbnail to a location at a foreground of the environment.

Claim 32

The system of claim 26 wherein the user input device comprises a mouse and the selection input comprises a mouse click.

Claim 33

The system of claim 26 wherein the processor, in response to the stored instructions and the move input: updates the state information and location of the active thumbnail; generates the video output based on the updated location of the active thumbnail; and renders, through the display device, the video output.

Claim 34

The system of claim 31 wherein the user input device comprises a mouse and the move input comprises a left button mouse drag.

Claim 35

The system of claim 24 wherein the processor, in response to the stored instructions and for the active thumbnail: updates the state information of the active thumbnail; opens an application with which the object, associated with and represented by the active thumbnail, is related, so as to define an opened application; loads the object into the opened application; generates a video output based on the object loaded onto the opened application and a preferred viewing location; and renders, through the video display device, the video output.

Claim 36

The system of claim 33 wherein the user input device comprises a mouse and the selection input comprises a mouse click.

Claim 37

The system of claim 23 wherein the memory stores virtual viewing point location information and the system further comprises a camera, and wherein the processor, in response to the executable instructions: determines a head location of the user relative to the video display device so as to define a determined head location; updates the virtual viewing point location information based on the determined head location so as to define an updated virtual viewing point location; generates, in response to the updated virtual viewing point location, a video output which simulates parallax in the three-dimensional surface of the environment and in the thumbnails displayed in the environment; and renders the video output through the video display device.