US6054989A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects and which provides spatialized audio
Assignee: Microsoft Technology Licensing LLC
•
Filing: 1998-09-14
•
Granted: 2000-04-25
•
Expires: 2018-09-14
Abstract
Abstract A graphical user interface in which object thumbnails are rendered on a simulated three-dimensional surface which (i) exploits spatial memory and (ii) allows more objects to be rendered on a given screen. The objects may be moved, continuously, on the surface with a two-dimensional input device.
Inventors
- 👤 George G. Robertson
- 👤 David D. Thiel
- 👤 Maarten Roderik Van Dantzich
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-29 22:58
CPC Classifications
G; G06; G06F; G06F3/00; G06F3/01; G06F3/048; G06F3/0481; G06F3/04815; G06F3/16; G06F3/167
Claims Analysis
Total Claims: 20
Claim 1
Independent Claim
292 words
A system which permits a user to interact with objects, the system comprising: a) an input facility for accepting user inputs; b) a storage facility containing i) location and state information for each of the objects, wherein the state information for each of the objects includes an indication of whether or not the object is active, ii) a visual representation of each of the objects, iii) a cursor location, iv) a three-dimensional environment including a three-dimensional surface, and v) a first audio cue; c) a processing unit which i) accepts user inputs from the input facility, ii) updates (a) the location and state information for each of the objects contained in the storage facility, and (b) the cursor location contained in the storage facility, based on the accepted user inputs, and iii) generates video outputs based on A) the location and state information for each of the objects, B) the visual representation of each of the objects, C) the cursor location, and D) the three-dimensional surface, contained in the storage facility; d) a video display unit for rendering the video outputs generated by the processing unit; and e) an audio output device, wherein the processing unit determines that an object is active if a cursor is on an object based on the cursor location and the location of the object, wherein, if an object is active and the input facility accepts a move input, then i) the processing unit updates the state and location of the object, ii) the processing unit generates a video output based on the updated location of the object, iii) the video display device renders the video output generated by the processing unit, and iv) the processing unit provides the first audio cue to the audio output device.
Claim 2
Independent Claim
292 words
A system which permits a user to interact with objects, the system comprising: a) an input facility for accepting user inputs; b) a storage facility containing i) location and state information for each of the objects, wherein the state information for each of the objects includes an indication of whether or not the object is active, ii) a visual representation of each of the objects, iii) a cursor location, iv) a three-dimensional environment including a three-dimensional surface, and v) a first audio cue; c) a processing unit which i) accepts user inputs from the input facility, ii) updates (a) the location and state information for each of the objects contained in the storage facility, and (b) the cursor location contained in the storage facility, based on the accepted user inputs, and iii) generates video outputs based on A) the location and state information for each of the objects, B) the visual representation of each of the objects, C) the cursor location, and D) the three-dimensional surface, contained in the storage facility; d) a video display unit for rendering the video outputs generated by the processing unit; and e) an audio output device, wherein the processing unit determines that an object is active if a cursor is on an object based on the cursor location and the location of the object, wherein, if an object is active and the input facility accepts a move input, then i) the processing unit updates the state and location of the object, ii) the processing unit generates a video output based on the updated location of the object, iii) the video display device renders the video output generated by the processing unit, and iv) the processing unit provides the first audio cue to the audio output device.
Claim 3
Dependent Claim
224 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a selection input and if an active object exists: i1) generating an animation moving the visual representation of the associated object to a preferred viewing location, which makes the object appear much closer and therefore larger, to be rendered on the video display device; and i2) generating a second audio cue associated with object selection.
Claim 4
Dependent Claim
224 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a selection input and if an active object exists: i1) generating an animation moving the visual representation of the associated object to a preferred viewing location, which makes the object appear much closer and therefore larger, to be rendered on the video display device; and i2) generating a second audio cue associated with object selection.
Claim 5
Dependent Claim
253 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein the second audio cue is spatialized such that object movement in a virtual foreground of the three-dimensional environment is louder than object movement in a virtual background of the three-dimensional environment.
Claim 6
Dependent Claim
253 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein the second audio cue is spatialized such that object movement in a virtual foreground of the three-dimensional environment is louder than object movement in a virtual background of the three-dimensional environment.
Claim 7
Dependent Claim
255 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein the second audio cue is spatialized such that object movement in a left hand side or right hand side of the three dimensional environment generates a left or right balanced audio cue, respectively.
Claim 8
Dependent Claim
255 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein the second audio cue is spatialized such that object movement in a left hand side or right hand side of the three dimensional environment generates a left or right balanced audio cue, respectively.
Claim 9
Dependent Claim
249 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein the second audio cue has a pitch which is proportional to a speed at which the object moves in the interface rendered on the video display device.
Claim 10
Dependent Claim
249 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein the second audio cue has a pitch which is proportional to a speed at which the object moves in the interface rendered on the video display device.
Claim 11
Dependent Claim
249 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein a reverberation ratio parameter of the second audio cue is adjusted based on a location of the object in the interface rendered on the video display device.
Claim 12
Dependent Claim
249 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein a reverberation ratio parameter of the second audio cue is adjusted based on a location of the object in the interface rendered on the video display device.
Claim 13
Dependent Claim
38 words
The man-machine interface method of claim 6 wherein the ratio of reverberation to dry sound is increased as the location of the object moves from the foreground of the three-dimensional environment to the background of the three-dimensional environment.
Claim 14
Dependent Claim
239 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein high frequency components of the second audio cue are adjusted based on the location of the object.
Claim 15
Dependent Claim
239 words
A man-machine interface method for permitting a user to act on objects, for use with a machine having a video display device and a user input device, the man-machine interface method comprising: a) generating a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determining a virtual location environment of each of the objects in the three-dimensional environment; c) generating visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepting inputs from the user input device; e) determining a cursor location based on the accepted inputs; f) generating the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defining that object as an active object; h) when an object is defined as an active object, generating a first audio cue associated with object activation; and i) if the user input provides a move input and if an active object exists, then: i1) updating a location of the active object; i2) generating the visual representation of the active object at its updated location, to be rendered on the video display device; and i3) generating an audio cue associated with moving an object; and j) wherein high frequency components of the second audio cue are adjusted based on the location of the object.
Claim 16
Dependent Claim
36 words
The man-machine interface method of claim 8 wherein the high frequency components of the audio cue are attenuated as the object moves from the foreground of the three-dimensional environment to the background of the three-dimensional environment.
Claim 17
Dependent Claim
249 words
A computer-based system having a man-machine interface, which permits a user to act on objects, for use in conjunction with a video display device and a user input device, the system comprising: a processor; a memory connected to the processor and storing computer executable instructions therein; and wherein the processor, in response to execution of the instructions: a) generates a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determines a virtual location environment of each of the objects in the three-dimensional environment; c) generates visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepts inputs from the user input device; e) determines a cursor location based on the accepted inputs; f) generates the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defines that object as an active object; h) when an object is defined as an active object, generates a first audio cue associated with object activation; and i) if the user input provides a selection input and if an active object exists: i1) generates an animation moving the visual representation of the associated object to a preferred viewing location, which makes the object appear much closer and therefore larger, to be rendered on the video display device; and i2) generates a second audio cue associated with object selection.
Claim 18
Dependent Claim
249 words
A computer-based system having a man-machine interface, which permits a user to act on objects, for use in conjunction with a video display device and a user input device, the system comprising: a processor; a memory connected to the processor and storing computer executable instructions therein; and wherein the processor, in response to execution of the instructions: a) generates a three-dimensional environment, having a three-dimensional surface, to be rendered on the video display device; b) determines a virtual location environment of each of the objects in the three-dimensional environment; c) generates visual representations of the objects, within the three-dimensional environment, at the determined locations, to be rendered on the video display device; d) accepts inputs from the user input device; e) determines a cursor location based on the accepted inputs; f) generates the cursor at the determined cursor location, to be rendered on the video display device; g) if the cursor is located on a location of one of the objects, defines that object as an active object; h) when an object is defined as an active object, generates a first audio cue associated with object activation; and i) if the user input provides a selection input and if an active object exists: i1) generates an animation moving the visual representation of the associated object to a preferred viewing location, which makes the object appear much closer and therefore larger, to be rendered on the video display device; and i2) generates a second audio cue associated with object selection.
Claim 19
Dependent Claim
37 words
The system in claim 10 wherein the processor, in response to the stored instructions, generates a pop-up textual information bar located over the visual representation of the active object, to be rendered on the video display device.
Claim 20
Dependent Claim
17 words
A computer readable medium having computer readable instructions stored therein for performing the steps of claim 2.
References Cited (Backward Citations)
US5042006A
Cited
US5155806A
Cited
US5461399A
Cited
US5956028A
Cited
US5898435A
Cited
US5847709A
Cited
US5956025A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-04-25 | 2003-10-25 | 2004-04-25 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-04-25 | 2007-10-25 | 2008-04-25 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-04-25 | 2011-10-25 | 2012-04-25 | $7700.00 | Window Closed / Lapsed |