US6144375A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Multi-perspective viewer for content-based interactivity
Assignee: Eastman Kodak Co
•
Filing: 1998-08-14
•
Granted: 2000-11-07
•
Expires: 2018-08-14
Abstract
Abstract A method and apparatus for interactively viewing a real-world environment. The viewer includes a user interface having a first window for displaying a two-dimensional representation of a three-dimensional model of the real world environment. The viewer further includes a plurality of other viewing areas for displaying and querying for views of the real-world environment. The viewer includes a content-based event timeline that graphically depicts multi-media events satisfying user queries. Several methods can be used to select an event for display. For example, the user can select an event by selecting the event representation from the timeline. Alternatively, the user can select an event by querying the system for selected objects, viewing angles, input devices, etc. The viewer synchronizes and links together all of the multi-media data types associated with a selected event. Thus, when the user selects an event, the viewer displays all of the multi-media information (such as audio/video and textual information) that is associated with the selected event.
Inventors
- 👤 Ramesh C. Jain
- 👤 Terry Randolph Hicks
- 👤 Asquith A. Bailey
- 👤 Ryan B. McKinley
- 👤 Don Yamato Kuramura
- 👤 Arun L. Katkere
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-24 18:28
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/40; G06F16/48; G11; G11B; G11B27/00; G11B27/10; G11B27/102; G11B27/105; Y; Y10; Y10S; Y10S707/00; Y10S707/99941; Y10S707/99944; Y10S707/99945; Y10S707/99948
Claims Analysis
Total Claims: 37
Claim 1
Independent Claim
148 words
A multi-perspective viewer for interactively viewing a multi-media program derived from a real-world environment, the multi-media program comprising a plurality of multi-media events represented by associated and corresponding multiple multi-media data types stored in a multi-media database, comprising: a. user interface means for accepting input from a user and for outputting information from the viewer to the user, wherein the user interface means includes at least one input device and a visual display; b. query means for querying the viewer for selected multi-media events, wherein the query means displays an event timeline on the visual display that graphically depicts events that satisfy a selected query; c. selecting means, responsive to input from the user interface means, for selecting an event from the event timeline; and d. display means, responsive to the selecting means, for displaying the multi-media data types associated with the selected event on the visual display.
Claim 2
Dependent Claim
148 words
A multi-perspective viewer for interactively viewing a multi-media program derived from a real-world environment, the multi-media program comprising a plurality of multi-media events represented by associated and corresponding multiple multi-media data types stored in a multi-media database, comprising: a. user interface means for accepting input from a user and for outputting information from the viewer to the user, wherein the user interface means includes at least one input device and a visual display; b. query means for querying the viewer for selected multi-media events, wherein the query means displays an event timeline on the visual display that graphically depicts events that satisfy a selected query; c. selecting means, responsive to input from the user interface means, for selecting an event from the event timeline; and d. display means, responsive to the selecting means, for displaying the multi-media data types associated with the selected event on the visual display.
Claim 3
Dependent Claim
27 words
The multi-perspective viewer of claim 1, wherein the database is created by filtering the multi-media program and capturing only the multi-media events that satisfy event selection criteria.
Claim 4
Dependent Claim
28 words
The multi-perspective viewer of claim 2, wherein the database is a spatial/temporal database and wherein the multiple multi-media data types are indexed for subsequent retrieval by query means.
Claim 5
Dependent Claim
13 words
The multi-perspective viewer of claim 3, wherein the database is a relational database.
Claim 6
Dependent Claim
11 words
The multi-perspective viewer of claim 3, wherein the database is object-oriented.
Claim 7
Dependent Claim
18 words
The multi-perspective viewer of claim 2 wherein the event selection criteria is specific to a particular multi-media program.
Claim 8
Dependent Claim
28 words
The multi-perspective viewer of claim 2 wherein the event selection criteria is determined by a personality module having pre-defined knowledge of the multi-media program processed by the viewer.
Claim 9
Dependent Claim
20 words
The multi-perspective viewer of claim 7, wherein the personality module comprises a utility process capable of identifying significant multi-media events.
Claim 10
Dependent Claim
18 words
The multi-perspective viewer of claim 2, wherein the multi-media event selection criteria include inputs from a system user.
Claim 11
Dependent Claim
17 words
The multi-perspective viewer of claim 9, wherein the system user inputs include annotation data and control information.
Claim 12
Dependent Claim
19 words
The multi-perspective viewer of claim 10, wherein the system user inputs include event starting and event ending control inputs.
Claim 13
Dependent Claim
18 words
The multi-perspective viewer of claim 10, wherein the system user inputs include delete, save and override control inputs.
Claim 14
Dependent Claim
33 words
The multi-perspective viewer of claim 3, wherein the multi-media database includes fully synchronized audio, video, statistical data, and other data corresponding to the plurality of mutli-media events that satisfy the event selection criteria.
Claim 15
Dependent Claim
18 words
The multi-perspective viewer of claim 13, wherein the multi-media database is stored in a hot-swappable audio-video hard disk.
Claim 16
Dependent Claim
116 words
The multi-perspective viewer of claim 1, wherein the display means controls the display of the following windows on the visual display: a. a video window capable of displaying real-time video clips of the selected event; b. a user control window whereby the user can control the display of information on the visual display; c. a 2D model window capable of displaying a 3D model of the selected event displayed in the video window; d. a model navigation control window that allows the user to control the display of video information in the video window and the display of the 3D model in the model window; and e. an event timeline window for displaying the event timeline.
Claim 17
Dependent Claim
63 words
The multi-perspective viewer of claim 15, wherein the display means further controls the display of the following windows on the visual display: a. an audio control window; b. a bookmark bin window; c. a context-sensitive VCR control window; d. a time code window; e. a camera angle control window; f. a statistics and textual information viewing window; and g. a query processing window.
Claim 18
Dependent Claim
27 words
The multi-perspective viewer of claim 15, wherein the video clips displayed in the video window are encoded and compressed prior to being stored in the multi-media database.
Claim 19
Dependent Claim
107 words
The multi-perspective viewer of claim 15, wherein the user control window includes the following user control buttons: a. a swap control button for which allows the user to interchange the positions of the video window and the model window on the visual display; b. a freeze-frame control button which allows the user to freeze the real-time video clip displayed in the video window; c. a bookmarks control button for creating user bookmarks; d. a preferences pop-up control button; e. a zoom control button which allows the user to zoom in and out on an image displayed by the real-time video clip; and f. a hyperlink control button.
Claim 20
Dependent Claim
23 words
The multi-perspective viewer of claim 15, wherein the model window comprises a 2D representation of a 3D environmental model of the real-world environment.
Claim 21
Dependent Claim
56 words
The multi-perspective viewer of claim 19, wherein the 3D environmental model comprises a spatial-temporal database representing a sensed real-world environment, and wherein the spatial coordinates of the 3D model approximately correspond to the spatial coordinates of the sensed real-world environment, and wherein objects in the 2D model window are linked to objects in the 3D model.
Claim 22
Dependent Claim
32 words
The multi-perspective viewer of claim 15, wherein the model navigation control window includes context-sensitive control buttons that allow the user to control the display of selected multi-media events on the visual display.
Claim 23
Dependent Claim
33 words
The multi-perspective viewer of claim 21, wherein the context-sensitive control buttons include an object tracking button that allows the user to track the progress of a selected object throughout a selected multi-media event.
Claim 24
Dependent Claim
26 words
The multi-perspective viewer of claim 15 wherein the event timeline comprises a two-dimensional representation of the synchronized and linked multi-media data types in the multi-media database.
Claim 25
Dependent Claim
61 words
The multi-perspective viewer of claim 23, wherein the event timeline is synchronized and linked to the display means and thereby the display of every window on the visual display, and wherein the user selects a desired multi-media event by interacting with the event timeline and thereby causing the display means to correspondingly update each of the windows on the visual display.
Claim 26
Dependent Claim
17 words
The multi-perspective viewer of claim 24, wherein the event timeline changes in response to the query means.
Claim 27
Dependent Claim
22 words
The multi-perspective viewer of claim 16, wherein the query processing window includes a plurality of pop-up menus for formulating complex event queries.
Claim 28
Dependent Claim
23 words
The multi-perspective viewer of claim 16, wherein the audio control window includes controls for controlling volume, muting, mono/stereo, and selection of audio sources.
Claim 29
Dependent Claim
21 words
The multi-perspective viewer of claim 26, wherein the plurality of pop-up menus includes an object pop-up menu including an object profile.
Claim 30
Dependent Claim
31 words
The multi-perspective viewer of claim 1, wherein the user is a sports coach, and wherein the viewer displays information on the visual display in response to input provided by the coach.
Claim 31
Dependent Claim
33 words
The multi-perspective viewer of claim 16, wherein the context-sensitive VCR control window includes a smart fast forward and reverse control button for forwarding and reversing the display of subsequent and previous events, respectively.
Claim 32
Independent Claim
97 words
A method for interactively displaying a multi-media program derived from a real-world environment, the multi-media program comprising a plurality of multi-media events represented by associated and corresponding multiple multi-media data types stored in a multi-media database, the method comprising the steps of: a. accepting a query wherein the query specifies selected multi-media events; b. displaying an event timeline on a visual display wherein the event timeline graphically depicts events that satisfy the query; c. selecting an event from the event timeline; and d. displaying the multi-media data types associated with the selected event on the visual display.
Claim 33
Independent Claim
97 words
A method for interactively displaying a multi-media program derived from a real-world environment, the multi-media program comprising a plurality of multi-media events represented by associated and corresponding multiple multi-media data types stored in a multi-media database, the method comprising the steps of: a. accepting a query wherein the query specifies selected multi-media events; b. displaying an event timeline on a visual display wherein the event timeline graphically depicts events that satisfy the query; c. selecting an event from the event timeline; and d. displaying the multi-media data types associated with the selected event on the visual display.
Claim 34
Dependent Claim
26 words
The method of claim 31, further comprising the step of outputting audio, textual and other data associated with the multi-media data types displayed in step (d).
Claim 35
Independent Claim
169 words
A system for interactively viewing a multi-media program derived from a real-world environment, comprising: a. a computer environment including an input device and a visual display; b. a plurality of sensors connected to the computer environment and sensing the real-world environment; c. capturing means executed by the computer environment for capturing a plurality of multi-media events sensed by the sensors; d. filtering means executed by the computer environment for filtering the plurality of multi-media events captured by the capturing means, and for producing a multi-media program, wherein the multi-media program includes only those multi-media events that satisfy event selection criteria; e. an environmental model stored in the computer environment, wherein the environmental model represents the multi-media program, and wherein the spatial coordinates of the environmental model approximately correspond to the spatial coordinates of the sensed real-world environment; and f. a viewer for interactively accessing selected multi-media events and objects from the multi-media program, wherein the viewer includes an event timeline that graphically displays events stored in the computer environment.
Claim 36
Independent Claim
169 words
A system for interactively viewing a multi-media program derived from a real-world environment, comprising: a. a computer environment including an input device and a visual display; b. a plurality of sensors connected to the computer environment and sensing the real-world environment; c. capturing means executed by the computer environment for capturing a plurality of multi-media events sensed by the sensors; d. filtering means executed by the computer environment for filtering the plurality of multi-media events captured by the capturing means, and for producing a multi-media program, wherein the multi-media program includes only those multi-media events that satisfy event selection criteria; e. an environmental model stored in the computer environment, wherein the environmental model represents the multi-media program, and wherein the spatial coordinates of the environmental model approximately correspond to the spatial coordinates of the sensed real-world environment; and f. a viewer for interactively accessing selected multi-media events and objects from the multi-media program, wherein the viewer includes an event timeline that graphically displays events stored in the computer environment.
Claim 37
Dependent Claim
36 words
The system for interactively viewing a multi-media program as defined in claim 33, wherein the sensors comprise a single video camera and a plurality of sources capable of producing non-video information related to the multi-media program.
References Cited (Backward Citations)
US5109425A
Cited
US5237648A
Cited
US5170440A
Cited
US5259037A
Cited
US5574845A
Cited
US5821945A
Cited
US5708767A
Cited
US5729471A
Cited
US5805806A
Cited
US5884056A
Cited
US6038368A
Cited
US5832499A
Cited
US5752244A
Cited
US5818935A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-11-07 | 2004-05-08 | 2004-11-07 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-11-07 | 2008-05-08 | 2008-11-07 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-11-07 | 2012-05-08 | 2012-11-07 | $7700.00 | Window Closed / Lapsed |