US20040189691A1
Expired / Lapsed
US
⏱️ 0.0 years remaining
User interface for adaptive video fast forward
Assignee: SZ DJI Technology Co Ltd
•
Filing: 2003-03-28
•
Granted: —
•
Expires: 2023-03-28
Abstract
Abstract A user interface (UI) for adaptive video fast forward provides a novel fully adaptive content-based UI for allowing user interaction with an image sequence or video relative to a user identified query sample. This query sample is drawn either from an image sequence being searched or from another image sequence entirely. The user interaction offered by the UI includes providing a user with computationally efficient searching, browsing and retrieval of one or more objects, frames or sequences of interest in video or image sequences, as well as automatic content-based variable-speed playback based on a computed similarity to the query sample. In addition, the UI also provides the capability to search for image frames or sequences that are dissimilar to the query sample, thereby allowing the user to quickly locate unusual or different activity within an image sequence.
Inventors
- 👤 Nebojsa Jojic
- 👤 Nemanja Petrovic
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:19
CPC Classifications
G; G11; G11B; G11B27/00; G11B27/10; G11B27/34; G06; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/732; G06F16/7328; G06F16/74; G06F16/745; G11B27/005; G11B27/102; G11B27/105; G11B27/19; G11B27/28; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933
Claims Analysis
Total Claims: 50
Claim 1
Independent Claim
138 words
1 . A computer-readable medium having computer executable instructions for providing automatic variable-speed playback of an image sequence, said computer executable instructions comprising: selecting an input image sequence via a graphical user interface; providing a playback of the input image sequence within a first display window within the graphical user interface; selecting a comparison sample via the graphical user interface; providing a graphical representation of the comparison sample in a second display window within the graphical user interface; providing a graphical representation of one or more image frames from the input image sequence that match the comparison sample in a third display window within the graphical user interface; and automatically varying a playback speed of the input image sequence with respect to a probabilistic likelihood of each frame of the input image sequence relative to the comparison sample.
Claim 2
Dependent Claim
138 words
1 . A computer-readable medium having computer executable instructions for providing automatic variable-speed playback of an image sequence, said computer executable instructions comprising: selecting an input image sequence via a graphical user interface; providing a playback of the input image sequence within a first display window within the graphical user interface; selecting a comparison sample via the graphical user interface; providing a graphical representation of the comparison sample in a second display window within the graphical user interface; providing a graphical representation of one or more image frames from the input image sequence that match the comparison sample in a third display window within the graphical user interface; and automatically varying a playback speed of the input image sequence with respect to a probabilistic likelihood of each frame of the input image sequence relative to the comparison sample.
Claim 3
Dependent Claim
31 words
2 . The computer-readable medium of claim 1 further comprising an image sequence index window for displaying thumbnail representations of image frames extracted at regular intervals from the input image sequence.
Claim 4
Dependent Claim
65 words
3 . The computer-readable medium of claim 2 wherein the thumbnail representations of the input image sequence are selectable via the user interface, and wherein selection of a particular thumbnail representation of the input image sequence causes the input image sequence to be played within the first display window beginning at the image frame represented by the selected thumbnail representation of the input image sequence.
Claim 5
Dependent Claim
60 words
4 . The computer-readable medium of claim 1 wherein a determination of whether an image frame from the input image sequence matches the comparison sample comprises: automatically learning a generative model from the comparison sample; and comparing the frames in the input image sequence to the generative model to determine the probabilistic likelihood of each frame under the generative model.
Claim 6
Dependent Claim
30 words
5 . The computer-readable medium of claim 4 further comprising a control for selecting image frames having a high probabilistic likelihood under the generative model as matching the comparison sample.
Claim 7
Dependent Claim
30 words
6 . The computer-readable medium of claim 4 further comprising a control for selecting image frames having a low probabilistic likelihood under the generative model as matching the comparison sample.
Claim 8
Dependent Claim
28 words
7 . The computer-readable medium of claim 4 further comprising a user adjustable similarity threshold for comparing the frames in the input image sequence to the generative model.
Claim 9
Dependent Claim
27 words
8 . The computer-readable medium of claim 1 further comprising a playback speed slider bar for real-time adjustment of the playback speed of the input image sequence.
Claim 10
Dependent Claim
23 words
9 . The computer-readable medium of claim 8 wherein the playback speed slider bar indicates the playback speed of the input image sequence.
Claim 11
Dependent Claim
26 words
10 . The computer-readable medium of claim 1 further comprising a video position slider bar for indicating a current playback position of the input image sequence.
Claim 12
Dependent Claim
28 words
11 . The computer-readable medium of claim 10 wherein the video position slider bar provides for real-time adjustment of the current playback position of the input image sequence.
Claim 13
Dependent Claim
30 words
12 . The computer-readable medium of claim 1 further comprising a control for extracting a subset of consecutive image frames from the input image sequence to form the comparison sample.
Claim 14
Dependent Claim
25 words
13 . The computer-readable medium of claim 1 further comprising a control for loading the comparison sample from at least one previously stored comparison sample.
Claim 15
Dependent Claim
20 words
14 . The computer-readable medium of claim 13 wherein the previously stored comparison sample includes a previously learned generative model.
Claim 16
Dependent Claim
31 words
15 . The computer-readable medium of claim 1 wherein selecting an input image sequence via the graphical user interface comprises automatically importing the input image sequence from at least one camera.
Claim 17
Dependent Claim
32 words
16 . The computer-readable medium of claim 1 wherein selecting an input image sequence via the graphical user interface comprises automatically loading the input image sequence from a computer readable storage medium.
Claim 18
Dependent Claim
32 words
17 . The computer-readable medium of claim 1 wherein providing the graphical representation of the comparison sample in the second display window comprises displaying a representative image frame from the comparison sample.
Claim 19
Dependent Claim
35 words
18 . The computer-readable medium of claim 1 wherein providing the graphical representation of the comparison sample in the second display window comprises displaying a looped playback of the image frames comprising the comparison sample.
Claim 20
Dependent Claim
35 words
19 . The computer-readable medium of claim 1 wherein the graphical representation of one or more image frames from the input image sequence that match the comparison sample are provided as user selectable thumbnail representations.
Claim 21
Dependent Claim
42 words
20 . The computer-readable medium of claim 19 wherein selection of any of the user selectable thumbnail representations causes playback of the one or more image frames from the input image sequence that match the comparison sample within the first display window.
Claim 22
Dependent Claim
37 words
21 . The computer-readable medium of claim 1 wherein automatically varying the playback speed of the input image sequence comprises increasing playback speed as the probabilistic likelihood decreases and decreasing payback speed as the probabilistic likelihood increases.
Claim 23
Dependent Claim
147 words
22 . A system for providing automatic fully adaptive content-based variable-speed playback of a video, comprising: providing a graphical user interface; selecting a video via the graphical user interface; selecting a number of consecutive image frames from the video as a query sample via the graphical user interface; automatically learning a generative model from the query sample; displaying the video in a video window within the graphical user interface; displaying the query sample in a query sample window within the graphical user interface; matching portions of the video to the query sample by determining a probabilistic likelihood of each image frame of the video under the generative model; displaying the matched portions of the video as thumbnail representations within a query match window within the graphical user interface; and automatically varying a playback speed of the video in inverse proportion to the probabilistic likelihood of each frame.
Claim 24
Dependent Claim
147 words
22 . A system for providing automatic fully adaptive content-based variable-speed playback of a video, comprising: providing a graphical user interface; selecting a video via the graphical user interface; selecting a number of consecutive image frames from the video as a query sample via the graphical user interface; automatically learning a generative model from the query sample; displaying the video in a video window within the graphical user interface; displaying the query sample in a query sample window within the graphical user interface; matching portions of the video to the query sample by determining a probabilistic likelihood of each image frame of the video under the generative model; displaying the matched portions of the video as thumbnail representations within a query match window within the graphical user interface; and automatically varying a playback speed of the video in inverse proportion to the probabilistic likelihood of each frame.
Claim 25
Dependent Claim
29 words
23 . The system of claim 22 wherein the playback speed of the video decreases as the likelihood increases, and wherein the payback speed increases as the likelihood decreases.
Claim 26
Dependent Claim
29 words
24 . The system of claim 23 wherein the playback speed the video is determined by a combined likelihood of one or more adjacent image frames in the video.
Claim 27
Dependent Claim
30 words
25 . The system of claim 22 wherein the generative model is based on a user definable number of color blobs for modeling multiple objects represented in the query sample.
Claim 28
Dependent Claim
17 words
26 . The system of claim 22 wherein the generative model is capable of modeling occluding objects.
Claim 29
Dependent Claim
23 words
27 . The system of claim 22 further comprising comparing the likelihood of each frame under the generative model to a similarity threshold.
Claim 30
Dependent Claim
52 words
28 . The system of claim 27 wherein image frames that have a likelihood greater than the similarity threshold are identified as being similar to the query sample, and wherein image frames that do not have a likelihood greater than the similarity threshold are identified as being dissimilar to the query sample.
Claim 31
Dependent Claim
24 words
29 . The system of claim 27 further comprising a control for allowing user adjustment of the similarity threshold via the graphical user interface.
Claim 32
Dependent Claim
28 words
30 . The system of claim 22 further comprising an image sequence index window for displaying thumbnail representations of image frames extracted at regular intervals from the video.
Claim 33
Dependent Claim
46 words
31 . The system of claim 30 wherein selection of any of the thumbnail representations of the video via the graphical user interface causes the input image sequence to be played within the video window beginning at the image frame represented by the selected thumbnail representation.
Claim 34
Dependent Claim
24 words
32 . The system of claim 22 further comprising a playback speed slider bar for real-time adjustment of the playback speed of the video.
Claim 35
Dependent Claim
25 words
33 . The system of claim 22 further comprising a video position slider bar for real-time adjustment of the current playback position of the video.
Claim 36
Dependent Claim
27 words
34 . The system of claim 22 further comprising a control for loading the query sample from a set of at least one previously stored query samples.
Claim 37
Dependent Claim
27 words
35 . The system of claim 22 further comprising a control for loading the generative model from a set of at least one previously computed generative models.
Claim 38
Dependent Claim
21 words
36 . The system of claim 22 further comprising a control for saving the learned generative model to a computer-readable medium.
Claim 39
Dependent Claim
157 words
37 . A computer-implemented process for using a graphical user interface for automatically identifying similar image frames in one or more image sequences, comprising: use a graphical user interface to select at least one image sequence, each image sequence having at least one image frame; use the graphical user interface to select a query sample consisting of at least one image frame from one of the at least one image sequences; use the graphical user interface to input a desired number of blobs to be modeled in learning a generative model of the query sample; automatically learn the generative model from the query sample; compare the frames in each image sequence to the generative model to determine a likelihood of each frame under the generative model; and provide an automatic variable speed playback of at least one of the image sequences within the graphical user interface relative to the likelihood of each frame under the generative model.
Claim 40
Dependent Claim
157 words
37 . A computer-implemented process for using a graphical user interface for automatically identifying similar image frames in one or more image sequences, comprising: use a graphical user interface to select at least one image sequence, each image sequence having at least one image frame; use the graphical user interface to select a query sample consisting of at least one image frame from one of the at least one image sequences; use the graphical user interface to input a desired number of blobs to be modeled in learning a generative model of the query sample; automatically learn the generative model from the query sample; compare the frames in each image sequence to the generative model to determine a likelihood of each frame under the generative model; and provide an automatic variable speed playback of at least one of the image sequences within the graphical user interface relative to the likelihood of each frame under the generative model.
Claim 41
Dependent Claim
25 words
38 . The computer-implemented process of claim 37 wherein the likelihood of each frame under the generative model is compared to an adjustable similarity threshold.
Claim 42
Dependent Claim
52 words
39 . The computer-implemented process of claim 38 wherein image frames that have a likelihood greater than the similarity threshold are similar to the at least one image frame of the query sample, and wherein all other image frames are dissimilar to the at least one image frame of the query sample.
Claim 43
Dependent Claim
28 words
40 . The computer-implemented process of claim 37 wherein the playback speed the at least one image sequence is inversely proportional to the likelihood of each image frame.
Claim 44
Dependent Claim
27 words
41 . The computer-implemented process of claim 40 wherein the playback speed decreases as the likelihood increases, and wherein the payback speed increases as the likelihood decreases.
Claim 45
Dependent Claim
36 words
42 . The computer-implemented process of claim 37 wherein the generative model models multiple objects in the query sample as blobs, and wherein the generative model is capable of modeling occluding objects in the query sample.
Claim 46
Dependent Claim
42 words
43 . The computer-implemented process of claim 37 wherein automatically learning the generative model from the query sample comprises performing an variational expectation-maximization (EM) analysis of each image frame in the query sample for learning a probabilistic explanation of the query sample.
Claim 47
Dependent Claim
33 words
44 . The computer-implemented process of claim 37 further comprising an image sequence index window for displaying thumbnail representations of image frames extracted at regular intervals from the at least one image sequence.
Claim 48
Dependent Claim
54 words
45 . The computer-implemented process of claim 44 wherein selection of any of the thumbnail representations of the at least one image sequence via the graphical user interface causes the at least one image sequence to be played within the graphical user interface beginning at the image frame represented by the selected thumbnail representation.
Claim 49
Dependent Claim
29 words
46 . The computer-implemented process of claim 37 further comprising a playback speed slider bar for real-time adjustment of the playback speed of the at least one image sequence.
Claim 50
Dependent Claim
30 words
47 . The computer-implemented process of claim 37 further comprising a video position slider bar for real-time adjustment of a current playback position of the at least one image sequence.
References Cited (Backward Citations)
US5537528A
Cited
US5802361A
Cited
US6282549B1
Cited
US6636220B1
Cited
US20020002550A1
Cited
US20030093790A1
Cited
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.