Patent US6774917B1 successfully refreshed from Google Patents.
US6774917B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Methods and apparatuses for interactive similarity searching, retrieval, and browsing of video
Assignee: Fujifilm Business Innovation Corp
•
Filing: 1999-03-11
•
Granted: 2004-08-10
•
Expires: 2019-03-11
Abstract
Abstract Method for interactive selecting video consisting of training images from a video for a video similarity search and for displaying the results of the similarity search are disclosed. The user selects a time interval in the video as a query definition of training images for training an image class statistical model. Time intervals can be as short as one frame or consist of disjoint segments or shots. A statistical model of the image class defined by the training images is calculated on-the-fly from feature vectors extracted from transforms of the training images. For each frame in the video, a feature vector is extracted from the transform of the frame, and a similarity measure is calculated using the feature vector and the image class statistical model. The similarity measure is derived from the likelihood of a Gaussian model producing the frame. The similarity is then presented graphically, which allows the time structure of the video to be visualized and browsed. Similarity can be rapidly calculated for other video files as well, which enables content-based retrieval by example. A content-aware video browser featuring interactive similarity measurement is presented. A method for selecting training segments involves mouse click-and-drag operations over a time bar representing the duration of the video; similarity results are displayed as shades in the time bar. Another method involves selecting periodic frames of the video as endpoints for the training segment.
Inventors
- 👤 Jonathan T. Foote
- 👤 Andreas Girgensohn
- 👤 Lynn Wilcox
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 21:50
CPC Classifications
G; G06; G06V; G06V20/00; G06V20/40; G06V20/48; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7847; G06F16/7864; G06F16/73; G06F16/732; G06F16/7328; G06F16/738
Claims Analysis
Total Claims: 32
Claim 1
Independent Claim
117 words
A method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; training a statistical model using the reduced feature vectors; and displaying a similarity measure of each frame in the video to the training video segment; where interactively defining the training segment comprises: providing a display window for viewing the video; displaying a time bar within a video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; and receiving user training input indicating one or more training video segments from the video.
Claim 2
Independent Claim
117 words
A method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; training a statistical model using the reduced feature vectors; and displaying a similarity measure of each frame in the video to the training video segment; where interactively defining the training segment comprises: providing a display window for viewing the video; displaying a time bar within a video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; and receiving user training input indicating one or more training video segments from the video.
Claim 3
Dependent Claim
37 words
A method as in claim 1 , further comprising the steps of: for each frame of the video, obtaining a reduced feature vector; and computing a similarity score using the reduced feature vector and the statistical model.
Claim 4
Dependent Claim
25 words
A method as in claim 2 , further comprising the step of: segmenting the video into similar and non-similar segments based upon the similarity scores.
Claim 5
Dependent Claim
37 words
A method as in claim 3 , wherein the step of segmenting the video into similar and non-similar segments based upon the similarity scores is performed by comparing the similarity scores to an interactively defined similarity threshold.
Claim 6
Dependent Claim
54 words
A method as in claim 2 , wherein the steps of obtaining reduced feature vectors corresponding to frames of the training video segment and, for each frame of the video, obtaining a reduced feature vector are performed by retrieval of the reduced feature vectors from a precomputed feature vector database corresponding to the video.
Claim 7
Dependent Claim
42 words
A method as in claim 2 , wherein the steps of obtaining reduced feature vectors corresponding to frames of the training video segment and, for each frame of the video, obtaining a reduced feature vector are performed transforming frames of the video.
Claim 8
Dependent Claim
37 words
A method as in claim 1 , wherein each reduced feature vector corresponding to a frame of the training video segment includes features representing chromatic components of the frame and features representing luminance components of the frame.
Claim 9
Dependent Claim
23 words
A method as in claim 7 , wherein each reduced feature vector includes fewer features representing chromatic components than features representing luminance components.
Claim 10
Dependent Claim
44 words
A method as in claim 1 , wherein each reduced feature vector corresponding to a frame of the training video segment includes features representing red components of the frame, features representing green components of the frame, and features representing blue components of the frame.
Claim 11
Independent Claim
94 words
A method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 12
Independent Claim
94 words
A method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 13
Dependent Claim
25 words
A method as in claim 10 , wherein the step of receiving user training input comprises: receiving user training mouse input along the time bar.
Claim 14
Dependent Claim
75 words
A method as in claim 10 , further comprising the step of: receiving user threshold input indicating a threshold level for comparison with the similarity measure of each frame for labeling each frame as similar or non-similar to the one or more training video segments; wherein a first contrasting color or pattern in the time bar indicates one of similar or non-similar and a second contrasting color or pattern indicates another of similar or non-similar.
Claim 15
Dependent Claim
28 words
A method as in claim 12 , further comprising the step of: indexing each similar frame which follows a non-similar frame as a beginning of a similar segment.
Claim 16
Dependent Claim
26 words
A method as in claim 12 , wherein the step of receiving user threshold input comprises: receiving user threshold mouse input along a threshold slider bar.
Claim 17
Dependent Claim
29 words
A method as in claim 12 , wherein the first contrasting color or pattern is a black shade, and the second contrasting color or pattern is a white shade.
Claim 18
Independent Claim
70 words
A method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
Claim 19
Independent Claim
70 words
A method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
Claim 20
Dependent Claim
30 words
A method as in claim 16 , wherein the step of receiving user training input comprises: receiving user training mouse input by detecting mouse clicks on adjacently displayed periodic frames.
Claim 21
Dependent Claim
145 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; training a statistical model using the reduced feature vectors; and displaying a similarity measure of each frame in the video to the training video segment; wherein interactively defining the training segment comprises: providing a display window for viewing the video; displaying a time bar within a video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; and receiving user training input indicating one or more training video segments from the video.
Claim 22
Dependent Claim
145 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; training a statistical model using the reduced feature vectors; and displaying a similarity measure of each frame in the video to the training video segment; wherein interactively defining the training segment comprises: providing a display window for viewing the video; displaying a time bar within a video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; and receiving user training input indicating one or more training video segments from the video.
Claim 23
Dependent Claim
122 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 24
Dependent Claim
122 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 25
Dependent Claim
98 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
Claim 26
Dependent Claim
98 words
A computer readable storage medium, comprising: computer readable program code embodied on said computer readable storage medium, said computer readable program code for programming a computer to perform a method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
Claim 27
Dependent Claim
86 words
A computer system, comprising: a processor; a user interface; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; and training a statistical model using the reduced feature vectors.
Claim 28
Dependent Claim
86 words
A computer system, comprising: a processor; a user interface; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of performing a similarity search of a video, the method comprising the steps of: interactively defining a training video segment from the video; obtaining reduced feature vectors corresponding to frames of the training video segment; and training a statistical model using the reduced feature vectors.
Claim 29
Dependent Claim
135 words
A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 30
Dependent Claim
135 words
A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programming the computer system to perform a method of presenting a video within a video browser, comprising the steps of: providing a display window for viewing the video; displaying a time bar within the video browser, wherein position within the time bar linearly corresponds to elapsed time from a beginning of the video; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each frame in the video to the training video segment using shades of the time bar at positions corresponding to each frame to indicate the similarity measure.
Claim 31
Dependent Claim
111 words
A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programing the computer system to perform a method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
Claim 32
Dependent Claim
111 words
A computer system, comprising: a display; a user interface; a processor; and a processor readable storage medium having processor readable program code embodied on said processor readable storage medium, said processor readable program code for programing the computer system to perform a method of presenting a video within a web-based interface, comprising the steps of: displaying periodic frames of the video separated by a predetermined time interval; receiving user training input indicating one or more training video segments from the video; and displaying a similarity measure of each displayed periodic frame in the video to the training video segment using shades surrounding each displayed periodic frame to indicate the similarity measure.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-08-10 | 2008-02-09 | 2008-08-10 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-08-10 | 2012-02-09 | 2012-08-10 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-08-10 | 2016-02-09 | 2016-08-10 | $7700.00 | Window Closed / Lapsed |