Patent US7555718B2 successfully refreshed from Google Patents.
US7555718B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for presenting video search results
Assignee: Fujifilm Business Innovation Corp
•
Filing: 2004-11-12
•
Granted: 2009-06-30
•
Expires: 2024-11-12
Abstract
Abstract The invention displays video search results in a form that makes it easy for users to determine which results are truly relevant. Each story returned as a search result is visualized as a collage of keyframes from the story's shots. The selected keyframes and their sizes depend on the corresponding shots' respective relevance. Shot relevance depends on the search retrieval score of the shot and, in some embodiments, also depends on the search retrieval score of the shot's parent story. Once areas have been determined, the keyframes are scaled and/or cropped to fit into the area. In one embodiment, users can mark one or more shots as being relevant to the search. In one embodiment, a timeline is created and displayed with one or more neighbor stories that are each part of the video and which are closest in time of creation to the selected story.
Inventors
- 👤 Andreas Girgensohn
- 👤 John E. Adcock
- 👤 Lynn D. Wilcox
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:14
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7847; G06F16/785; G06F16/73; G06F16/732; G06F16/7328; G06F16/738; G06F16/74; G06F16/745; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933
Claims Analysis
Total Claims: 33
Claim 1
Independent Claim
138 words
A method for presenting video search results, including one or more videos, comprising the following steps: a) receiving a set of video search results for the one or more videos, wherein the video search results comprise one or more stories for each of the one or more videos; b) selecting from each story a set of shots; c) selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the collage indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and d) creating for each story a collage comprising the keyframes, wherein the collage can be used to present the video search results.
Claim 2
Independent Claim
138 words
A method for presenting video search results, including one or more videos, comprising the following steps: a) receiving a set of video search results for the one or more videos, wherein the video search results comprise one or more stories for each of the one or more videos; b) selecting from each story a set of shots; c) selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the collage indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and d) creating for each story a collage comprising the keyframes, wherein the collage can be used to present the video search results.
Claim 3
Dependent Claim
38 words
The method of claim 1 , further comprising the following steps: a) creating a panel displaying candidate keyframes from each story; and b) allowing a user to select keyframes to be included in the collage by marking them.
Claim 4
Dependent Claim
25 words
The method of claim 2 , wherein the selected keyframes are indicated by an overlay employing one of an inclusion-indicating color and an inclusion-indicating pattern.
Claim 5
Dependent Claim
28 words
The method of claim 3 , wherein one of the inclusion-indicating color and the inclusion-indicating pattern of a keyframe indicates relevance of the shot corresponding to the keyframe.
Claim 6
Dependent Claim
33 words
The method of claim 3 , wherein one of the inclusion-indicating color of a keyframe and the inclusion-indicating pattern of a keyframe indicates that the shot corresponding to the keyframe was previously visited.
Claim 7
Dependent Claim
12 words
The method of claim 2 , wherein the selected keyframes are displayed.
Claim 8
Dependent Claim
17 words
The method of claim 2 , wherein the selected keyframes are indicated by an inclusion-indicating color overlay.
Claim 9
Dependent Claim
38 words
The method of claim 1 , further comprising the following steps: a) creating a panel displaying candidate keyframes from each story; and b) allowing a user to select keyframes to be excluded from the collage by marking them.
Claim 10
Dependent Claim
25 words
The method of claim 8 , wherein the excluded keyframes are indicated by an overlay employing one of an exclusion-indicating color and an exclusion-indicating pattern.
Claim 11
Dependent Claim
19 words
The method of claim 1 , wherein the keyframes can be selected by a user to provide further information.
Claim 12
Dependent Claim
33 words
The method of claim 1 , wherein the relevance is computed as the average of the search retrieval score of the shot and the search retrieval score of the story comprising the shot.
Claim 13
Dependent Claim
28 words
The method of claim 1 , wherein the area of each keyframe in the collage is proportional to the relevance to the video search of the corresponding shot.
Claim 14
Dependent Claim
11 words
The method of claim 1 , wherein the keyframes are cropped.
Claim 15
Dependent Claim
16 words
The method of claim 13 , wherein the keyframes are cropped according to an aspect ratio.
Claim 16
Dependent Claim
18 words
The method of claim 13 , wherein regions of interest are used as a basis for the cropping.
Claim 17
Dependent Claim
11 words
The method of claim 1 , wherein the keyframes are scaled.
Claim 18
Dependent Claim
13 words
The method of claim 1 , wherein the keyframes are scaled and cropped.
Claim 19
Dependent Claim
21 words
The method of claim 1 , wherein the number of keyframes comprised in the set of shots keyframes is approximately four.
Claim 20
Dependent Claim
41 words
The method of claim 1 , comprising the additional step of: d) creating a timeline displayed with one or more neighbor stories which are each comprised in the video and which are closest in time of creation to a selected story.
Claim 21
Dependent Claim
42 words
The method of claim 19 , comprising the additional steps of: e) for each of the one or more neighbor stories, selecting a set of corresponding neighbor keyframes; and f) for each neighbor story, creating a neighbor collage comprising the neighbor keyframes.
Claim 22
Dependent Claim
24 words
The method of claim 20 , wherein the collage corresponding to a selected story is displayed in a larger format than the neighbor collages.
Claim 23
Dependent Claim
21 words
The method of claim 1 , in which a tool tip is displayed comprising information about one or more selected stories.
Claim 24
Dependent Claim
25 words
The method of claim 22 , wherein the tool tip displays one or more keywords that best distinguish a selected story from the other stories.
Claim 25
Dependent Claim
11 words
The method of claim 23 , wherein the keywords are emphasized.
Claim 26
Dependent Claim
160 words
In a computer, a method for presenting video search results comprising one or more stories, comprising the following steps: a) initiating, by a user of the computer, a video search capable of generating a set of video search results comprising one or more stories; b) receiving, by the user, a set of shots selected by the computer from each story; c) receiving, by the user, one or more representative keyframes selected by the computer from each shot, wherein an area of each keyframe in the story indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; d) receiving, by the user, for each story, a collage created by the computer comprising the keyframes; and e) selecting, by the user, one or more of the stories for display of the corresponding collages.
Claim 27
Dependent Claim
160 words
In a computer, a method for presenting video search results comprising one or more stories, comprising the following steps: a) initiating, by a user of the computer, a video search capable of generating a set of video search results comprising one or more stories; b) receiving, by the user, a set of shots selected by the computer from each story; c) receiving, by the user, one or more representative keyframes selected by the computer from each shot, wherein an area of each keyframe in the story indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; d) receiving, by the user, for each story, a collage created by the computer comprising the keyframes; and e) selecting, by the user, one or more of the stories for display of the corresponding collages.
Claim 28
Independent Claim
181 words
A system for detecting useful images and for ranking images in order of usefulness, comprising: a) one or more processors capable of, given a set of video search results comprising one or more stories, selecting from each story a set of shots; capable of selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the story indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and capable of creating for each story a collage comprising the keyframes; and b) a machine-readable storage medium, including operations stored thereon that, when processed by one or more processors, cause the system to perform the steps of: 1) given a set of video search results comprising one or more stories, selecting from each story a set of shots; 2) selecting from each shot one or more representative keyframes, and 3) creating for each story a collage comprising the keyframes.
Claim 29
Independent Claim
181 words
A system for detecting useful images and for ranking images in order of usefulness, comprising: a) one or more processors capable of, given a set of video search results comprising one or more stories, selecting from each story a set of shots; capable of selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the story indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and capable of creating for each story a collage comprising the keyframes; and b) a machine-readable storage medium, including operations stored thereon that, when processed by one or more processors, cause the system to perform the steps of: 1) given a set of video search results comprising one or more stories, selecting from each story a set of shots; 2) selecting from each shot one or more representative keyframes, and 3) creating for each story a collage comprising the keyframes.
Claim 30
Dependent Claim
114 words
A machine-readable storage medium, including operations stored thereon that, when processed by one or more processors, causes a system to perform the steps of: a) given a set of video search results comprising one or more stories, selecting from each story a set of shots; b) selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the story indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and c) creating for each story a collage comprising the keyframes.
Claim 31
Dependent Claim
114 words
A machine-readable storage medium, including operations stored thereon that, when processed by one or more processors, causes a system to perform the steps of: a) given a set of video search results comprising one or more stories, selecting from each story a set of shots; b) selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the story indicates a relevance of the video search results to the shot selection, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and c) creating for each story a collage comprising the keyframes.
Claim 32
Independent Claim
112 words
An apparatus for detecting useful images and for ranking images in order of usefulness, comprising: a) given a set of video search results comprising one or more stories, means for selecting from each story a set of shots; b) means for selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the story indicates relevance to the video search of the shot corresponding to the keyframe, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and c) means for creating for each story a collage comprising the keyframes.
Claim 33
Independent Claim
112 words
An apparatus for detecting useful images and for ranking images in order of usefulness, comprising: a) given a set of video search results comprising one or more stories, means for selecting from each story a set of shots; b) means for selecting from each shot one or more representative keyframes, wherein an area of each keyframe in the story indicates relevance to the video search of the shot corresponding to the keyframe, wherein the relevance is determined by a combination of a search retrieval score of the shot and a search retrieval score of the story comprising the shot; and c) means for creating for each story a collage comprising the keyframes.
References Cited (Backward Citations)
US6366303B1
Cited
US5821945A
Cited
US5903892A
Cited
US6957386B2
Cited
US6363380B1
Cited
US7263659B2
Cited
US6535639B1
Cited
US20030061612A1
Cited
US20030095720A1
Cited
US20030189588A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2012-06-29 | 2012-12-29 | 2013-06-30 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2016-06-29 | 2016-12-29 | 2017-06-30 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2020-06-29 | 2020-12-29 | 2021-06-30 | $7700.00 | Window Closed / Lapsed |