US7035435B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Scalable video summarization and navigation system and method
Assignee: Hewlett Packard Development Co LP
•
Filing: 2002-05-07
•
Granted: 2006-04-25
•
Expires: 2022-05-07
Abstract
Abstract A method and system for automatically summarizing a video document. The video document is decomposed into scenes, shots and frames, and an importance value is assigned to each scene, shot and frame. Keyframes are allocated among the shots based on the importance value of each shot. The allocated number of keyframes are then selected from each shot. The number of keyframes may be altered for greater or lesser detail in response to user input.
Inventors
- 👤 Ying Li
- 👤 Tong Zhang
- 👤 Daniel R. Tretter
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:34
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7847; G06F16/786; G06F16/73; G06F16/738; G06F16/739; G06F16/7837; G06F16/784; Y; Y10; Y10S; Y10S707/00; Y10S707/99941; Y10S707/99944; Y10S707/99945; Y10S707/99948
Claims Analysis
Total Claims: 34
Claim 1
Independent Claim
67 words
A method for automatically summarizing a video document comprising the steps of: decomposing the video document into scenes, shots and frames; assigning an importance value to each scene, shot and frame; allocating a number of keyframes among the shots based on the importance value of each shot; selecting the allocated number of keyframes from each shot; and altering the number of keyframes in response to user input.
Claim 2
Independent Claim
67 words
A method for automatically summarizing a video document comprising the steps of: decomposing the video document into scenes, shots and frames; assigning an importance value to each scene, shot and frame; allocating a number of keyframes among the shots based on the importance value of each shot; selecting the allocated number of keyframes from each shot; and altering the number of keyframes in response to user input.
Claim 3
Dependent Claim
21 words
The method of claim 1 , wherein the number of keyframes allocated among the shots is a predetermined number of keyframes.
Claim 4
Dependent Claim
33 words
The method of claim 2 , wherein altering the number of keyframes in response to user input includes the step of removing one or more keyframes from the predetermined number of selected keyframes.
Claim 5
Dependent Claim
33 words
The method of claim 3 , wherein removing one or more keyframes from the predetermined number of selected keyframes includes the step of removing the least important keyframe in the least important shot.
Claim 6
Dependent Claim
33 words
The method of claim 2 , wherein altering the number of keyframes in response to user input includes the step of adding one or more keyframes to the predetermined number of selected keyframes.
Claim 7
Dependent Claim
50 words
The method of claim 1 , wherein assigning an importance value to each scene, shot and frame includes the steps of defining a scene importance algorithm, a shot importance algorithm and a frame importance algorithm and computing the importance value of the scenes, shots and frames, respectively, with the algorithms.
Claim 8
Dependent Claim
38 words
The method of claim 6 , wherein computing a scene importance value includes the steps of measuring a length of the scene, quantifying the activity level of the scene, and counting the number of shots in the scene.
Claim 9
Dependent Claim
22 words
The method of claim 6 , wherein computing a scene importance value includes the step of measuring a length of the scene.
Claim 10
Dependent Claim
25 words
The method of claim 8 , wherein measuring a length of the scene includes the step of counting the number of frames in the scene.
Claim 11
Dependent Claim
23 words
The method of claim 6 , wherein computing a scene importance value includes the step of quantifying the activity level of the scene.
Claim 12
Dependent Claim
41 words
The method of claim 10 , wherein quantifying the activity level of the scene includes the steps of computing a color histogram difference for each consecutive frame pair within the scene and using their average to indicate the scene activity level.
Claim 13
Dependent Claim
24 words
The method of claim 6 , wherein computing a scene importance value includes the step of counting the number of shots in the scene.
Claim 14
Dependent Claim
36 words
The method of claim 6 , wherein computing a shot importance value includes the steps of measuring a length of the shot, quantifying the activity level within the shot, and quantifying camera motion within the shot.
Claim 15
Dependent Claim
22 words
The method of claim 6 , wherein computing a shot importance value includes the step of measuring a length of the shot.
Claim 16
Dependent Claim
25 words
The method of claim 14 , wherein measuring a length of the shot includes the step of counting the number of frames in the shot.
Claim 17
Dependent Claim
23 words
The method of claim 6 , wherein computing a shot importance value includes the step of quantifying the activity level within the shot.
Claim 18
Dependent Claim
41 words
The method of claim 16 , wherein quantifying the activity level within the shot includes the steps of computing the amount of motion between every referenced frame pair within the shot and using their average to indicate the shot activity level.
Claim 19
Dependent Claim
22 words
The method of claim 6 , wherein computing a shot importance value includes the step of quantifying camera motion within the shot.
Claim 20
Dependent Claim
52 words
The method of claim 18 , wherein quantifying camera motion within the shot includes the steps of categorizing the direction motion vectors of each frame into a discrete number of directions and computing a motion vector histogram, and computing a directional motion ratio and an average magnitude of the directional motion vectors.
Claim 21
Dependent Claim
36 words
The method of claim 6 , wherein computing a frame importance value includes the steps of determining the percentage of skin-colored pixels in the frame, and counting the number of detected human faces in the frame.
Claim 22
Dependent Claim
25 words
The method of claim 20 , wherein computing a frame importance value further includes the step of calculating the edge energy distribution of the frame.
Claim 23
Dependent Claim
27 words
The method of claim 20 , wherein computing a frame importance value further includes the step of quantifying the amount of motion activity contained in the frame.
Claim 24
Dependent Claim
28 words
The method of claim 1 , wherein selecting the allocated number of keyframes from each shot includes the step of selecting the keyframes with the highest importance value.
Claim 25
Dependent Claim
24 words
The method of claim 1 , wherein selecting the allocated number of keyframes from each shot comprises selecting the keyframes using time-constrained keyframe selection.
Claim 26
Dependent Claim
24 words
The method of claim 1 , wherein selecting the allocated number of keyframes from each shot comprises selecting the keyframes using importance-adapted keyframe selection.
Claim 27
Dependent Claim
48 words
The method of claim 1 , wherein allocating a number of keyframes among the shots includes the steps of assigning the keyframes among the scenes based on the importance value of the scenes, and within each scene, distributing the assigned keyframes to the component shots of each scene.
Claim 28
Dependent Claim
19 words
The method of claim 1 , further comprising the step of displaying the selected keyframes on a display device.
Claim 29
Dependent Claim
95 words
A computer readable medium containing instructions for controlling a computer system to perform a method for summarizing a video document comprising the steps of: decomposing the video document into scenes, shots and frames; assigning an importance value to each scene, shot and frame; allocating a number of keyframes among the scenes based on the importance value of each scene; allocating a number of keyframes among the shots based on the importance value of each shot; selecting the allocated number of keyframes from each shot; and altering the number of keyframes in response to user input.
Claim 30
Dependent Claim
95 words
A computer readable medium containing instructions for controlling a computer system to perform a method for summarizing a video document comprising the steps of: decomposing the video document into scenes, shots and frames; assigning an importance value to each scene, shot and frame; allocating a number of keyframes among the scenes based on the importance value of each scene; allocating a number of keyframes among the shots based on the importance value of each shot; selecting the allocated number of keyframes from each shot; and altering the number of keyframes in response to user input.
Claim 31
Independent Claim
78 words
A method for summarizing and navigating a video document comprising the steps of: decomposing the video document into a hierarchal tree of scenes, shots and frames; assigning an importance value to each scene, shot and frame; allocating a predetermined number of keyframes among the scenes; assigning the allocated keyframes in each scene among the shots of each scene; selecting the assigned number of keyframes from each shot; and altering the number of keyframes in response to user input.
Claim 32
Independent Claim
78 words
A method for summarizing and navigating a video document comprising the steps of: decomposing the video document into a hierarchal tree of scenes, shots and frames; assigning an importance value to each scene, shot and frame; allocating a predetermined number of keyframes among the scenes; assigning the allocated keyframes in each scene among the shots of each scene; selecting the assigned number of keyframes from each shot; and altering the number of keyframes in response to user input.
Claim 33
Dependent Claim
33 words
The method of claim 29 , wherein altering the number of keyframes in response to user input includes the step of removing one or more keyframes from the predetermined number of selected keyframes.
Claim 34
Dependent Claim
33 words
The method of claim 29 , wherein altering the number of keyframes in response to user input includes the step of adding one or more keyframes to the predetermined number of selected keyframes.
References Cited (Backward Citations)
US5635982A
Cited
US5708767A
Cited
US5933549A
Cited
US6738100B2
Cited
US6340971B1
Cited
US5995095A
Cited
US6320669B1
Cited
US6252975B1
Cited
US6342904B1
Cited
US6535639B1
Cited
EP1045316A2
Cited
US20040170321A1
Cited
US6549643B1
Cited
US20040125877A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-04-24 | 2009-10-24 | 2010-04-25 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-04-24 | 2013-10-24 | 2014-04-25 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-04-24 | 2017-10-24 | 2018-04-25 | $7700.00 | Window Closed / Lapsed |