Patent US7212666B2 successfully refreshed from Google Patents.
US7212666B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Generating visually representative video thumbnails
Assignee: SZ DJI Technology Co Ltd
•
Filing: 2003-04-01
•
Granted: 2007-05-01
•
Expires: 2023-04-01
Abstract
Abstract An algorithm identifies a salient video frame from a video sequence for use as a video thumbnail. The identification of a video thumbnail is based on a frame goodness measure. The algorithm calculates a color histogram of a frame, and then calculates the entropy and standard deviation of the color histogram. The frame goodness measure is a weighted combination of the entropy and the standard deviation. A video frame having the highest value of frame goodness measure for a video sequence is determined as the video thumbnail for a video sequence.
Inventors
- 👤 Dong Zhang
- 👤 Yijin Wang
- 👤 Hong-Jiang Zhang
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:37
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/738; G06F16/739; E; E03; E03D; E03D9/00; E03D9/08; G06F18/00; G06F18/20; G06F18/28; G06V; G06V10/00; G06V10/70; G06V10/77; G06V10/772; G06V20/00; G06V20/40; G11; G11B; G11B27/00; G11B27/10; G11B27/102; G11B27/19; G11B27/28
Claims Analysis
Total Claims: 53
Claim 1
Independent Claim
65 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor per for performing operations comprising: receiving a video sequence; and calculating frame goodness measures for frames within the video sequence from histogram entropies and histogram standard deviations; and presenting a visually representative thumbnail based on the frame goodness measures for a user to browse video data.
Claim 2
Dependent Claim
65 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor per for performing operations comprising: receiving a video sequence; and calculating frame goodness measures for frames within the video sequence from histogram entropies and histogram standard deviations; and presenting a visually representative thumbnail based on the frame goodness measures for a user to browse video data.
Claim 3
Dependent Claim
23 words
The computer-readable medium of claim 1 , comprising further processor-executable instructions configured for determining a maximum goodness measure from the frame goodness measures.
Claim 4
Dependent Claim
29 words
The computer-readable medium of claim 2 , wherein the computer-program instructions further comprise instructions for generating a video thumbnail from a frame that corresponds with the maximum goodness measure.
Claim 5
Dependent Claim
25 words
The computer-readable medium of claim 1 , wherein each frame goodness measure is a weighted combination of a histogram entropy and a standard deviation entropy.
Claim 6
Dependent Claim
19 words
The computer-readable medium of claim 1 , wherein the computer-program instructions further comprise instructions for calculating each histogram entropy.
Claim 7
Dependent Claim
23 words
The computer-readable medium of claim 5 , wherein the computer-program instructions for calculating each histogram entropy comprise instructions for calculating a color histogram.
Claim 8
Dependent Claim
22 words
A processor-readable medium as recited in claim 1 , wherein the computer-program instructions further comprise instructions for calculating each histogram standard deviation.
Claim 9
Dependent Claim
25 words
The computer-readable medium of claim 7 , wherein the computer-program instructions for calculating each histogram standard deviation further comprise instructions for calculating a color histogram.
Claim 10
Dependent Claim
57 words
The computer-readable medium of claim 1 , wherein the computer-program instructions for calculating a frame goodness measure comprise instructions for: subdividing a frame into a plurality of blocks; calculating a block goodness measure for each block; determining a minimum block goodness measure; and selecting the minimum block goodness measure as a frame goodness measure for the frame.
Claim 11
Dependent Claim
55 words
The computer-readable medium of claim 1 , wherein the computer-program instructions for calculating a frame goodness measure comprise instructions for: calculating a color histogram; calculating an entropy of the color histogram; calculating a standard deviation of the color histogram; and calculating the goodness measure as a weighted combination of the entropy and the standard deviation.
Claim 12
Dependent Claim
80 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising: temporally segmenting a video sequence; calculating frame goodness measures within a particular temporal segment using histogram entropies and histogram standard deviations; determining from the frame goodness measures, a local maximum goodness measure for the particular temporal segment; and presenting a visually representative thumbnail based on the local maximum goodness measure for a user to browse video data.
Claim 13
Dependent Claim
80 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising: temporally segmenting a video sequence; calculating frame goodness measures within a particular temporal segment using histogram entropies and histogram standard deviations; determining from the frame goodness measures, a local maximum goodness measure for the particular temporal segment; and presenting a visually representative thumbnail based on the local maximum goodness measure for a user to browse video data.
Claim 14
Dependent Claim
29 words
The computer-readable medium of claim 11 , wherein the computer-program instructions further comprise instructions for generating a video thumbnail from a frame associated with the local maximum goodness measure.
Claim 15
Dependent Claim
31 words
The computer-readable medium of claim 11 wherein a frame goodness measure is calculated for every nth frame within the particular temporal segment and n is equal to or greater than 1.
Claim 16
Dependent Claim
24 words
The computer-readable medium of claim 11 , wherein each goodness measure is a weighted combination of a histogram entropy and a standard deviation entropy.
Claim 17
Dependent Claim
40 words
The computer-readable medium of claim 11 , wherein the particular temporal segment is selected from: a beginning temporal segment of the video sequence; a middle temporal segment of the video sequence; and an ending temporal segment of the video sequence.
Claim 18
Dependent Claim
58 words
The computer-readable medium of claim 11 , wherein the computer-program instructions for calculating a frame goodness measure further comprise instructions for: subdividing a frame into a plurality of blocks; calculating a block goodness measure for each block; determining a minimum block goodness measure; and selecting the minimum block goodness measure as a frame goodness measure for the frame.
Claim 19
Dependent Claim
96 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising: segmenting a video sequence into a plurality of video shots; calculating a color histogram of a video frame in a first video shot; calculating an entropy of the color histogram; calculating a standard deviation of the color histogram; calculating a frame goodness measure as a weighted combination of the entropy and the standard deviation; and presenting a visually representative thumbnail based on the frame goodness measure for a user to browse video data.
Claim 20
Dependent Claim
96 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising: segmenting a video sequence into a plurality of video shots; calculating a color histogram of a video frame in a first video shot; calculating an entropy of the color histogram; calculating a standard deviation of the color histogram; calculating a frame goodness measure as a weighted combination of the entropy and the standard deviation; and presenting a visually representative thumbnail based on the frame goodness measure for a user to browse video data.
Claim 21
Dependent Claim
47 words
The computer-readable medium of claim 17 , wherein the computer-program instructions further comprise instructions for: calculating additional frame goodness measures for additional video frames in the first video shot; and determining a local maximum frame goodness measure for the first video shot by comparing frame goodness measures.
Claim 22
Dependent Claim
47 words
The computer-readable medium of claim 18 , wherein the computer-program instructions further comprise instructions for: calculating additional local maximum frame goodness measures for additional video shots in the video sequence; and determining a thumbnail video frame for the video sequence by comparing local maximum frame goodness measures.
Claim 23
Dependent Claim
46 words
The computer-readable medium of claim 19 , wherein the computer-program instructions for determining further comprise instructions for: identifying a highest valued local maximum frame goodness measure; and selecting a video frame corresponding to the highest valued local maximum frame goodness measure as the thumbnail video frame.
Claim 24
Dependent Claim
97 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising: segmenting a video sequence into a plurality of shots; selecting a frame from each of the plurality of shots; for each selected frame, calculating a frame goodness measure as a function of a histogram entropy and a histogram standard deviation; determining a thumbnail frame from the selected frames as a function of the frame goodness measures; and wherein the thumbnail frame is for presentation to a user to facilitate user browsing of video data.
Claim 25
Dependent Claim
97 words
A tangible computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for performing operations comprising: segmenting a video sequence into a plurality of shots; selecting a frame from each of the plurality of shots; for each selected frame, calculating a frame goodness measure as a function of a histogram entropy and a histogram standard deviation; determining a thumbnail frame from the selected frames as a function of the frame goodness measures; and wherein the thumbnail frame is for presentation to a user to facilitate user browsing of video data.
Claim 26
Dependent Claim
43 words
The computer-readable medium of claim 21 , wherein the computer-program instructions for determining further comprise instructions for: identifying a maximum frame goodness measure from the frame goodness measures; and determining the thumbnail frame as a frame associated with the maximum frame goodness measure.
Claim 27
Dependent Claim
55 words
The computer-readable medium of claim 21 , wherein the computer-program instructions for calculating a goodness measure further comprise instructions for: calculating a color histogram; calculating an entropy of the color histogram; calculating a standard deviation of the color histogram; and calculating the goodness measure as a weighted combination of the entropy and the standard deviation.
Claim 28
Dependent Claim
22 words
The computer-readable medium of claim 21 , wherein the computer-program instructions for segmenting further comprise instructions for performing a shot boundary detection.
Claim 29
Dependent Claim
66 words
A computer-implemented method comprising: calculating a color histogram of a video frame in a video sequence; calculating an entropy of the color histogram; calculating a standard deviation of the color histogram; calculating a frame goodness measure as a weighted combination of the entropy and the standard; and display in a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 30
Dependent Claim
66 words
A computer-implemented method comprising: calculating a color histogram of a video frame in a video sequence; calculating an entropy of the color histogram; calculating a standard deviation of the color histogram; calculating a frame goodness measure as a weighted combination of the entropy and the standard; and display in a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 31
Dependent Claim
36 words
The method of claim 25 , further comprising: calculating additional frame goodness measures for additional video frames in the video sequence; and determining a thumbnail video frame for the video sequence by comparing frame goodness measures.
Claim 32
Dependent Claim
34 words
The method of claim 26 , wherein the determining further comprises: identifying a maximum frame goodness measure; and selecting a video frame corresponding to the maximum frame goodness measure as the thumbnail video frame.
Claim 33
Dependent Claim
61 words
A computer-implemented method comprising: temporally segmenting a video sequence; calculating frame goodness measures within a particular temporal segment using histogram entropies and histogram standard deviations; determining from the frame goodness measures, a local maximum goodness measure for the particular temporal segment; and displaying a visually representative video thumbnail based on the local maximum goodness measure for user browsing of video data.
Claim 34
Dependent Claim
61 words
A computer-implemented method comprising: temporally segmenting a video sequence; calculating frame goodness measures within a particular temporal segment using histogram entropies and histogram standard deviations; determining from the frame goodness measures, a local maximum goodness measure for the particular temporal segment; and displaying a visually representative video thumbnail based on the local maximum goodness measure for user browsing of video data.
Claim 35
Dependent Claim
22 words
The method of claim 28 , further comprising generating a video thumbnail from a frame associated with the local maximum goodness measure.
Claim 36
Dependent Claim
71 words
A computer-implemented method comprising: segmenting a video sequence into a plurality of shots; selecting a frame from each of the plurality of shots; for each selected frame, calculating a frame goodness measure as a function of a histogram entropy and a histogram standard deviation; determining a thumbnail frame from the selected frames as a function of the frame goodness measures; and displaying the thumbnail frame for user browsing of video data.
Claim 37
Dependent Claim
71 words
A computer-implemented method comprising: segmenting a video sequence into a plurality of shots; selecting a frame from each of the plurality of shots; for each selected frame, calculating a frame goodness measure as a function of a histogram entropy and a histogram standard deviation; determining a thumbnail frame from the selected frames as a function of the frame goodness measures; and displaying the thumbnail frame for user browsing of video data.
Claim 38
Dependent Claim
37 words
The method of claim 30 , wherein the determining further comprises: identifying a maximum frame goodness measure from the frame goodness measures; and determining the thumbnail frame as a frame associated with the maximum frame goodness measure.
Claim 39
Dependent Claim
94 words
A computing device comprising: a processor; and a memory coupled to the processor, the memory comprising video content and computer-program instructions executable by the processor, the computer-program instructions being configured such that when executed by the processor to implement a video thumbnail algorithm the video thumbnail algorithm being configured to determine a visually representative video thumbnail frame by calculating a frame goodness measure for each of a plurality of video frames within the video content, each frame goodness measure comprising a weighted combination of a frame histogram entropy and a frame histogram standard deviation.
Claim 40
Dependent Claim
94 words
A computing device comprising: a processor; and a memory coupled to the processor, the memory comprising video content and computer-program instructions executable by the processor, the computer-program instructions being configured such that when executed by the processor to implement a video thumbnail algorithm the video thumbnail algorithm being configured to determine a visually representative video thumbnail frame by calculating a frame goodness measure for each of a plurality of video frames within the video content, each frame goodness measure comprising a weighted combination of a frame histogram entropy and a frame histogram standard deviation.
Claim 41
Dependent Claim
77 words
A video device comprising: a processor; and a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor for performing operations comprising: recording video content; and determining a video thumbnail; and wherein the determining comprises calculating a frame goodness measure for each of a plurality of video frames within the video content, each frame goodness measure being based on a weighted combination of a frame histogram entropy and a frame histogram standard deviation.
Claim 42
Dependent Claim
77 words
A video device comprising: a processor; and a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor for performing operations comprising: recording video content; and determining a video thumbnail; and wherein the determining comprises calculating a frame goodness measure for each of a plurality of video frames within the video content, each frame goodness measure being based on a weighted combination of a frame histogram entropy and a frame histogram standard deviation.
Claim 43
Dependent Claim
20 words
The video device of claim 33 , further comprising a hardware circuit block configured to implement the video thumbnail algorithm.
Claim 44
Dependent Claim
78 words
A video recording device comprising: means for calculating a color histogram of a video frame in a video sequence; means for calculating an entropy of the color histogram; means for calculating a standard deviation of the color histogram; means for calculating a frame goodness measure as a weighted combination of the entropy and the standard deviation; and means for displaying in a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 45
Dependent Claim
78 words
A video recording device comprising: means for calculating a color histogram of a video frame in a video sequence; means for calculating an entropy of the color histogram; means for calculating a standard deviation of the color histogram; means for calculating a frame goodness measure as a weighted combination of the entropy and the standard deviation; and means for displaying in a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 46
Dependent Claim
49 words
A computer comprising: means for receiving a video sequence; means for calculating frame goodness measures for frames within the video sequence from histogram entropies and histogram standard deviations; and means for displaying a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 47
Dependent Claim
49 words
A computer comprising: means for receiving a video sequence; means for calculating frame goodness measures for frames within the video sequence from histogram entropies and histogram standard deviations; and means for displaying a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 48
Dependent Claim
74 words
A computer comprising: means for calculating a color histogram of a video frame in a video sequence; means for calculating an entropy of the color histogram; means for calculating a standard deviation of the color histogram; means for calculating a frame goodness measure as a weighted combination of the entropy and the standard; and means for displaying a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 49
Dependent Claim
74 words
A computer comprising: means for calculating a color histogram of a video frame in a video sequence; means for calculating an entropy of the color histogram; means for calculating a standard deviation of the color histogram; means for calculating a frame goodness measure as a weighted combination of the entropy and the standard; and means for displaying a visually representative video thumbnail based on the frame goodness measure for user browsing of video data.
Claim 50
Dependent Claim
67 words
A computer comprising: means for temporally segmenting a video sequence; means for calculating frame goodness measures within a particular temporal segment using histogram entropies and histogram standard deviations; means for determining from the frame goodness measures, a local maximum goodness measure for the particular temporal segment; an means for displaying a visually representative video thumbnail based on the maximum goodness measure for user browsing of video data.
Claim 51
Dependent Claim
67 words
A computer comprising: means for temporally segmenting a video sequence; means for calculating frame goodness measures within a particular temporal segment using histogram entropies and histogram standard deviations; means for determining from the frame goodness measures, a local maximum goodness measure for the particular temporal segment; an means for displaying a visually representative video thumbnail based on the maximum goodness measure for user browsing of video data.
Claim 52
Dependent Claim
79 words
A computer comprising: means for segmenting a video sequence into a plurality of shots; means for selecting a frame from each of the plurality of shots; means for calculating a frame goodness measure as a function of a histogram entropy and a histogram standard deviation for each selected frame; means for determining a thumbnail frame from the selected frames as a function of the frame goodness; and means for displaying the thumbnail frame for user browsing of video data.
Claim 53
Dependent Claim
79 words
A computer comprising: means for segmenting a video sequence into a plurality of shots; means for selecting a frame from each of the plurality of shots; means for calculating a frame goodness measure as a function of a histogram entropy and a histogram standard deviation for each selected frame; means for determining a thumbnail frame from the selected frames as a function of the frame goodness; and means for displaying the thumbnail frame for user browsing of video data.
References Cited (Backward Citations)
EP0690413A2
Cited
US5892847A
Cited
US6453073B2
Cited
US5881176A
Cited
US5835163A
Cited
US5956026A
Cited
US5995095A
Cited
US6970602B1
Cited
US20020133486A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-04-30 | 2010-10-30 | 2011-05-01 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-04-30 | 2014-10-30 | 2015-05-01 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-04-30 | 2018-10-30 | 2019-05-01 | $7700.00 | Window Closed / Lapsed |