US7151852B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and system for segmentation, classification, and summarization of video images
Assignee: NEC Corp
•
Filing: 2003-10-21
•
Granted: 2006-12-19
•
Expires: 2023-10-21
Abstract
Abstract In a technique for video segmentation, classification and summarization based on the singular value decomposition, frames of the input video sequence are represented by vectors composed of concatenated histograms descriptive of the spatial distributions of colors within the video frames. The singular value decomposition maps these vectors into a refined feature space. In the refined feature space produced by the singular value decomposition, the invention uses a metric to measure the amount of information contained in each video shot of the input video sequence. The most static video shot is defined as an information unit, and the content value computed from this shot is used as a threshold to cluster the remaining frames. The clustered frames are displayed using a set of static keyframes or a summary video sequence. The video segmentation technique relies on the distance between the frames in the refined feature space to calculate the similarity between frames in the input video sequence. The input video sequence is segmented based on the values of the calculated similarities. Finally, average video attribute values in each segment are used in classifying the segments.
Inventors
- 👤 Yihong Gong
- 👤 Xin Liu
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:18
CPC Classifications
G; G06; G06V; G06V20/00; G06V20/40; H; H04; H04N; H04N5/00; H04N5/14; H04N5/147; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/738; G06F16/739; G06F16/78; G06F16/783; G06F16/7847; G06F16/785; G11; G11B; G11B27/00; G11B27/02; G11B27/031; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933
Claims Analysis
Total Claims: 28
Claim 1
Independent Claim
106 words
A method for segmenting an input video sequence, said input video sequence comprising a plurality of frames, said method comprising: grouping said plurality of frames into a plurality of video shots, wherein grouping is determined using a minimum time for frame inclusion, a maximum time for frame exclusion and a range of time from the minimum time to the maximum time, for which further analysis is performed; computing a similarity between each of said plurality of frames and a frame preceding said each of said plurality of frames in time; segmenting said input video sequence into said plurality of video shots according to said computed similarity.
Claim 2
Independent Claim
106 words
A method for segmenting an input video sequence, said input video sequence comprising a plurality of frames, said method comprising: grouping said plurality of frames into a plurality of video shots, wherein grouping is determined using a minimum time for frame inclusion, a maximum time for frame exclusion and a range of time from the minimum time to the maximum time, for which further analysis is performed; computing a similarity between each of said plurality of frames and a frame preceding said each of said plurality of frames in time; segmenting said input video sequence into said plurality of video shots according to said computed similarity.
Claim 3
Dependent Claim
22 words
The method of claim 1 , wherein said similarity is calculated using a refined feature space representation of said input video sequence.
Claim 4
Dependent Claim
24 words
The method of claim 2 , wherein said refined feature space representation is created using a singular value decomposition of said input video sequence.
Claim 5
Dependent Claim
25 words
The method of claim 3 , wherein said singular value decomposition is performed using frames selected with a fixed interval from said input video sequence.
Claim 6
Dependent Claim
30 words
The method of claim 4 , wherein said selected frames are arranged into a feature frame matrix, and wherein said singular value decomposition is performed on said feature frame matrix.
Claim 7
Dependent Claim
37 words
The method of claim 5 , wherein said performed singular value decomposition produces a matrix, each column of said produced matrix comprising a frame in said refined feature space representing a frame in said input video sequence.
Claim 8
Dependent Claim
19 words
The method of claim 1 , further comprising (d) extracting features from each of said plurality of video shots.
Claim 9
Dependent Claim
38 words
The method of claim 1 , wherein said computed similarity is compared to at least a first threshold similarity and a second threshold similarity, and said input video sequence is segmented according to a result of said comparison.
Claim 10
Dependent Claim
42 words
The method of claim 1 , wherein if in said computed similarity is below a first threshold similarity, said each of said plurality of frames is put into a one of said plurality of video shots containing a precedent in time frame.
Claim 11
Dependent Claim
31 words
The method of claim 1 , wherein if in said computed similarity is above a second threshold similarity, said each of said plurality of frames is designated as a shot boundary.
Claim 12
Dependent Claim
55 words
The method of claim 1 , wherein if in said computed similarity is between a first threshold similarity and a second threshold similarity, said each of said plurality of frames is put into a one of said plurality of video shots according to a further analysis performed using additional frames from said plurality of frames.
Claim 13
Dependent Claim
102 words
A computer-readable medium containing a program for segmenting an input video sequence, said input video sequence comprising a plurality of frames, said program comprising: grouping said plurality of frames into a plurality of video shots, wherein grouping is determined using a minimum time for frame inclusion, a maximum time for frame exclusion and a range of time from the minimum time to the maximum time, for which further analysis is performed; computing a similarity between each of said plurality of frames and a subsequent in time frame; segmenting said input video sequence into a plurality of shots according to said computed similarity.
Claim 14
Dependent Claim
102 words
A computer-readable medium containing a program for segmenting an input video sequence, said input video sequence comprising a plurality of frames, said program comprising: grouping said plurality of frames into a plurality of video shots, wherein grouping is determined using a minimum time for frame inclusion, a maximum time for frame exclusion and a range of time from the minimum time to the maximum time, for which further analysis is performed; computing a similarity between each of said plurality of frames and a subsequent in time frame; segmenting said input video sequence into a plurality of shots according to said computed similarity.
Claim 15
Dependent Claim
23 words
The computer-readable medium of claim 12 , wherein said similarity is calculated using a refined feature space representation of said input video sequence.
Claim 16
Dependent Claim
25 words
The computer-readable medium of claim 13 , wherein said refined feature space representation is created using a singular value decomposition of said input video sequence.
Claim 17
Dependent Claim
26 words
The computer-readable medium of claim 14 , wherein said singular value decomposition is performed using frames selected with a fixed interval from said input video sequence.
Claim 18
Dependent Claim
31 words
The computer-readable medium of claim 15 , wherein said selected frames are arranged into a feature frame matrix, and wherein said singular value decomposition is performed on said feature frame matrix.
Claim 19
Dependent Claim
38 words
The computer-readable medium of claim 16 , wherein said performed singular value decomposition produces a matrix, each column of said produced matrix comprising a frame in said refined feature space representing a frame in said input video sequence.
Claim 20
Dependent Claim
22 words
The computer-readable medium of claim 12 , wherein said program further comprises extracting features from each of said plurality of video shots.
Claim 21
Dependent Claim
34 words
The computer-readable medium of claim 12 , wherein in said computed similarity is compared to at least two threshold similarities, and said input video sequence is segmented according to a result of said comparison.
Claim 22
Dependent Claim
43 words
The computer-readable medium of claim 12 , wherein if in said computed similarity is below a first threshold similarity, said each of said plurality of frames is put into a one of said plurality of video shots containing a precedent in time frame.
Claim 23
Dependent Claim
32 words
The computer-readable medium of claim 12 , wherein if in said computed similarity is above a second threshold similarity, said each of said plurality of frames is designated as a shot boundary.
Claim 24
Dependent Claim
56 words
The computer-readable medium of claim 12 , wherein if in said computed similarity is between a first threshold similarity and a second threshold similarity, said each of said plurality of frames is put into a one of said plurality of video shots according to a further analysis performed using additional frames from said plurality of frames.
Claim 25
Dependent Claim
30 words
The method of claim 1 , further comprising extracting features from each of said plurality of video shots and using said extracted features to index said plurality of video shots.
Claim 26
Dependent Claim
26 words
The method of claim 23 , wherein said extracted features are features of a video frame representative of said each of said plurality of video shots.
Claim 27
Dependent Claim
35 words
The computer-readable medium of claim 12 , wherein said program further comprises (c) extracting features from each of said plurality of video shots and using said extracted features to index said plurality of video shots.
Claim 28
Dependent Claim
27 words
The computer-readable medium of claim 25 , wherein said extracted features are features of a video frame representative of said each of said plurality of video shots.
References Cited (Backward Citations)
US5635982A
Cited
US5818439A
Cited
US5911008A
Cited
US5734735A
Cited
US6496228B1
Cited
US6195458B1
Cited
US6172685B1
Cited
US5956026A
Cited
US6393054B1
Cited
US6574378B1
Cited
US6404925B1
Cited
US6535639B1
Cited
US6807306B1
Cited
US6842197B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-12-18 | 2010-06-19 | 2010-12-19 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-12-18 | 2014-06-19 | 2014-12-19 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-12-18 | 2018-06-19 | 2018-12-19 | $7700.00 | Window Closed / Lapsed |