US7143352B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Blind summarization of video content
Assignee: Mitsubishi Electric Research Laboratories Inc
•
Filing: 2002-11-01
•
Granted: 2006-11-28
•
Expires: 2022-11-01
Abstract
Abstract A method summarizes unknown content of a video. First, low-level features of the video are selected. The video is then partitioned into segments according to the low-level features. The segments are grouped into disjoint clusters where each cluster contains similar segments. The clusters are labeled according to the low-level features, and parameters characterizing the clusters are assigned. High-level patterns among the labels are found, and the these patterns are used to extract frames from the video according to form a content-adaptive summary of the unknown content of the video.
Inventors
- 👤 Ajay Divakaran
- 👤 Kadir A. Peker
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:16
CPC Classifications
G; G06; G06V; G06V20/00; G06V20/40; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/738; G06F16/739; G06F16/78; G06F16/783; G06F16/7834; G06F16/7847; G06F16/785; G06F16/786
Claims Analysis
Total Claims: 21
Claim 1
Independent Claim
105 words
A method for summarizing unknown content of a video, comprising: selecting low-level features of the video; partitioning the video into segments according to the low-level features; generating time-series data from the video based on the selected low-level features of the video; self-correlating the time-series data to determine the similar segments grouping the segments into a plurality of disjoint clusters, each cluster containing similar segments; labeling the plurality of clusters with labels according to the low-level features; finding high-level patterns among the labels; and extracting frames from the video according to the high-level patterns to form a content-adaptive summary of the unknown content of the video.
Claim 2
Independent Claim
105 words
A method for summarizing unknown content of a video, comprising: selecting low-level features of the video; partitioning the video into segments according to the low-level features; generating time-series data from the video based on the selected low-level features of the video; self-correlating the time-series data to determine the similar segments grouping the segments into a plurality of disjoint clusters, each cluster containing similar segments; labeling the plurality of clusters with labels according to the low-level features; finding high-level patterns among the labels; and extracting frames from the video according to the high-level patterns to form a content-adaptive summary of the unknown content of the video.
Claim 3
Dependent Claim
18 words
The method of claim 1 wherein the summary is based on a combination of common and uncommon patterns.
Claim 4
Dependent Claim
17 words
The method of claim 1 wherein a production style of the content of the video is unknown.
Claim 5
Dependent Claim
13 words
The method of claim 1 wherein the video includes visual and audio data.
Claim 6
Dependent Claim
26 words
The method of claim 1 wherein the low-level features are selected from a group consisting of motion activity, color, texture, audio, semantic descriptors, and combinations thereof.
Claim 7
Dependent Claim
11 words
The method of claim 1 further comprising: smoothing the time-series data.
Claim 8
Dependent Claim
11 words
The method of claim 1 further comprising: weighting the time-series data.
Claim 9
Dependent Claim
19 words
The method of claim 1 further comprising: associating the plurality of clusters by associations to infer the high-level patterns.
Claim 10
Dependent Claim
17 words
The method of claim 1 wherein all similar segments are found at a plurality of temporal resolutions.
Claim 11
Dependent Claim
27 words
The method of claim 1 further comprising: self-correlating the time series data in a point cross-distance matrix to measure point cross-distances indicative of similarity of the segments.
Claim 12
Dependent Claim
20 words
The method of claim 10 wherein the point cross-distances are represented by substantially diagonal lines in the point cross-distance matrix.
Claim 13
Dependent Claim
17 words
The method of claim 1 further comprising: time warping the time-series data to find the similar segments.
Claim 14
Dependent Claim
14 words
The method of claim 12 wherein the time warping is performed by dynamic programming.
Claim 15
Dependent Claim
12 words
The method of claim 1 further comprising: correlating the labels in time.
Claim 16
Dependent Claim
11 words
The method of claim 14 wherein the time correlation is simultaneous.
Claim 17
Dependent Claim
21 words
The method of claim 1 further comprising: generating conditions and results of occurrences of the labels to discover the high-level patterns.
Claim 18
Dependent Claim
14 words
The method of claim 8 wherein strong associations correspond to common and uncommon patterns.
Claim 19
Dependent Claim
24 words
The method of claim 1 wherein the grouping further comprises: dividing overlapping segments so the overlapping segments have an identical overlap or no overlap.
Claim 20
Dependent Claim
58 words
The method of claim 1 further comprising: assigning parameters to each cluster, the cluster parameters including a cluster compactness, a number of segments in the cluster, a total length of the segments in the cluster measured in frames, and an average distance between closest points of the segments to indicate a spreading of the segments in the video.
Claim 21
Dependent Claim
19 words
The method of claim 1 further comprising: evaluating a cost function to determine a required length of the summary.
References Cited (Backward Citations)
US5638460A
Cited
US20040170385A1
Cited
US6278446B1
Cited
US6369835B1
Cited
US6690725B1
Cited
US20050188328A1
Cited
US20040170321A1
Cited
JP2001243236A
Cited
US20020018594A1
Cited
US6697523B1
Cited
US20030081937A1
Cited
US20030016945A1
Cited
US20030058268A1
Cited
US20030206710A1
Cited
US20030083871A1
Cited
US6956904B2
Cited
US20030190060A1
Cited
US20030210886A1
Cited
US20030210261A1
Cited
US20040086180A1
Cited
US20040085323A1
Cited
US7103222B2
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-11-27 | 2010-05-29 | 2010-11-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-11-27 | 2014-05-29 | 2014-11-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-11-27 | 2018-05-29 | 2018-11-28 | $7700.00 | Window Closed / Lapsed |