Patent EP1456779B1 successfully refreshed from Google Patents.
← Back to Patent Explorer
EP1456779B1 Expired / Lapsed EP ⏱️ 0.0 years remaining

Method for summarizing unknown content of video

Assignee: Mitsubishi Electric Corp
•
Filing: 2003-10-15
•
Granted: 2007-04-18
•
Expires: 2023-10-15
📄 Original PDF 🌐 Google Patents 🔄 Sync Now

Abstract

Abstract A method for summarizing unknown content of a video. First, low-level features of the video are selected. The video is then partitioned into segments according to the low-level features. The segments are grouped into disjoint clusters where each cluster contains similar segments. The clusters are labeled according to the low-level features, and parameters characterizing the clusters are assigned. High-level patterns among the labels are found, and the these patterns are used to extract frames from the video according to form a content-adaptive summary of the unknown content of the video.

Inventors

  • 👤 Ajay Divakaran
  • 👤 Kadir A. Peker

Bibliographic Details

Application Number —
Jurisdiction EP
Data Source google_patents
Last Synchronized 2026-11-09 11:10
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/738; G06F16/739; G06F16/78; G06F16/783; G06F16/7834; G06F16/7847; G06F16/785; G06F16/786; G06V; G06V20/00; G06V20/40

Claims Analysis

Total Claims: 19
Claim 1
Independent Claim 171 words
A computerized method for summarizing unknown content of a video, comprising the steps of selecting one or more low-level features of the video; partitioning the video into segments, the segments being sequences of frames, according to the low-level features and grouping the segments into a plurality of disjoint clusters, each cluster containing similar segments, wherein similarity is based on a distance metric and wherein similar segments are identified by using time series data of the low-level features in a self-correlated cross-distance matrix and by finding minimum cost paths between pairs of points in the cross-distance matrix, said paths satisfying path cost criteria and direction constraints; labeling the plurality of clusters according to the low-level features by choosing labels out of sets of labels describing the low-level features; finding high-level patterns among the labels by looking for labels correlated in time, the correlation either being simultaneous or time shifted; and extracting frames from the video according to the high-level patterns to form a content-adaptive summary of the unknown content of the video.
Claim 2
Dependent Claim 17 words
The method of claim 1 wherein all similar segments are found at a plurality of temporal resolutions.
Claim 3
Dependent Claim 35 words
The method of claim 1 further comprising the steps of: generating time-series data from the video based on the selected low-level features of the video; and self-correlating the time-series data to determine the similar segments.
Claim 4
Dependent Claim 31 words
The method of claim 3 further comprising the step of : self-correlating the time series data in a point cross-distance matrix to measure point cross-distances indicative of similarity of the segments.
Claim 5
Dependent Claim 18 words
The method of claim 1 wherein the summary is based on a combination of common and uncommon patterns.
Claim 6
Dependent Claim 17 words
The method of claim 1 wherein a production style of the content of the video is unknown.
Claim 7
Dependent Claim 13 words
The method of claim 1 wherein the video includes visual and audio data.
Claim 8
Dependent Claim 26 words
The method of claim 1 wherein the low-level features are selected from a group consisting of motion activity, color, texture, audio, semantic descriptors, and combinations thereof.
Claim 9
Dependent Claim 14 words
The method of claim 3 further comprising the step of: smoothing the time-series data.
Claim 10
Dependent Claim 14 words
The method of claim 3 further comprising the step of: weighting the time-series data.
Claim 11
Dependent Claim 22 words
The method of claim 1 further comprising the step of: associating the plurality of clusters by associations to infer the high-level patterns.
Claim 12
Dependent Claim 20 words
The method of claim 4 wherein the point cross-distances are represented by substantially diagonal lines in the point cross-distance matrix.
Claim 13
Dependent Claim 20 words
The method of claim 3 further comprising the step of: time warping the time-series data to find the similar segments.
Claim 14
Dependent Claim 14 words
The method of claim 13 wherein the time warping is performed by dynamic programming.
Claim 15
Dependent Claim 25 words
The method of claim 1 further comprising the step of : generating conditions and results of occurrences of the labels to discover the high-level patterns.
Claim 16
Dependent Claim 14 words
The method of claim 11 wherein strong associations correspond to common and uncommon patterns.
Claim 17
Dependent Claim 27 words
The method of claim 1 wherein the grouping further comprises the step of: dividing overlapping segments so the overlapping segments have an identical overlap or no overlap.
Claim 18
Dependent Claim 61 words
The method of claim 1 further comprising the step of: assigning parameters to each cluster, the cluster parameters including a cluster compactness, a number of segments in the cluster, a total length of the segments in the cluster measured in frames, and an average distance between closest points of the segments to indicate a spreading of the segments in the video.
Claim 19
Dependent Claim 22 words
The method of claim 1 further comprising the step of: evaluating a cost function to determine a required length of the summary.

References Cited (Backward Citations)

No citation records indexed.

USPTO Annuity & Maintenance Windows

Maintenance schedule is calculated for granted US utility patents.

Assigned Categories & Tags

No categories assigned to this patent yet.

Assign Category Tag