US8200063B2
Active / In Force
US
⏱️ 0.9 years remaining
System and method for video summarization
Assignee: Fujifilm Business Innovation Corp
•
Filing: 2007-09-24
•
Granted: 2012-06-12
•
Expires: 2027-09-24
Abstract
Abstract The subject invention relates to a system and method for video summarization, and more specifically to a system for segmenting and classifying data from a video in order to create a summary video that preserves and summarizes relevant content. In one embodiment, the system first extracts appearance, motion, and audio features from a video in order to create video segments corresponding to the extracted features. The video segments are then classified as dynamic or static depending on the appearance-based and motion-based features extracted from each video segment. The classified video segments are then grouped into clusters to eliminate redundant content. Select video segments from each cluster are selected as summary segments, and the summary segments are compiled to form a summary video. The parameters for any of the steps in the summarization of the video can be altered so that a user can adapt the system to any type of video, although the system is designed to summarize unstructured videos where the content is unknown. In another aspect, audio features can also be used to further summarize video with certain audio properties.
Inventors
- 👤 Francine Chen
- 👤 Matthew Cooper
- 👤 John Adcock
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:38
CPC Classifications
G; G11; G11B; G11B27/00; G11B27/10; G11B27/19; G11B27/28; G06; G06F; G06F16/00; G06F16/70; G06F16/73; G06F16/738; G06F16/739; G06F16/78; G06F16/783; G06F16/7834; G06F16/7847; G06F16/785; G06F16/786
Claims Analysis
Total Claims: 24
Claim 1
Independent Claim
156 words
A system for summarizing a video, comprising: receiving an input video signal; extracting a plurality of features from the video, wherein separating the video into a first set of segments based upon a first feature of the plurality of features; wherein each segment is separated based on the first feature of the plurality of features; separating the video into a second set of segments based upon a second feature of the plurality of features; wherein each segment is separated based on the second feature of the plurality of features; classifying the video segments based upon the plurality of features; wherein at least one of the first and second sets of segments is classified based on both the first feature and the second feature of the plurality of features; grouping similarly classified video segments into clusters selecting a plurality of segments from the clusters to comprise summary segments; and compiling the summary segments into a summary video.
Claim 2
Independent Claim
156 words
A system for summarizing a video, comprising: receiving an input video signal; extracting a plurality of features from the video, wherein separating the video into a first set of segments based upon a first feature of the plurality of features; wherein each segment is separated based on the first feature of the plurality of features; separating the video into a second set of segments based upon a second feature of the plurality of features; wherein each segment is separated based on the second feature of the plurality of features; classifying the video segments based upon the plurality of features; wherein at least one of the first and second sets of segments is classified based on both the first feature and the second feature of the plurality of features; grouping similarly classified video segments into clusters selecting a plurality of segments from the clusters to comprise summary segments; and compiling the summary segments into a summary video.
Claim 3
Dependent Claim
18 words
The system of claim 1 , wherein the plurality of features extracted is based on motion and appearance.
Claim 4
Dependent Claim
21 words
The system of claim 2 , wherein the appearance features are a color histogram for each frame of a video segment.
Claim 5
Dependent Claim
20 words
The system of claim 2 , wherein the motion feature includes the type of camera motion in a video segment.
Claim 6
Dependent Claim
20 words
The system of claim 4 , wherein the type of camera motion is a camera pan or a camera zoom.
Claim 7
Dependent Claim
18 words
The system of claim 2 , wherein the plurality of features extracted includes features derived from the audio.
Claim 8
Dependent Claim
23 words
The system of claim 6 , wherein the audio features extracted are used to separate the video into video segments based on sound.
Claim 9
Dependent Claim
29 words
The system of claim 2 , wherein the motion and appearance features extracted are used to separate the video into segments that have similar appearance features and motion features.
Claim 10
Dependent Claim
32 words
The system of claim 8 , wherein the separation of video into segments based on appearance features and the separation of video into segments based on motion features are carried out separately.
Claim 11
Dependent Claim
23 words
The system of claim 1 , wherein the video segments are classified as dynamic video segments, static video segments, or ignored video segments.
Claim 12
Dependent Claim
29 words
The system of claim 10 , wherein the classifying of video segments includes combining the segments separated based on appearance features and the segments separated based on motion features.
Claim 13
Dependent Claim
20 words
The system of claim 11 , wherein the dynamic video segments and static video segments are grouped into clusters separately.
Claim 14
Dependent Claim
22 words
The system of claim 12 , wherein the dynamic video segments are grouped into clusters using both appearance features and motion features.
Claim 15
Dependent Claim
21 words
The system of claim 13 , wherein the grouping of dynamic video segments into clusters includes dimension reduction and spectral clustering.
Claim 16
Dependent Claim
18 words
The system of claim 14 , wherein the static video segments are grouped into clusters using appearance features.
Claim 17
Dependent Claim
18 words
The system of claim 15 , wherein the grouping of static video segments into clusters uses agglomerative clustering.
Claim 18
Dependent Claim
19 words
The system of claim 1 , wherein a similarity matrix is calculated for all video segments in the clusters.
Claim 19
Dependent Claim
29 words
The system of claim 17 , wherein the selecting of a plurality of video segments from the clusters to comprise summary segments is calculated based on the similarity matrix.
Claim 20
Dependent Claim
19 words
The system of claim 18 , wherein one video segment per cluster is selected to comprise the summary segments.
Claim 21
Dependent Claim
30 words
The system of claim 1 , wherein the summary segments are compiled in order by the beginning time of each segment, such that the summary video is in temporal order.
Claim 22
Dependent Claim
21 words
The system of claim 1 , wherein the summary video further includes information displayed to a user regarding the summary segments.
Claim 23
Independent Claim
159 words
A system for summarizing a video, comprising: receiving an input signal of a video; extracting a plurality of features from the video, wherein the features include motion and appearance; creating a first set of video segments based on the motion features and creating a second set of video segments based on the appearance features; classifying at least one of the first and second sets of video segments based upon the both motion and appearance features, wherein the video segments are classified as dynamic video segments, static video segments, or ignore video segments; grouping similarly classified video segments into clusters, wherein the dynamic video segments and static video segments are clustered separately; selecting a plurality of segments from the clusters to comprise summary segments, wherein one video segment from each cluster is selected; and compiling the summary segments into a summary video, wherein the summary video also includes data relating to the selected summary segments in relation to the video.
Claim 24
Independent Claim
159 words
A system for summarizing a video, comprising: receiving an input signal of a video; extracting a plurality of features from the video, wherein the features include motion and appearance; creating a first set of video segments based on the motion features and creating a second set of video segments based on the appearance features; classifying at least one of the first and second sets of video segments based upon the both motion and appearance features, wherein the video segments are classified as dynamic video segments, static video segments, or ignore video segments; grouping similarly classified video segments into clusters, wherein the dynamic video segments and static video segments are clustered separately; selecting a plurality of segments from the clusters to comprise summary segments, wherein one video segment from each cluster is selected; and compiling the summary segments into a summary video, wherein the summary video also includes data relating to the selected summary segments in relation to the video.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2015-06-12 | 2015-12-12 | 2016-06-12 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2019-06-12 | 2019-12-12 | 2020-06-12 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2023-06-12 | 2023-12-12 | 2024-06-12 | $7700.00 | Window Closed / Lapsed |