US6578040B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for indexing of topics using foils
Assignee: International Business Machines Corp
•
Filing: 2000-06-14
•
Granted: 2003-06-10
•
Expires: 2020-06-14
Abstract
Abstract A method and structure for indexing multi-media data comprising deriving keywords from a first media type (slides), matching the keywords to a second media type (video), identifying an appearance of the first media type in the second media type, and calculating a co-occurrence of the keywords and the appearance of the first media type in the second media type. The invention produces an index of the second media type based on the co-occurrence.
Inventors
- 👤 Tanveer Fathima Syeda-Mahmood
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:30
CPC Classifications
G; G06; G06V; G06V20/00; G06V20/40; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7834; G06F16/7844; G06F16/7847; G06F16/7854; Y; Y10; Y10S; Y10S707/00; Y10S707/99941; Y10S707/99943
Claims Analysis
Total Claims: 30
Claim 1
Independent Claim
57 words
A computer implemented method of indexing multi-media data comprising: deriving keywords from a first media type; matching said keywords to a second media type; identifying an appearance of said first media type in said second media type; and calculating a co-occurrence of said keywords and said appearance of said first media type in said second media type.
Claim 2
Dependent Claim
57 words
A computer implemented method of indexing multi-media data comprising: deriving keywords from a first media type; matching said keywords to a second media type; identifying an appearance of said first media type in said second media type; and calculating a co-occurrence of said keywords and said appearance of said first media type in said second media type.
Claim 3
Dependent Claim
20 words
The method in claim 1 , further comprising producing an index of said second media type based on said co-occurrence.
Claim 4
Dependent Claim
44 words
The method in claim 1 , wherein said identifying comprises: generating keyframes from said second media type; extracting geometric keyframe features from said keyframes and geometric slide features from said first media type; and matching said geometric slide features and said geometric keyframe features.
Claim 5
Dependent Claim
26 words
The method in claim 3 , further comprising identifying background matching regions in said keyframes having colors matching colors of backgrounds in said second media type.
Claim 6
Dependent Claim
22 words
The method in claim 4 , wherein said extracting of said geometric keyframe features is performed only in said background matching regions.
Claim 7
Dependent Claim
35 words
The method in claim 3 , wherein said matching identifies which portion of said first media type has a highest number of geometric slide features matching geometric keyframe features in a keyframe of said keyframes.
Claim 8
Dependent Claim
50 words
The method in claim 3 , wherein said extracting includes: identifying changes in image intensity as edges; forming curves connecting said edges; identifying corners where said curves change direction; grouping said curves into curve-groups; and designating a sequence of three consecutive features in each of said curve-groups as basis triples.
Claim 9
Dependent Claim
40 words
The method in claim 7 , wherein said matching comprises: computing coordinates of said basis triples; identifying which portion of said first media type has a highest number of basis triples matching basis triples in a keyframe of said keyframes.
Claim 10
Dependent Claim
39 words
The method in claim 3 , wherein said generating of said keyframes comprises: dividing said second media type into portions based upon scene changes; and selecting one frame from each portion of said second media type as a keyframe.
Claim 11
Dependent Claim
57 words
The method in claim 1 , wherein said calculating comprises: processing said multi-media data to extract relevance scores and time reference points of matches to individual media modes; identifying overlapping time periods of matching keywords and said appearance of said first media type in said second media type; and ranking a relevance of said overlapping time periods.
Claim 12
Dependent Claim
20 words
The method in claim 10 , wherein said ranking includes finding an overlapping time period having a highest relevance score.
Claim 13
Dependent Claim
21 words
The method in claim 10 , wherein said ranking includes segmenting said overlapping time period to identify beginning and ending events.
Claim 14
Dependent Claim
19 words
The method in claim 10 , wherein said ranking includes finding a largest number of different modes of overlap.
Claim 15
Dependent Claim
20 words
The method in claim 10 , wherein said modes comprise two or more of audio, video, text, and graphic display.
Claim 16
Dependent Claim
54 words
A computer implemented method of indexing multi-media data comprising: deriving keywords from slides; matching said keywords to an audio portion of video data; identifying an appearance of said slides in a video portion of said video data; and calculating a co-occurrence of said keywords and said appearance of said slides in said video data.
Claim 17
Dependent Claim
54 words
A computer implemented method of indexing multi-media data comprising: deriving keywords from slides; matching said keywords to an audio portion of video data; identifying an appearance of said slides in a video portion of said video data; and calculating a co-occurrence of said keywords and said appearance of said slides in said video data.
Claim 18
Dependent Claim
19 words
The method in claim 15 , further comprising producing an index of said video data based on said co-occurrence.
Claim 19
Dependent Claim
41 words
The method in claim 15 , wherein said identifying comprises: generating keyframes from said video portion; extracting geometric keyframe features from said keyframes and geometric slide features from said slides; and matching said geometric slide features and said geometric keyframe features.
Claim 20
Dependent Claim
24 words
The method in claim 17 , further comprising identifying background matching regions in said keyframes having colors matching colors of backgrounds in said slides.
Claim 21
Dependent Claim
22 words
The method in claim 18 , wherein said extracting of said geometric keyframe features is performed only in said background matching regions.
Claim 22
Dependent Claim
33 words
The method in claim 17 , wherein said matching identifies which slide of said slides has a highest number of geometric slide features matching geometric keyframe features in a keyframe of said keyframes.
Claim 23
Dependent Claim
50 words
The method in claim 17 , wherein said extracting includes: identifying changes in image intensity as edges; forming curves connecting said edges; identifying corners where said curves change direction; grouping said curves into curve-groups; and designating a sequence of three consecutive features in each of said curve-groups as basis triples.
Claim 24
Dependent Claim
39 words
The method in claim 21 , wherein said matching comprises: computing coordinates of said basis triples; identifying which slide of said slides has a highest number of basis triples matching basis triples in a key frame of said keyframes.
Claim 25
Dependent Claim
36 words
The method in claim 17 , wherein said generating of said keyframes comprises: dividing said video data into portions based upon scene changes; and selecting one frame from each portion of said video as a keyframe.
Claim 26
Dependent Claim
61 words
The method in claim 15 , wherein said calculating comprises: processing said video data to extract relevance scores and time reference points of matching keywords and said appearance of said slide in said video portion; identifying overlapping time periods of matching keywords and said appearance of said slides in said video portion; and ranking a relevance of said overlapping time periods.
Claim 27
Dependent Claim
20 words
The method in claim 24 , wherein said ranking includes finding an overlapping time period having a highest relevance score.
Claim 28
Dependent Claim
21 words
The method in claim 24 , wherein said ranking includes segmenting said overlapping time period to identify beginning and ending events.
Claim 29
Dependent Claim
19 words
The method in claim 24 , wherein said ranking includes finding a largest number of different modes of overlap.
Claim 30
Dependent Claim
20 words
The method in claim 24 , wherein said modes comprise two or more of audio, video, text, and graphic display.
References Cited (Backward Citations)
US4660107A
Cited
US6044365A
Cited
US6021412A
Cited
US5884014A
Cited
US6195093B1
Cited
US6342904B1
Cited
US6353450B1
Cited
US6404925B1
Cited
US6397213B1
Cited
US6369835B1
Cited
US6249281B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-06-09 | 2006-12-09 | 2007-06-10 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-06-09 | 2010-12-09 | 2011-06-10 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-06-09 | 2014-12-09 | 2015-06-10 | $7700.00 | Window Closed / Lapsed |