US5835667A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for creating a searchable digital video library and a system and method of using such a library
Assignee: Carnegie Mellon University
•
Filing: 1994-10-14
•
Granted: 1998-11-10
•
Expires: 2014-10-14
Abstract
Abstract An apparatus and method of creating a digital library from audio data and video images. The method includes the steps of transcribing the audio data and marking the transcribed audio data with a first set of time-stamps and indexing the transcribed audio data. The method also includes the steps of digitizing the video data and marking the digitized video data with a second set of time-stamps related to the first set of time-stamps and segmenting the digitized video data into paragraphs according to a set of rules. The steps of storing the indexed audio data and the digitized video data with their respective sets of time-stamps is also provided. The method also includes the step of passing the transcribed audio data through a natural language interpreter before indexing the transcribed audio data. A method and apparatus for searching the digital library is disclosed.
Inventors
- 👤 Howard D. Wactlar
- 👤 Takeo Kanade
- 👤 Michael L. Mauldin
- 👤 D. Raj Reddy
- 👤 Scott M. Stevens
- 👤 Michael G. Christel
- 👤 Michael A. Smith
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-25 16:05
CPC Classifications
G; G11; G11B; G11B27/00; G11B27/10; G11B27/34; G06; G06F; G06F16/00; G06F16/70; G06F16/78; G06F16/783; G06F16/7834; G06F16/7844; G06F16/7847; G06F16/785; G11B27/02; G11B27/031; G11B27/034; G11B27/19; G11B27/28
Claims Analysis
Total Claims: 36
Claim 1
Independent Claim
134 words
A method of creating an independent digital library from existing audio data and video images comprising the steps of: transcribing said audio data and marking said transcribed audio data with a first set of time-stamps; indexing said transcribed audio data; digitizing said video data and marking said digitized video data with a second set of time-stamps related to said first set of time-stamps; segmenting said digitized video data into video paragraphs according to a set of rules based upon scene characterization of said video images and the processing of said audio data; and storing said indexed audio data and said segmented digitized video data with their respective sets of time-stamps to create the digital library which can be accessed through said indexed audio data without returning to the existing audio data and video images.
Claim 2
Independent Claim
134 words
A method of creating an independent digital library from existing audio data and video images comprising the steps of: transcribing said audio data and marking said transcribed audio data with a first set of time-stamps; indexing said transcribed audio data; digitizing said video data and marking said digitized video data with a second set of time-stamps related to said first set of time-stamps; segmenting said digitized video data into video paragraphs according to a set of rules based upon scene characterization of said video images and the processing of said audio data; and storing said indexed audio data and said segmented digitized video data with their respective sets of time-stamps to create the digital library which can be accessed through said indexed audio data without returning to the existing audio data and video images.
Claim 3
Dependent Claim
26 words
The method of claim 1 additionally comprising the step of passing said transcribed audio data through a natural language interpreter before indexing said transcribed audio data.
Claim 4
Dependent Claim
15 words
The method of claim 2 wherein said natural language interpreter updates said set of rules.
Claim 5
Independent Claim
149 words
An apparatus for creating an independent digital library from existing audio data and video images, comprising: means for transcribing said audio data and marking said transcribed audio data with a first set of time-stamps; means for indexing said transcribed audio data; means for digitizing said video data and marking said digitized video data with a second set of time-stamps related to said first set of time-stamps; means for storing a set of rules based upon scene characterization of said video images and the processing of said audio data; means for segmenting said digitized video data into video paragraphs according to said stored set of rules; and means for storing said indexed audio data and said segmented digitized video data with their respective sets of time-stamps to create the digital library which can be accessed through said indexed audio data without returning to the existing audio data and video images.
Claim 6
Independent Claim
149 words
An apparatus for creating an independent digital library from existing audio data and video images, comprising: means for transcribing said audio data and marking said transcribed audio data with a first set of time-stamps; means for indexing said transcribed audio data; means for digitizing said video data and marking said digitized video data with a second set of time-stamps related to said first set of time-stamps; means for storing a set of rules based upon scene characterization of said video images and the processing of said audio data; means for segmenting said digitized video data into video paragraphs according to said stored set of rules; and means for storing said indexed audio data and said segmented digitized video data with their respective sets of time-stamps to create the digital library which can be accessed through said indexed audio data without returning to the existing audio data and video images.
Claim 7
Dependent Claim
22 words
The apparatus of claim 4 additionally comprising natural language interpreter means for processing said transcribed audio data before said data is indexed.
Claim 8
Dependent Claim
16 words
The apparatus of claim 4 wherein said natural language interpreter means updates said set of rules.
Claim 9
Dependent Claim
30 words
The method of claim 1 additionally comprising the step of generating a set of icons after segmenting said digitized video data into video paragraphs according to said set of rules.
Claim 10
Dependent Claim
17 words
The method of claim 7 wherein said set of icons is a set of intelligent moving icons.
Claim 11
Dependent Claim
17 words
The method of claim 8 wherein said set of intelligent moving icons is generated using data-dependent heuristics.
Claim 12
Dependent Claim
32 words
The method of claim 1 additionally comprising the step of compressing said digitized video data before storing said indexed audio data and said digitized video data with their respective sets of time-stamps.
Claim 13
Dependent Claim
92 words
The method of claim 1 wherein the step of transcribing said audio data and marking said transcribed audio data with a first set of time stamps includes the steps of: producing a set of possible word occurrences, with each word occurrence having a start time and a plurality of possible end times; producing a plurality of possible begin times for each of said end times; generating a set of N-best hypotheses for said audio data; and selecting a best-scoring hypothesis from said set of N-best hypotheses to produce said transcribed audio data.
Claim 14
Dependent Claim
21 words
The method of claim 11 wherein said set of possible word occurrences is produced using a forward time synchronous pass function.
Claim 15
Dependent Claim
20 words
The method of claim 11 wherein said plurality of possible begin times are produced using a reverse time synchronous function.
Claim 16
Dependent Claim
60 words
The method of claim 2 wherein the step of passing said transcribed audio data through a natural language interpreter before indexing said transcribed audio data includes the steps of: summarizing said transcribed audio data; tagging said transcribed audio data using data extraction techniques; and correcting said tagged transcribed audio data using semantic and syntactic constraints and a phonetic knowledge base.
Claim 17
Dependent Claim
20 words
The method of claim 1 wherein said digitized video data are segmented into video paragraphs using comprehensive image statistic rules.
Claim 18
Dependent Claim
19 words
The method of claim 1 wherein said digitized video data are segmented into video paragraphs using camera motion rules.
Claim 19
Dependent Claim
19 words
The method of claim 1 wherein said digitized video data are segmented into video paragraphs using object motion rules.
Claim 20
Dependent Claim
21 words
The method of claim 1 wherein said digitized video data are segmented into video paragraphs using deformable templates and filtering rules.
Claim 21
Dependent Claim
19 words
The method of claim 1 wherein said digitized video data are segmented into video paragraphs using object presence rules.
Claim 22
Dependent Claim
19 words
The method of claim 1 wherein said digitized video data are segmented into video paragraphs using three-dimensional understanding rules.
Claim 23
Dependent Claim
29 words
The apparatus of claim 4 additionally comprising means for generating a set of icons after said digitized video data is segmented into paragraphs according to said set of rules.
Claim 24
Dependent Claim
17 words
The apparatus of claim 21 wherein said set of icons is a set of intelligent moving icons.
Claim 25
Dependent Claim
19 words
The apparatus of claim 22 wherein said means for generating said set of intelligent moving icons uses data-dependent heuristics.
Claim 26
Dependent Claim
32 words
The apparatus of claim 4 additionally comprising means for compressing said digitized video data before said indexed audio data and said digitized video data are stored with their respective sets of time-stamps.
Claim 27
Dependent Claim
97 words
The apparatus of claim 4 wherein said means for transcribing said audio data and marking said transcribed audio data with a first set of time stamps comprises: means for producing a set of possible word occurrences, with each word occurrence having a start time and a plurality of possible end times; means for producing a plurality of possible begin times for each of said end times; means for generating a set of N-best hypotheses for said audio data; and means for selecting a best-scoring hypothesis from said set of N-best hypotheses to produce said transcribed audio data.
Claim 28
Dependent Claim
23 words
The apparatus of claim 25 wherein said means for producing said set of possible word occurrences uses a forward time synchronous pass function.
Claim 29
Dependent Claim
22 words
The apparatus of claim 25 wherein said means for producing said plurality of possible begin times uses a reverse time synchronous function.
Claim 30
Dependent Claim
63 words
The apparatus of claim 5 wherein said means for passing said transcribed audio data through a natural language interpreter before indexing said transcribed audio data comprises: means for summarizing said transcribed audio data; means for tagging said transcribed audio data using data extraction techniques; and means for correcting said tagged transcribed audio data using semantic and syntactic constraints and a phonetic knowledge base.
Claim 31
Dependent Claim
22 words
The apparatus of claim 4 wherein said means for segmenting said digitized video data into video paragraphs uses comprehensive image statistic rules.
Claim 32
Dependent Claim
21 words
The apparatus of claim 4 wherein said means for segmenting said digitized video data into video paragraphs uses camera motion rules.
Claim 33
Dependent Claim
21 words
The apparatus of claim 4 wherein said means for segmenting said digitized video data into video paragraphs uses object motion rules.
Claim 34
Dependent Claim
24 words
The apparatus of claim 4 wherein said means for segmenting said digitized video data into video paragraphs uses deformable templates and an filtering rules.
Claim 35
Dependent Claim
21 words
The apparatus of claim 4 wherein said means for segmenting said digitized video data into video paragraphs uses object presence rules.
Claim 36
Dependent Claim
21 words
The apparatus of claim 4 wherein said means for segmenting said digitized video data into video paragraphs uses three-dimensional understanding rules.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-11-09 | 2002-05-11 | 2002-11-10 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-11-09 | 2006-05-11 | 2006-11-10 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-11-09 | 2010-05-11 | 2010-11-10 | $7700.00 | Window Closed / Lapsed |