Patent US6434520B1 successfully refreshed from Google Patents.
US6434520B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for indexing and querying audio archives
Assignee: Nuance Communications Inc
•
Filing: 1999-04-16
•
Granted: 2002-08-13
•
Expires: 2019-04-16
Abstract
Abstract A system and method for indexing segments of audio/multimedia files and data streams for storage in a database according to audio information such as speaker identity, the background environment and channel (music, street noise, car noise, telephone, studio noise, speech plus music, speech plus noise, speech over speech), and/or the transcription of the spoken utterances. The content or topic of the transcribed text can also be determined using natural language understanding to index based on the context of the transcription. A user can then retrieve desired segments of the audio file from the database by generating a query having one or more desired parameters based on the indexed information.
Inventors
- 👤 Dimitri Kanevsky
- 👤 Stephane H. Maes
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 18:21
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/26; G06; G06F; G06F16/00; G06F16/60; G06F16/68; G06F16/683; G06F16/685; G10L17/00
Claims Analysis
Total Claims: 37
Claim 1
Independent Claim
86 words
A method for processing an audio data file, comprising the steps of: segmenting the audio data file into segments based on detected speaker changes; performing speaker identification for each segment and assigning at least one speaker identification tag to each segment based on an identified speaker; verifying the identity of the speaker associated with the at least one identification tag for each segment; and indexing the segments of the audio data file for storage in a database in accordance with the identification tags of verified speakers.
Claim 2
Independent Claim
86 words
A method for processing an audio data file, comprising the steps of: segmenting the audio data file into segments based on detected speaker changes; performing speaker identification for each segment and assigning at least one speaker identification tag to each segment based on an identified speaker; verifying the identity of the speaker associated with the at least one identification tag for each segment; and indexing the segments of the audio data file for storage in a database in accordance with the identification tags of verified speakers.
Claim 3
Dependent Claim
19 words
The method of claim 1 , further including the step of enrolling an unknown speaker of a corresponding segment.
Claim 4
Dependent Claim
33 words
The method of claim 1 , further including the step of retrieving at least one segment from the database in accordance with a user query based on an identity of a desired speaker.
Claim 5
Dependent Claim
49 words
The method of claim 3 , wherein the user query is a speaker identification tag and the retrieving step includes the step of selecting for output at least one segment which is indexed with at least one speaker identification tag that matches the speaker identification tag of the query.
Claim 6
Dependent Claim
33 words
The method of claim 3 , further including the steps of: generating a voiceprint for each segment of the audio data file; and storing each voiceprint with its corresponding segment in the database.
Claim 7
Dependent Claim
58 words
The method of claim 5 , wherein the user query is a voiceprint associated with the desired speaker, and wherein the retrieving step includes the steps of: comparing the input speaker voiceprint with each of the stored voiceprints of the segments; and selecting at least one segment having a corresponding voiceprint stored therewith that matches the input voiceprint.
Claim 8
Dependent Claim
65 words
The method of claim 5 , wherein the user query is an audio segment of the desired speaker, and wherein the retrieving step includes the steps of: generating a voiceprint from the input audio segment; comparing the generated voiceprint with each of the stored voiceprints of the segments; and selecting at least one segment having a corresponding voiceprint stored therewith that matches the generated voiceprint.
Claim 9
Dependent Claim
24 words
The method of claim 5 , further including the step of storing for each segment one of a corresponding waveform, acoustic features, and both.
Claim 10
Dependent Claim
77 words
The method of claim 8 , wherein the user query is an audio segment of the desired speaker, and wherein the retrieving step includes the steps of: comparing the audio segment with one of the stored waveforms and stored acoustic features of each of the segments in the database; and selecting at least one segment having a corresponding one of a waveform and acoustic features stored therewith that match the audio segment of the speaker of interest.
Claim 11
Dependent Claim
27 words
The method of claim 9 , wherein the audio segment of the speaker of interest is one of input by the user and selected from the database.
Claim 12
Dependent Claim
79 words
The method of claim 1 , further including the steps of: segmenting the audio data file into segments based on detected changes in environment; and identifying at least one environment of each segment and assigning at least one environment tag to each segment corresponding to the at least one identified environment; wherein the indexing step further includes indexing the segments of the audio data file for storage in the database in accordance with the environment tags of the segments.
Claim 13
Dependent Claim
29 words
The method of claim 11 , wherein the step of detecting changes in environment includes detecting changes in one of a background noise, a channel, and a combination thereof.
Claim 14
Dependent Claim
44 words
The method of claim 11 , including the step of retrieving at least one segment from the database in accordance with a user query based on one of an identity of a desired speaker, the identity of a desired environment, and a combination thereof.
Claim 15
Dependent Claim
29 words
The method of claim 1 , further including the steps of: recognizing spoken words of each segment; and storing the recognized words for each corresponding segment in the database.
Claim 16
Dependent Claim
47 words
The method of claim 14 , wherein the recognizing step includes the steps of: identifying one of channel acoustic components, background acoustic components, and a combination thereof, for each segment; and decoding the spoken words of each segment using trained models based on the identified acoustic components.
Claim 17
Dependent Claim
59 words
The method of claim 14 , further including the steps of: performing natural language understanding (NLU) of the recognized words of each segment to determine at least one NLU topic of each segment; wherein the indexing step further includes indexing the segments of the audio data file for storage in the database in accordance with the determined NLU topics.
Claim 18
Dependent Claim
55 words
The method of claim 16 , including the step of retrieving at least one segment from the database in accordance with a user query based on one of an identity of a speaker of interest, at least one user-selected keyword, context of the recognized words text, at least one NLU topic, and a combination thereof.
Claim 19
Dependent Claim
106 words
A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for processing an audio data file, the method steps comprising: segmenting the audio data file into segments based on detected speaker changes; performing speaker identification for each segment and assigning at least one speaker identification tag to each segment based on an identified speaker; verifying the identity of the speaker associated with the at least one identification tag for each segment; and indexing the segments of the audio data file for storage in a database in accordance with the identification tags of verified speakers.
Claim 20
Dependent Claim
106 words
A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for processing an audio data file, the method steps comprising: segmenting the audio data file into segments based on detected speaker changes; performing speaker identification for each segment and assigning at least one speaker identification tag to each segment based on an identified speaker; verifying the identity of the speaker associated with the at least one identification tag for each segment; and indexing the segments of the audio data file for storage in a database in accordance with the identification tags of verified speakers.
Claim 21
Dependent Claim
38 words
The program storage device of claim 18 , further including instructions for performing the step of retrieving at least one segment from the database in accordance with a user query based on an identity of a desired speaker.
Claim 22
Dependent Claim
87 words
The program storage device of claim 18 , further including instructions for performing the steps of: segmenting the audio data file into segments based on detected changes in environment; and identifying at least one environment of each segment to assign at least one environment tag to each segment corresponding to the at least one identified environment; wherein the instructions for performing the indexing step further include instructions for indexing the segments of the audio data file for storage in the database in accordance with the environment tags.
Claim 23
Dependent Claim
30 words
The program storage device of claim 20 , wherein the step of detecting changes in environment includes detecting changes in one of a background, a channel, and a combination thereof.
Claim 24
Dependent Claim
50 words
The program storage device of claim 20 , further including instructions for performing the step of retrieving at least one segment from the database in accordance with a user query based on one of an identity of a desired speaker, the identity of a desired environment, and a combination thereof.
Claim 25
Dependent Claim
34 words
The program storage device of claim 18 , further including instructions for performing the steps of: recognizing spoken words of each segment; and storing the recognized words for each corresponding segment in the database.
Claim 26
Dependent Claim
56 words
The program storage device of claim 23 , wherein the instructions for performing the recognizing step include instructions for performing the steps of: identifying one of channel acoustic components, background acoustic components, and a combination thereof, for each segment; and decoding the spoken words of each segment using trained models based on the identified acoustic components.
Claim 27
Dependent Claim
69 words
The program storage device of claim 23 , further including instruction for performing the steps of: performing natural language understanding (NLU) of the recognized words of each segment to determine at least one NLU topic of each segment; wherein the instructions for performing the indexing step include instructions for indexing the segments of the audio data file for storage in the database in accordance with the determined NLU topics.
Claim 28
Dependent Claim
61 words
The program storage device of claim 25 , further including instructions for performing the step of retrieving at least one segment from the database in accordance with a user query based on one of an identity of a speaker of interest, at least one user-selected keyword, context of the recognized words text, at least one NLU topic, and a combination thereof.
Claim 29
Independent Claim
93 words
A system for managing a database of audio data files, comprising: a segments for dividing an input audio data file into segments by detecting speaker changes in the input audio data file; a speaker identifier for identifying a speaker of each segment and assigning at least one identity tag to each segment; a speaker verifier for verifying the at least one identity tag of each segment; and an indexer for indexing the segments of the input audio data file for storage in the database in accordance with the identity tags of verified speakers.
Claim 30
Independent Claim
93 words
A system for managing a database of audio data files, comprising: a segments for dividing an input audio data file into segments by detecting speaker changes in the input audio data file; a speaker identifier for identifying a speaker of each segment and assigning at least one identity tag to each segment; a speaker verifier for verifying the at least one identity tag of each segment; and an indexer for indexing the segments of the input audio data file for storage in the database in accordance with the identity tags of verified speakers.
Claim 31
Dependent Claim
33 words
The system of claim 27 , further comprising a search engine for retrieving at least one segment from the database by processing a user query based on an identity of a desired speaker.
Claim 32
Dependent Claim
82 words
The system of claim 27 , wherein the segments divides the audio data file into segments based on detected changes in environment, and wherein the system further comprises an environment identifier for identifying at least one environment of each segment and assigning at least one environment tag to each segment corresponding to the at least one identified environment, and wherein the indexer further indexes the segments of the audio data file for storage in the database in accordance with the environment tags.
Claim 33
Dependent Claim
20 words
The system of claim 29 , wherein the detected environment changes include one of background, channels, and a combination thereof.
Claim 34
Dependent Claim
66 words
The system of claim 29 , further comprising a search engine for retrieving at least one segment from the database indexed by one of an identity of a desired speaker, the identity of a desired environment, and a combination thereof, by processing a user query based on one of the identity of a desired speaker, the identity of a desired is environment, and a combination thereof.
Claim 35
Dependent Claim
36 words
The system of claim 27 , further comprising: a speech recognizer for recognizing spoken words of each segment, wherein the recognized words for each segment are stored in the database and indexed to the corresponding segment.
Claim 36
Dependent Claim
55 words
The system of claim 32 , further comprising means for performing natural language understanding (NLU) of the recognized words of each segment to determine at least one NLU topic of each segment, wherein the indexer indexes the segments of the audio data file for storage in the database in accordance with the determined NLU topics.
Claim 37
Dependent Claim
56 words
The system of claim 33 , further comprising a search engine for retrieving at least one segment from the database by processing a user query based on one of an identity of a speaker of interest, at least one user-selected keyword, context of the recognized words text, at least one NLU topic, and a combination thereof.
References Cited (Backward Citations)
US3936805A
Cited
WO1992011634A1
Cited
US5465290A
Cited
EP0507743A2
Cited
US5550966A
Cited
US5649060A
Cited
US5655058A
Cited
US5606643A
Cited
US5598507A
Cited
US5659662A
Cited
US5737532A
Cited
US5774841A
Cited
US5937383A
Cited
US5918223A
Cited
US5960399A
Cited
US5897616A
Cited
US6161090A
Cited
US6185527B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-08-12 | 2006-02-11 | 2006-08-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-08-12 | 2010-02-11 | 2010-08-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-08-12 | 2014-02-11 | 2014-08-13 | $7700.00 | Window Closed / Lapsed |