Patent US6990448B2 successfully refreshed from Google Patents.
US6990448B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Database annotation and retrieval including phoneme data
Assignee: Canon Inc
•
Filing: 2001-08-23
•
Granted: 2006-01-24
•
Expires: 2021-08-23
Abstract
Abstract The data structure is used in accessing a plurality of data files. The data stucture comprises a plurality of annotation storage areas adapted to correspond with the data files, each annotation storage area containing an annotation an annotation representing a time sequential signal and each annotation storage area comprising a plurality of block storage areas each containing phoneme and word data forming a respective temporal block of the annotation and each block having an associated time index identifying a timing of the block within the corresponding annotation. Each block storage area includes a plurality of node storage areas, each asociated with a node which represents a point in time at which a word and/or phoneme begins or ends within the corresponding annotation, and each node storage area having a time offset storage area containing a time offset defining the point in time represented by the node relative to the time index associated with the corresponding block. Each node storage area further has one or more phoneme link storage areas, each having a phoneme storage area containing data identifying a phoneme associated with corresponding node; or one or more word link storage areas, each having a word storage area containing data identifying a word associated with the corresponding node. One or more of the node storage areas has at least one of the phoneme link storage areas and at least one of the word link storage areas.
Inventors
- 👤 Jason Peter Andrew Charlesworth
- 👤 Philip Neil Garner
- 👤 Jebu Jacob Rajan
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 12:37
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/08; G10L15/18; G10L15/183; G10L15/187; G06; G06F; G06F16/00; G06F16/40; G06F16/48; G06F16/60; G06F16/68; G06F16/683; G06F16/685; G06F16/70; G06F16/78; G06F16/783; G06F16/7844
Claims Analysis
Total Claims: 58
Claim 1
Independent Claim
140 words
An apparatus for generating annotation data for annotating a data file comprising audio data, the apparatus comprising: an automatic speech recognizer operable to generate phoneme data for audio data in the data file; a word decoder operable to process the phoneme data generated by the automatic speech recognizer to identify words within the phoneme data; an annotation generator operable to generate annotation data for the data file by combining the generated phoneme data and the words identified by said word decoder, the annotation data being for use in subsequent retrieval of the data file; an associator operable to associate the generated annotation data with the data file; and a storage device operable to store the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 2
Independent Claim
140 words
An apparatus for generating annotation data for annotating a data file comprising audio data, the apparatus comprising: an automatic speech recognizer operable to generate phoneme data for audio data in the data file; a word decoder operable to process the phoneme data generated by the automatic speech recognizer to identify words within the phoneme data; an annotation generator operable to generate annotation data for the data file by combining the generated phoneme data and the words identified by said word decoder, the annotation data being for use in subsequent retrieval of the data file; an associator operable to associate the generated annotation data with the data file; and a storage device operable to store the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 3
Dependent Claim
96 words
An apparatus according to claim 1 , wherein the annotation data defines a phoneme and word lattice and wherein said annotation generator comprises: (i) a first generator operable to generate data defining a plurality of nodes within the lattice and a plurality of links connecting the nodes within the lattice; (ii) a second generator operable to generate data associating a plurality of phonemes of the phoneme data with a respective plurality of links; and (iii) a third generator operable to generate data associating at least one of the words with at least one of the links.
Claim 4
Dependent Claim
28 words
An apparatus according to claim 2 , wherein said annotation generator is operable to generate the data defining the phoneme and word lattice in blocks of the nodes.
Claim 5
Dependent Claim
24 words
An apparatus according to claim 3 , wherein said annotation generator is operable to generate data which defines each block's location within the database.
Claim 6
Dependent Claim
25 words
An apparatus according to claim 2 , wherein said annotation generator is operable to generate data defining time stamp information for each of the nodes.
Claim 7
Dependent Claim
27 words
An apparatus according to claim 5 , wherein said annotation generator is operable to generate the phoneme and word lattice data in blocks of equal time duration.
Claim 8
Dependent Claim
37 words
An apparatus according to claim 5 , wherein said data file includes a time sequential signal, and wherein said annotation generator is operable to generate time stamp data which is time synchronized with the time sequential signal.
Claim 9
Dependent Claim
33 words
An apparatus according to claim 2 , wherein said first generator is operable to define at least one node which is connected to a plurality of other nodes by a plurality of links.
Claim 10
Dependent Claim
47 words
An apparatus according to claim 1 , wherein the data file includes audio data which defines the speech of a plurality of speakers, and wherein said annotation generator is operable to generate data which defines separate phoneme and word annotation data for the speech of each speaker.
Claim 11
Dependent Claim
25 words
An apparatus according to claim 1 , wherein said speech recognizer is operable to generate data defining a weighting for phonemes in the phoneme data.
Claim 12
Dependent Claim
27 words
An apparatus according to claim 1 , wherein said word decoder is operable to generate data defining a weighting for the words identified within the phoneme data.
Claim 13
Dependent Claim
31 words
An apparatus according to claim 1 , wherein said annotation generator is operable to generate annotation data that defines a phoneme and word lattice arranged in a time-ordered sequence of blocks.
Claim 14
Dependent Claim
78 words
An apparatus according to claim 12 , wherein the phoneme and word lattice is associated with a time-sequential signal, and wherein said annotation generator is operable to generate the phoneme and word lattice so that: (i) the time-ordered sequence of blocks of the phoneme and word lattice is time-synchronized with the time-sequential signal; and (ii) each block of the phoneme and word lattice includes an associated time index identifying a timing of the block within the time-sequential signal.
Claim 15
Dependent Claim
74 words
An apparatus according to claim 13 , wherein said annotation generator is operable to generate node data which defines a plurality of nodes within the lattice, each node representing a point in time at which a word and/or phoneme begins or ends within the associated time-sequential signal and wherein each node includes a time-offset value defining the point in time represented by the node relative to the time index associated with the corresponding block.
Claim 16
Dependent Claim
66 words
An apparatus according to claim 1 , wherein said automatic speech recognizer is operable to generate phoneme data that includes a phoneme lattice which identifies a number of different possible phoneme strings which correspond to speech within the audio data, and wherein said annotation generator is operable to generate the annotation data by combining the generated phoneme lattice with the words identified by said word decoder.
Claim 17
Dependent Claim
30 words
An apparatus according to claim 1 , wherein said annotation generator is operable to generate header data relating to the speech recognizer that generated the phoneme data for the annotation.
Claim 18
Dependent Claim
32 words
An apparatus according to claim 16 , when said annotation generator is operable to generate header data including data identifying the language and the phoneme sets used by said automatic speech recognizer.
Claim 19
Independent Claim
135 words
An apparatus for generating annotation data for annotating a data file, the apparatus comprising; an input operable to receive an input voice signal; a speech recognizer operable (i) to convert the input voice signal into phoneme data and (ii) to process the phoneme data to identify words within the phoneme data; an annotation generator operable to generate annotation data for the data file by combining the phoneme data and the words obtained by said speech recognizer, the annotation data being for use in subsequent retrieval of the data file; an associator operable to associate the generated annotation data with the data file; and a storage device operable to store the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 20
Independent Claim
135 words
An apparatus for generating annotation data for annotating a data file, the apparatus comprising; an input operable to receive an input voice signal; a speech recognizer operable (i) to convert the input voice signal into phoneme data and (ii) to process the phoneme data to identify words within the phoneme data; an annotation generator operable to generate annotation data for the data file by combining the phoneme data and the words obtained by said speech recognizer, the annotation data being for use in subsequent retrieval of the data file; an associator operable to associate the generated annotation data with the data file; and a storage device operable to store the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 21
Dependent Claim
25 words
An apparatus according to claim 18 , wherein said speech recognizer is operable to generate data defining a weighting for phonemes in the phoneme data.
Claim 22
Dependent Claim
22 words
An apparatus according to claim 19 , wherein said speech recognizer is operable to generate data defining a weighting for the words.
Claim 23
Dependent Claim
66 words
An apparatus according to claim 18 , wherein said speech recognizer is operable to generate phoneme data that includes a phoneme lattice which identifies a number of different possible phoneme strings which correspond to speech within the input voice signal, and wherein said annotation generator is operable to generate the annotation data by combining the generated phoneme lattice with the words identified by said speech recognizer.
Claim 24
Dependent Claim
30 words
An apparatus according to claim 18 , wherein said annotation generator is operable to generate header data relating to the speech recognizer that generated the phoneme data for the annotation.
Claim 25
Dependent Claim
31 words
An apparatus according to claim 22 , wherein said annotation generator is operable to generate header data including data identifying the language and the phoneme sets used by said speech recognizer.
Claim 26
Independent Claim
132 words
An apparatus for generating annotation data for annotating a data file, the apparatus comprising: an input operable to receive image data representative of text; a character recognizer operable to convert the image data into text data; a converter operable to convert words in the text data into phoneme data; and an annotation generator operable to generate annotation data for the data file by combining the phoneme data and words in the text data, the annotation data being for use in subsequent retrieval of the data file; an associator operable to associate the generated annotation data with the data file; and a storage device operable to store the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 27
Independent Claim
132 words
An apparatus for generating annotation data for annotating a data file, the apparatus comprising: an input operable to receive image data representative of text; a character recognizer operable to convert the image data into text data; a converter operable to convert words in the text data into phoneme data; and an annotation generator operable to generate annotation data for the data file by combining the phoneme data and words in the text data, the annotation data being for use in subsequent retrieval of the data file; an associator operable to associate the generated annotation data with the data file; and a storage device operable to store the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 28
Dependent Claim
32 words
An apparatus according to claim 24 , wherein said converter comprises an automatic phonetic transcription unit which generates the phoneme data from words within the text data output by said character recognizer.
Claim 29
Dependent Claim
18 words
An apparatus according to claim 24 , wherein said input comprises a document scanner or a facsimile machine.
Claim 30
Independent Claim
129 words
A method of generating annotation data for annotating a data file comprising audio data, the method comprising the steps of: using an automatic speech recognizer to generate phoneme data for audio data in the data file; using a word decoder to process the phoneme data generated by the automatic speech recognizer to identify words within the phoneme data; generating annotation data for the data file by combining the generated phoneme data and the words identified by the word decoder, the annotation data being for use in subsequent retrieval of the data file; associating the generated annotation data with the data file; and storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 31
Independent Claim
129 words
A method of generating annotation data for annotating a data file comprising audio data, the method comprising the steps of: using an automatic speech recognizer to generate phoneme data for audio data in the data file; using a word decoder to process the phoneme data generated by the automatic speech recognizer to identify words within the phoneme data; generating annotation data for the data file by combining the generated phoneme data and the words identified by the word decoder, the annotation data being for use in subsequent retrieval of the data file; associating the generated annotation data with the data file; and storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 32
Dependent Claim
87 words
A method according to claim 27 , wherein the annotation data defines a phoneme and word lattice and wherein said step of generating annotation data comprises the steps of: (i) generating data defining a plurality of nodes within the lattice and a plurality of links connecting the nodes within the lattice; (ii) generating data associating a plurality of phonemes of the phoneme data with a respective plurality of links; and (iii) generating data associating at least one of the words with at least one of the links.
Claim 33
Dependent Claim
25 words
A method according to claim 28 , wherein said step of generating annotation data generates data defining time stamp information for each of the nodes.
Claim 34
Dependent Claim
24 words
A method according to claim 29 , wherein said step of generating annotation data generates data which defines a block's location within a database.
Claim 35
Dependent Claim
37 words
A method according to claim 29 , wherein the data file includes a time sequential signal, and wherein said step of generating annotation data generates time stamp data which is time synchronized with the time sequential signal.
Claim 36
Dependent Claim
23 words
A method according to claim 28 , wherein the speech recognizer generates data defining a weighting for the phonemes associated with the links.
Claim 37
Dependent Claim
23 words
A method according to claim 28 , wherein the word decoder generates data defining a weighting for the words associated with the links.
Claim 38
Dependent Claim
40 words
A method according to claim 28 , wherein said step of defining a plurality of nodes and a plurality of links defines at least one node which is connected to a plurality of other nodes by a plurality of links.
Claim 39
Dependent Claim
27 words
A method according to claim 28 , wherein said step of generating annotation data generates data defining the phoneme and word lattice in blocks of the nodes.
Claim 40
Dependent Claim
44 words
A method according to claim 27 , wherein the audio data includes audio data which defines the speech of a plurality of speakers, and wherein said generating step generates data which defines separate phoneme and word annotation data for the speech of each speaker.
Claim 41
Independent Claim
120 words
A method of generating annotation data for annotating a data file, the method comprising the steps of: receiving an input voice signal; processing the input voice signal using a speech recognizer to generate phoneme data for the input voice signal; processing the generated phoneme data to identify words within the phoneme data; generating annotation data by combining the phoneme data and the word data generated for the input voice signal, the annotation data being for use in subsequent retrieval of the data file; associating the generated annotation data with the data file; and storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 42
Independent Claim
120 words
A method of generating annotation data for annotating a data file, the method comprising the steps of: receiving an input voice signal; processing the input voice signal using a speech recognizer to generate phoneme data for the input voice signal; processing the generated phoneme data to identify words within the phoneme data; generating annotation data by combining the phoneme data and the word data generated for the input voice signal, the annotation data being for use in subsequent retrieval of the data file; associating the generated annotation data with the data file; and storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 43
Dependent Claim
21 words
A method according to claim 37 , wherein the speech recognizer generates data defining a weighting for the generated phoneme data.
Claim 44
Dependent Claim
29 words
A method according to claim 37 , wherein said step of processing the generated phoneme data generates data defining a weighting for the words identified within the phoneme data.
Claim 45
Independent Claim
113 words
A method of generating annotation data for use in annotating a data file, the method comprising the steps of: receiving image data representative of text; converting the image data into text data using a character recognizer; converting words in the text data into phoneme data; generating annotation data for the data file by combining the phoneme data and words within the text data, the annotation data being for use in subsequent retrieval of the data file; associating the generated annotation data with the data file; and storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 46
Independent Claim
113 words
A method of generating annotation data for use in annotating a data file, the method comprising the steps of: receiving image data representative of text; converting the image data into text data using a character recognizer; converting words in the text data into phoneme data; generating annotation data for the data file by combining the phoneme data and words within the text data, the annotation data being for use in subsequent retrieval of the data file; associating the generated annotation data with the data file; and storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 47
Dependent Claim
38 words
A method according to claim 40 , wherein said step of converting words into phoneme data uses an automatic phonetic transcription unit which generates the phoneme data for words within the text data output by the character recognizer.
Claim 48
Dependent Claim
19 words
A method according to claim 40 , wherein said receiving step uses a document scanner or a facsimile machine.
Claim 49
Dependent Claim
153 words
A computer readable medium storing computer program code for causing a programmable processing apparatus to become operable to generate annotation data for annotating a data file comprising audio data, the computer readable medium comprising: code for causing an automatic speech recognizer to generate phoneme data for audio data in the data file; code for causing a word decoder to process the phoneme data generated by the automatic speech recognizer to identify words within the phoneme data; code for generating annotation data for the data file by combining the generated phoneme data and the words identified by the word decoder, the annotation data being for use in subsequent retrieval of the data file; code for associating the generated annotation data with the data file; and code for storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 50
Dependent Claim
153 words
A computer readable medium storing computer program code for causing a programmable processing apparatus to become operable to generate annotation data for annotating a data file comprising audio data, the computer readable medium comprising: code for causing an automatic speech recognizer to generate phoneme data for audio data in the data file; code for causing a word decoder to process the phoneme data generated by the automatic speech recognizer to identify words within the phoneme data; code for generating annotation data for the data file by combining the generated phoneme data and the words identified by the word decoder, the annotation data being for use in subsequent retrieval of the data file; code for associating the generated annotation data with the data file; and code for storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 51
Dependent Claim
134 words
A computer readable medium storing computer executable instructions for causing a programmable processing apparatus to become operable to generate annotation data for annotating a data file, the medium comprising: code for receiving an input voice signal; code for causing an automatic speech recognizer to convert the input voice signal into phoneme data; code for processing the phoneme data to identify words within the phoneme data; code for generating annotation data by combining the generated phoneme data and the identified words, the annotation data being for use in subsequent retrieval of the data file; code for associating the generated annotation data with the data file; and code for storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 52
Dependent Claim
134 words
A computer readable medium storing computer executable instructions for causing a programmable processing apparatus to become operable to generate annotation data for annotating a data file, the medium comprising: code for receiving an input voice signal; code for causing an automatic speech recognizer to convert the input voice signal into phoneme data; code for processing the phoneme data to identify words within the phoneme data; code for generating annotation data by combining the generated phoneme data and the identified words, the annotation data being for use in subsequent retrieval of the data file; code for associating the generated annotation data with the data file; and code for storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 53
Dependent Claim
135 words
A computer readable medium storing computer executable instructions for causing a programmable processing apparatus to become operable to generate annotation data for annotating a data file, the medium comprising: code for receiving image data representative of text; code for converting the image data into text data using a character recognizer; code for converting words in the text data into phoneme data; code for generating annotation data for the data file by combining the phoneme data and words within the text data, the annotation data being for use in subsequent retrieval of the data file; code for associating the generated annotation data with the data file; and code for storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 54
Dependent Claim
135 words
A computer readable medium storing computer executable instructions for causing a programmable processing apparatus to become operable to generate annotation data for annotating a data file, the medium comprising: code for receiving image data representative of text; code for converting the image data into text data using a character recognizer; code for converting words in the text data into phoneme data; code for generating annotation data for the data file by combining the phoneme data and words within the text data, the annotation data being for use in subsequent retrieval of the data file; code for associating the generated annotation data with the data file; and code for storing the annotation data in a database to allow the associated data file to be retrieved by searching for words and phonemes within the stored annotation data.
Claim 55
Independent Claim
111 words
An apparatus for generating annotation data for use in annotating a data file comprising audio data, the apparatus comprising: an automatic speech recognizer operable to generate phoneme data for audio data in the data file; a word decoder operable to identify words within the phoneme data generated by said automatic speech recognizer; and an annotation generator operable to generate annotation data by combining the generated phoneme data and the words identified by said word decoder and operable to generate header data relating to the speech recognizer that generated the phoneme data for the annotation, which header data includes data identifying the language and phoneme set used by said automatic speech recognizer.
Claim 56
Independent Claim
111 words
An apparatus for generating annotation data for use in annotating a data file comprising audio data, the apparatus comprising: an automatic speech recognizer operable to generate phoneme data for audio data in the data file; a word decoder operable to identify words within the phoneme data generated by said automatic speech recognizer; and an annotation generator operable to generate annotation data by combining the generated phoneme data and the words identified by said word decoder and operable to generate header data relating to the speech recognizer that generated the phoneme data for the annotation, which header data includes data identifying the language and phoneme set used by said automatic speech recognizer.
Claim 57
Independent Claim
93 words
An apparatus for generating annotation data for use in annotating a data file, the apparatus comprising: an input operable to receive an input voice signal; a speech recognizer operable to convert the input voice signal into phoneme data and words; and an annotation generator operable to generate annotation data by combining the phoneme data and the words and operable to generate header data relating to the speech recognizer that generated the phoneme data for the annotation, which header data includes data identifying the language and the phoneme set used by the speech recognizer.
Claim 58
Independent Claim
93 words
An apparatus for generating annotation data for use in annotating a data file, the apparatus comprising: an input operable to receive an input voice signal; a speech recognizer operable to convert the input voice signal into phoneme data and words; and an annotation generator operable to generate annotation data by combining the phoneme data and the words and operable to generate header data relating to the speech recognizer that generated the phoneme data for the annotation, which header data includes data identifying the language and the phoneme set used by the speech recognizer.
References Cited (Backward Citations)
US4227176A
Cited
US4736429A
Cited
US5131043A
Cited
US4975959A
Cited
US4980918A
Cited
US4903305A
Cited
US4985924A
Cited
US5075896A
Cited
US6236964B1
Cited
US5136655A
Cited
US5202952A
Cited
US5390278A
Cited
US5333275A
Cited
US5594641A
Cited
US5577249A
Cited
EP0597798A1
Cited
US5638425A
Cited
US5640487A
Cited
US5787414A
Cited
US5500920A
Cited
US5649060A
Cited
EP0649144A1
Cited
US5752227A
Cited
EP0689153A2
Cited
US6272242B1
Cited
US5799267A
Cited
US5737723A
Cited
US5835667A
Cited
US6389395B1
Cited
US5680605A
Cited
US5999902A
Cited
US5781884A
Cited
US5675706A
Cited
US5729741A
Cited
US5873061A
Cited
US6070140A
Cited
US6023536A
Cited
US5721939A
Cited
US5684925A
Cited
US5737489A
Cited
US5907821A
Cited
US6567778B1
Cited
EP0789349A2
Cited
GB2302199A
Cited
US5870740A
Cited
US5708759A
Cited
US6172675B1
Cited
US5852822A
Cited
EP0849723A2
Cited
US6122613A
Cited
WO1998047084A1
Cited
WO1999005681A1
Cited
US6487532B1
Cited
GB2331816A
Cited
US6026398A
Cited
US6061679A
Cited
US5983177A
Cited
US6289140B1
Cited
US6182039B1
Cited
US6243680B1
Cited
US6321226B1
Cited
US6192337B1
Cited
US6490563B2
Cited
US6314400B1
Cited
WO2000031723A1
Cited
WO2000054168A2
Cited
US6463413B1
Cited
GB2349260A
Cited
US6662180B1
Cited
US6567816B1
Cited
US6535850B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-01-23 | 2009-07-25 | 2010-01-24 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-01-23 | 2013-07-25 | 2014-01-24 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-01-23 | 2017-07-25 | 2018-01-24 | $7700.00 | Window Closed / Lapsed |