US5809462A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for interfacing and training a neural network for phoneme recognition
Assignee: Ericsson Messaging Systems Inc
•
Filing: 1997-02-28
•
Granted: 1998-09-15
•
Expires: 2017-02-28
Abstract
Abstract An automated speech recognition system converts a speech signal into a compact, coded representation that correlates to a speech phoneme set. A number of different neural network pattern matching schemes may be used to perform the necessary speech coding. An integrated user interface guides a user unfamiliar with the details of speech recognition or neural networks to quickly develop and test a neural network for phoneme recognition. To train the neural network, digitized voice data containing known phonemes that the user wants the neural network to ultimately recognize are processed by the integrated user interface. The digitized speech is segmented into phonemes with each segment being labelled with a corresponding phoneme code. Based on a user selected transformation method and transformation parameters, each segment is transformed into a series of multiple dimension vectors representative of the speech characteristics of that segment. These vectors are iteratively presented to a neural network to train/adapt that neural network to consistently distinguish and recognize these vectors and assign an appropriate phoneme code to each vector. Simultaneous display of the digitized speech, segments, vector sets, and a representation of the trained neural network assist the user in visually confirming the acceptability of the phoneme training set. A user may also selectively audibly confirm the acceptability of the digitization scheme, the segments, and the transform vectors so that satisfactory training data are presented to the neural network. If the user finds a particular step or parameter produces an unacceptable result, the user may modify one or more of the parameters and verify whether the modification effected an improvement in performance. The trained neural network is also automatically tested by presenting a test speech signal to the integrated user interface and observing both audibly and visually automatic segmentation of the speech, transformation into multidimensional vectors, and the resulting neural network assigned phoneme codes. A method of decoding such phoneme codes using the neural network is also disclosed.
Inventors
- 👤 Paul A. Nussbaum
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:56
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/06; G10L15/063; G10L15/08; G10L15/16; G10L15/02; G10L2015/025; G10L2015/0638
Claims Analysis
Total Claims: 33
Claim 1
Independent Claim
77 words
A method for training a neural network comprising the steps of: inputting an audio signal including a training set of phonemes; dividing the signal into segments, each segment representing a phoneme; identifying each segment; transforming segments into time independent vectors; verifying that at least one of the segments or vectors represents to a degree acceptable to an operator a corresponding one of the training set of phonemes; and training the neural network using the time independent vectors.
Claim 2
Independent Claim
77 words
A method for training a neural network comprising the steps of: inputting an audio signal including a training set of phonemes; dividing the signal into segments, each segment representing a phoneme; identifying each segment; transforming segments into time independent vectors; verifying that at least one of the segments or vectors represents to a degree acceptable to an operator a corresponding one of the training set of phonemes; and training the neural network using the time independent vectors.
Claim 3
Dependent Claim
25 words
The method according to claim 1, wherein the trained neural network generates a centroid or a centroid estimate for each phoneme based on corresponding vectors.
Claim 4
Dependent Claim
61 words
The method according to claim 2, further comprising: displaying the audio signal, segments, vectors, and centroids or centroid estimates, wherein the operator visually confirms the acceptability of any one of the segments, vectors, and centroids or centroid estimates based on a degree of visual similarity between any combination of corresponding portions of the displayed audio signal, segments, vectors, or centroids/centroid estimates.
Claim 5
Dependent Claim
66 words
The method according to claim 2, further comprising: selectively reproducing a sound corresponding to one of the audio signal, segments, vectors, and centroids or centroid estimates to audibly confirm the acceptability of any one of the segments, vectors, and centroids or centroid estimates based on a degree of audible similarity between any combination of corresponding segments, vectors, or centroids/centroid estimates with the training set of phonemes.
Claim 6
Dependent Claim
41 words
The method according to claim 1, wherein the dividing step includes displaying a digitized version of the input audio signal and the operator manually divides using a cursor the displayed signal into segments that correspond to each of the training phonemes.
Claim 7
Dependent Claim
47 words
The method according to claim 1, wherein the identifying step includes the operator manually assigning a code to each segment wherein the operator may identify a segment with more than one code if the operator is uncertain of the number of phonemes that the segment may include.
Claim 8
Dependent Claim
27 words
The method according to claim 6, wherein the neural network determines which code should be used to classify the segment or whether the segment represents plural phonemes.
Claim 9
Dependent Claim
43 words
The method according to claim 1, wherein the transforming step includes transforming a segment into one or more multi-dimensional vectors in the frequency domain using a Fourier transform, each vector including multiple elements some of which represent spectral power in different frequency bands.
Claim 10
Dependent Claim
18 words
The method according to claim 8, wherein the fundamental frequency of the segment is filtered out before transformation.
Claim 11
Dependent Claim
35 words
The method according to claim 8, further comprising: reverse transforming the one or more multi-dimensional vectors and audibly reproducing resulting signal to confirm audible similarity with a corresponding one of the training set of phonemes.
Claim 12
Dependent Claim
21 words
The method according to claim 10, further comprising: modifying one or more transformation parameters as a result of the audible confirmation.
Claim 13
Dependent Claim
23 words
The method according to claim 1, wherein the neural network is selected by an operator from UCL, SCL, auto-associative, and backpropagation neural networks.
Claim 14
Dependent Claim
31 words
The method according to claim 1, further comprising: testing the developed neural network using another audio signal to determine how well the neural network recognizes phonemes in the another audio signal.
Claim 15
Dependent Claim
88 words
A user interface for developing a neural network for speech recognition, comprising: a display screen that displays speech segments representing phonemes of a speech signal, multiple dimension vectors, each segment being represented by corresponding one or more vectors, a corresponding phoneme code, and a centroid or an estimate of a centroid associated with each phoneme code to permit comparison between adjacent vectors, where similar sounds are coded with similar vectors, wherein the display screen provides qualitative assistance to a user in developing a neural network for speech recognition.
Claim 16
Dependent Claim
88 words
A user interface for developing a neural network for speech recognition, comprising: a display screen that displays speech segments representing phonemes of a speech signal, multiple dimension vectors, each segment being represented by corresponding one or more vectors, a corresponding phoneme code, and a centroid or an estimate of a centroid associated with each phoneme code to permit comparison between adjacent vectors, where similar sounds are coded with similar vectors, wherein the display screen provides qualitative assistance to a user in developing a neural network for speech recognition.
Claim 17
Dependent Claim
30 words
The user interface in claim 14 including a speaker in addition to the display screen, wherein the user visibly and audibly confirms the quality of the training set of phonemes.
Claim 18
Dependent Claim
46 words
The user interface in claim 14, further comprising: means for displaying an amplitude of the speech signal; means for digitizing the speech signal; means for dividing the digitized signals into speech segments; and means for indicating a starting point and an ending point of each segment.
Claim 19
Dependent Claim
33 words
The user interface in claim 16, further comprising: means for transforming each segment into one or more vectors having plural dimensions, wherein the one or more vectors are displayed on the display screen.
Claim 20
Dependent Claim
28 words
The user interface in claim 17, further comprising: means for assigning the corresponding phoneme code to the one or more vectors, wherein the corresponding phoneme code is displayed.
Claim 21
Dependent Claim
43 words
The user interface in claim 17, further comprising: means for generating the centroid or an estimate of a centroid corresponding to the one or more plural dimension vectors representing a particular phoneme, wherein the centroid or the estimate of a centroid is displayed.
Claim 22
Dependent Claim
57 words
The user interface in claim 19, wherein plural centroid vectors or estimates of centroid vectors are determined for each of plural segments, the user interface further comprising: means for selecting for one of the segments one of the plural centroid vectors or estimates of a centroid vector having a greatest likelihood of corresponding to the one segment.
Claim 23
Dependent Claim
19 words
The user interface in claim 16, further comprising: means for generating an audible signal corresponding to the digitized speech.
Claim 24
Dependent Claim
22 words
The user interface in claim 21, further comprising: means for generating an audible signal corresponding to one or more of the segments.
Claim 25
Dependent Claim
22 words
The user interface in claim 17, further comprising: means for generating an audible signal corresponding to transformed vectors corresponding to a segment.
Claim 26
Dependent Claim
23 words
The user interface in claim 19, further comprising: means for generating an audible signal corresponding to decoded centroids or decoded estimates of centroids.
Claim 27
Independent Claim
110 words
A method for processing speech information using a neural network trained to recognize speech phonemes and generate corresponding phoneme codes, where for each possible phoneme code, an exemplary centroid vector is identified based on an internal structure of the trained neural network, comprising the steps of: dividing an input speech signal into segments; transforming each segment into a set of vectors; the neural network assigning a phoneme code to each vector; converting a sequence of the phoneme codes into a corresponding sequence of exemplary centroid vectors; reverse transforming the sequence of exemplary centroid vectors into an output segment of speech; and generating an audio signal using the output speech segment.
Claim 28
Independent Claim
110 words
A method for processing speech information using a neural network trained to recognize speech phonemes and generate corresponding phoneme codes, where for each possible phoneme code, an exemplary centroid vector is identified based on an internal structure of the trained neural network, comprising the steps of: dividing an input speech signal into segments; transforming each segment into a set of vectors; the neural network assigning a phoneme code to each vector; converting a sequence of the phoneme codes into a corresponding sequence of exemplary centroid vectors; reverse transforming the sequence of exemplary centroid vectors into an output segment of speech; and generating an audio signal using the output speech segment.
Claim 29
Dependent Claim
17 words
The method in claim 25, wherein each of the codes represents a compressed amount of speech information.
Claim 30
Dependent Claim
19 words
The method in claim 25, further comprising: displaying one of the phoneme codes and a corresponding exemplary centroid vector.
Claim 31
Dependent Claim
20 words
The method in claim 25, wherein the internal structure of the neural network includes the trained neural network weight connections.
Claim 32
Dependent Claim
17 words
The method in claim 25, further comprising: generating an audio signal corresponding to the exemplary centroid vector.
Claim 33
Dependent Claim
13 words
The method in claim 25, further comprising: scaling the output segment of speech.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-09-14 | 2002-03-16 | 2002-09-15 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-09-14 | 2006-03-16 | 2006-09-15 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-09-14 | 2010-03-16 | 2010-09-15 | $7700.00 | Window Closed / Lapsed |