US5509103A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method of training neural networks used for speech recognition
Assignee: Google Technology Holdings LLC
•
Filing: 1994-06-03
•
Granted: 1996-04-16
•
Expires: 2014-06-03
Abstract
Abstract A speech-recognition system for recognizing isolated words includes pre-processing circuitry for performing analog-to-digital conversion and cepstral analysis, and a plurality of neural networks which compute discriminant functions based on polynomial expansions. The system may be implemented using either hardware or software or a combination thereof. The speech wave-form of a spoken word is analyzed and converted into a sequence of data frames. The sequence of frames is partitioned into data blocks, and the data blocks are then broadcast to a plurality of neural networks. Using the data blocks, the neural networks compute polynomial expansions. The output of the neural networks is used to determine the identity of the spoken word. The neural networks utilize a matrix-inversion or alternatively a least-squares estimation training algorithm which does not require repetitive training and which yields a global minimum to each given set of training examples.
Inventors
- 👤 Shay-Ping T. Wang
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-27 09:10
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/08; G10L15/16; G06; G06F; G06F18/00; G06F18/20; G06F18/24; G06F18/245; G06F18/2453; G06N; G06N3/00; G06N3/02; G06N3/04; G06N3/045; G06N3/0499; G06N3/08; G06N3/09; G10L15/26; G10L15/02; G10L15/06; G10L15/063; G10L15/10; G10L15/22; G10L2015/223
Claims Analysis
Total Claims: 22
Claim 1
Independent Claim
155 words
A method of training a neural network used in a speech-recognition system, said neural network comprising a plurality of neurons, said method comprising the following steps: receiving an example spoken word; performing analog-to-digital conversion of said example spoken word to produce a digitized signal; performing cepstral analysis of said digitized signal to produce a sequence of data frames; generating a plurality of data blocks from said sequence of data frames; determining if said neural network is to recognize said example spoken word; (i) if so, equating neural network output to one; (ii) if not, equating said neural network output to zero; and training said neural network by calculating a plurality of weight values for said neural network, wherein if the number of said neurons is equal to the number of said data blocks, a matrix-inversion technique is used to calculate said weight values, otherwise, a least-squares estimation technique is used to calculate said weight values.
Claim 2
Independent Claim
155 words
A method of training a neural network used in a speech-recognition system, said neural network comprising a plurality of neurons, said method comprising the following steps: receiving an example spoken word; performing analog-to-digital conversion of said example spoken word to produce a digitized signal; performing cepstral analysis of said digitized signal to produce a sequence of data frames; generating a plurality of data blocks from said sequence of data frames; determining if said neural network is to recognize said example spoken word; (i) if so, equating neural network output to one; (ii) if not, equating said neural network output to zero; and training said neural network by calculating a plurality of weight values for said neural network, wherein if the number of said neurons is equal to the number of said data blocks, a matrix-inversion technique is used to calculate said weight values, otherwise, a least-squares estimation technique is used to calculate said weight values.
Claim 3
Dependent Claim
193 words
The method recited in claim 1, wherein the step of generating includes the following sub-steps: (a) equating a current frame to a first one of said sequence of frames; (b) equating a current data block to a first data block, said plurality of data blocks including said first data block and a last data block; (c) assigning said current frame to said current data block; (d) determining whether there is a next frame in said sequence of frames; (i) if so, proceeding to step (e); (ii) if not, proceeding to step (k); (e) equating said current frame to said next frame; (f) equating said current data block to a next data block; (g) assigning said current frame to said current data block; (h) determining if said current data block equals said last data block; (i) if so, proceeding to step (j); (ii) if not, returning to step (d); (j) equating said next data block to said first data block, and returning to step (d); and (k) determining if said current data block equals said last data block; (i) if not, assigning said current frame to remaining ones of said plurality of data blocks.
Claim 4
Dependent Claim
243 words
The method recited in claim 1, wherein said plurality of data blocks includes a first data block and a last data block and the step of generating includes the following sub-steps: (a) determining the number of data frames in said sequence of data frames; (b) defining the number of data frames per data block; (c) based on the number of data frames per block, calculating the number of data blocks to be generated; (d) equating a current frame to a first one of said sequence of data frames; (e) equating a current data block to said first data block; (f) assigning said current frame to said current data block; (g) determining whether there is a next frame in said sequence of frames; (i) if so, proceeding to step (h); (ii) if not, proceeding to step (n); (h) equating said current frame to said next frame; (j) equating said current data block to a next data block of said plurality of data blocks; (k) assigning said current frame to said current data block; (l) determining if said current data block is said last data block; (i) if so, proceeding to step (m); (ii) if not, returning to step (g); (m) equating said next data block to said first data block, and returning to step (g); and (n) determining if said current data block is said last data block; (i) if not, assigning said current frame to remaining ones of said plurality of data blocks.
Claim 5
Dependent Claim
47 words
The method recited in claim 3, wherein step (c) includes the following sub-steps: dividing said number of frames by said frames per data block to produce a result; and rounding up said result to a nearest integer to generate the number of data blocks to be generated.
Claim 6
Dependent Claim
36 words
The method recited in claim 1, wherein each frame in said sequence of frames comprises a plurality of data points, wherein each of said data points represents the amplitude of cepstrally-smoothed spectrum at a specific frequency.
Claim 7
Dependent Claim
19 words
The method recited in claim 1, wherein said neural network computes a discriminant function based upon a polynomial expansion.
Claim 8
Dependent Claim
84 words
The method recited in claim 6, wherein said polynomial expansion has the form: ##EQU5## wherein y represents the output of the neural network; wherein w i-1 represents the weight value for the ith neuron; wherein x 1 , x 2 , . . . , x n represent inputs to said neural network; wherein g 1i , . . . , g ni represent gating functions for the ith neuron which are applied to said inputs; and wherein m and n are positive integers.
Claim 9
Dependent Claim
65 words
The method recited in claim 6, wherein said polynomial expansion has the form: ##EQU6## wherein y represents the output of the neural network; wherein w i-1 represents the weight value for the ith neuron; wherein x i , x j , x k represent inputs to said neural network; wherein n is a positive integer; and wherein f 1 (i) through f6.sub.(i,j,k) are index-generating functions.
Claim 10
Dependent Claim
112 words
The method recited in claim 6, wherein said polynomial expansion has the form: ##EQU7## wherein y represents the output of the neural network; wherein w 0 , w 1 , . . . , w N-1 represent weight values, N being a positive integer representing the Nth neuron, and the term w i in general representing the weight value associated with the i+1 neuron; wherein x 1 , x 2 , . . . , x n represent inputs to said neural network, n being a positive integer; and wherein g 1N , . . . , g nN represent gating functions for the Nth neuron which are applied to said inputs.
Claim 11
Dependent Claim
19 words
The method recited in claim 1, wherein said neural network computes a discriminant function based upon an orthogonal function.
Claim 12
Dependent Claim
34 words
The method recited in claim 10, wherein said orthogonal function is a sine, cosine, exponential, logarithmic, Fourier transformation, Legendre polynomial, or a non-linear function such as a Volterra function or a radial basis function.
Claim 13
Dependent Claim
24 words
The method recited in claim 1, wherein said neural network computes a discriminant function based upon a combination of polynomial expansion and orthogonal functions.
Claim 14
Dependent Claim
17 words
The method recited in claim 1, wherein every training example is used only once by said method.
Claim 15
Dependent Claim
18 words
The method recited in claim 1, wherein said neural networks are contained on at least one integrated circuit.
Claim 16
Dependent Claim
16 words
The method recited in claim 1, wherein said neural networks are contained in a computer program.
Claim 17
Dependent Claim
16 words
The method recited in claim 1, wherein said least-squares estimation technique utilizes a least-squares error technique.
Claim 18
Dependent Claim
16 words
The method recited in claim 1, wherein said least-squares estimation technique utilizes an extended least-squares technique.
Claim 19
Dependent Claim
15 words
The method recited in claim 1, wherein said least-squares estimation technique utilizes a Kalman filter.
Claim 20
Dependent Claim
15 words
The method recited in claim 1, wherein said least-squares estimation technique utilizes a maximum-likelihood algorithm.
Claim 21
Dependent Claim
15 words
The method recited in claim 1, wherein said least-squares estimation technique utilizes a pseudo-inverse algorithm.
Claim 22
Dependent Claim
15 words
The method recited in claim 1, wherein said least-squares estimation technique utilizes a Bayesian estimation.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1999-04-16 | 1999-10-16 | 2000-04-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2003-04-16 | 2003-10-16 | 2004-04-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2007-04-16 | 2007-10-16 | 2008-04-16 | $7700.00 | Window Closed / Lapsed |