Automatic determination of labels and markov word models in a speech recognition system

Claim 2

Speech processing apparatus comprising: an acoustic processor for producing as a first output, in response to speech input, one label after another at successive time intervals, each label being selected from an alphabet of labels; dictionary means for storing statistical data for each of a plurality of vocabulary words as Markov model word baseforms, wherein each baseform is characterized by a sequence of Markov models each Markov model having a plurality of arcs, wherein the dictionary means includes storage for (i) the respective probability of each arc in each Markov model, and (ii) a respective probability of producing each label in the alphabet at each of some arcs in each Markov model; means, coupled to said acoustic processor, for re-specifying the labels in the alphabet which can be selectively produced as outputs; and baseform constructor means, coupled to said dictionary means, for up-dating the stored data for the Markov model word baseforms based on re-specified labels generated by the acoustic processor; wherein said label re-specifying means re-­specifies labels based on the up-dated stored data for the Markov model word baseforms.

Claim 3

Apparatus as in Claim 1 wherein said acoustic processor produces as a second output one feature vector after another at the successive time intervals; wherein each different Markov model corresponds to one respective label; and wherein said label re-specifying means includes: alignment processor means for aligning a string of labels generated by the acoustic processor means against a word baseform, said alignment processor means aligning successive substrings in the string with successive Markov models in the word baseform; and estimator means for receiving as input from the acoustic processor the feature vectors corresponding to the labels aligned with a given Markov model and computing mean and covariance values of the feature vectors received for the given Markov model; and label specifier means, coupled to the estimator means, for associating in storage the mean and covariance values for each Markov model with the label corresponding thereto.

Claim 4

Apparatus as in Claim 2 wherein said acoustic processor includes: means for comparing each successive feature vector generated thereby against the mean and covariance values associated with each label and determining which label with associated mean and covariance values is closest to the feature vector; and means for assigning to each successive interval the label with associated mean and covariance values closest to the feature vector at said each interval.

Claim 5

Apparatus as in Claim 3 further comprising: training means for computing Markov model data based on a string of labels generated by the acoustic processor in response to the utterance or a known text during the training session; said training means being connected to enter the Markov model data into said baseform constructor means.

Claim 6

Apparatus as in Claim 4 where said acoustic processor includes: means for assigning to each successive time interval the label with associated mean and covariance values closest to the feature vector for said each interval; the apparatus further comprising: acoustic match means for (a) storing the con­structed word baseforms and data computed therefor by said training means and (b) computing the measure of match between the string of labels generated by said acoustic processor and the sequence of Markov models constructed by said baseform constructor means for each word baseform.

Claim 7

Apparatus as in Claim 5 further comprising: language model processor means for storing occurrence data of word sequences derived from sample database text and for computing a like­lihood score of each word based on the stored occurrence data; and stack decoder means, connected to receive as inputs (i) language model likelihood scores for words as computed by said language model processor means and (ii) acoustic match scores for words as computed by said acoustic match means, for selecting likely next words for each of a plurality of word paths based on the acoustic match scores and language model scores computed for the words; said stack decoder means including means for computing, for each path of words previously decoded to have a likelihood above a prescribed threshold, the likelihood of said each path after selected words are appended thereto.

Claim 8

A computerized method of processing speech for speech recognition comprising the steps of: (a) generating, in an accoustic processor, one feature vector after another for one time interval after another in response to uttered speech; (b) for each time interval, assigning one of an alphabet of stored labels thereto which corresponds to one prototype vector of an alphabet of proto­type vectors said one prototype vector being the closest to the feature vector generated for a given time interval; (c) storing in computer memory each word of a vocabulary in a computer memory as a sequence of Markov models, which includes the step of: selecting a set of Markov models wherein each Markov model corresponds to a respective label; and storing, for each Markov model, a plurality of arc probabilities and label probabilities, wherein each label probability corresponds to the likelihood of a respective label being produced at a given Markov model arc; (d) for an uttered known word sequence, aligning labels which are generated according to step (a) with each successive Markov model included in the known word sequence; and (e) for a subject Markov model, re-specifying the prototype vector based on the feature vectors corresponding to each label aligned with the subject Markov model and associating the re-­specified prototype vector with the label corresponding to the subject Markov model.

Claim 9

A computerized method of processing speech for speech recognition comprising the steps of: (a) generating, in an accoustic processor, one feature vector after another for one time interval after another in response to uttered speech; (b) for each time interval, assigning one of an alphabet of stored labels thereto which corresponds to one prototype vector of an alphabet of proto­type vectors said one prototype vector being the closest to the feature vector generated for a given time interval; (c) storing in computer memory each word of a vocabulary in a computer memory as a sequence of Markov models, which includes the step of: selecting a set of Markov models wherein each Markov model corresponds to a respective label; and storing, for each Markov model, a plurality of arc probabilities and label probabilities, wherein each label probability corresponds to the likelihood of a respective label being produced at a given Markov model arc; (d) for an uttered known word sequence, aligning labels which are generated according to step (a) with each successive Markov model included in the known word sequence; and (e) for a subject Markov model, re-specifying the prototype vector based on the feature vectors corresponding to each label aligned with the subject Markov model and associating the re-­specified prototype vector with the label corresponding to the subject Markov model.

Claim 10

The method of Claim 7 comprising the further step of: (f) repeating step (e) for each Markov model as the subject Markov model.

Claim 11

The method of Claim 7 wherein step (e) includes the steps of: (g) computing and storing the mean and covariance values over all the feature vectors corresponding to the labels aligned with the subject Markov model; and (h) re-specifying the label corresponding to the subject Markov model as the mean and covariance values computed in step (g) for the subject Markov model.

Claim 12

The method of Claim 9 comprising the further step of: (j) repeating step (e) for each Markov model as the subject Markov model.

Claim 13

The method of Claim 10 comprising the further steps of: (k) for an uttered known word sequence, (i) generating successive feature vectors, (ii) comparing each feature vector against the mean value and covariance value computed in step (g) for each label, and (iii) assigning to each interval the re-specified label having the closest mean value and covariance value to the feature vector corresponding to the interval; and (ℓ) re-constructing the word baseforms based on the re-specified labels.

Claim 14

The method of Claim 11 wherein step (ℓ) includes the steps of: (m) transforming each of multiple utterances of the word segment into a respective string of labels; (n) determining the best single Markov model P1 for producing the multiple label strings; (p) determining the best two-model baseform of the form P1P2 or P2P1 for producing the multiple label strings; (q) aligning the best two-model baseform against each label string; (r) splitting each label string into a left portion and a right portion with the left portion corresponding to the first model of the two-model baseform and the right portion corresponding to the second model of the two-model baseform; (s) identifying each left portion as a left substring and each right portion as a right substring; (t) processing the set of left substrings in the same manner as the set of label strings correspond­ing to the multiple utterances including the further step of inhibiting further splitting of a substring when the single model baseform thereof has a higher probability of producing the sub­string than does the best two-model baseform; (u) processing the set of right substrings in the same manner as the set of label strings correspond­ing to the multiple utterances, including the further step of inhibiting further splitting of a substring when the single model baseform thereof has a higher probability of producing the sub­string than does the best two-model baseform; and (v) concatenating the unsplit single models in an order corresponding to the order of the label substrings to which they correspond.

Claim 15

The method of Claim 12 wherein step (ℓ) includes the further steps of: (w) aligning the concatenated baseform against each of the label strings and identifying, for each model in the concatenated baseform, the substring in each label string which corresponds thereto, the substrings corresponding to a given model being a set of common substrings; (x) for each set of common substrings, determining the model having the highest joint probability of producing the common substrings; and (y) for each common substring, replacing the model therefor in the concatenated baseform by the determined model of highest joint probability; the baseform resulting from the replacing of models being a refined baseform.

Claim 16

The method of Claim 13 wherein step (ℓ) further includes the step of: (z) repeating steps (w) through (y) until no models are replaced.