Method and apparatus for pattern recognition employing the Hidden Markov Model

Claim 2

A computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and weighted sum calculating means for generating a second calculation signal representing the weighted sum of logarithmic values of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted arithmetic mean of said logarithmic values of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the weighted sum or the weighted arithmetic mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 3

A computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and power product calculating means for generating a second calculation signal representing the product of powers of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted geometric mean of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said power or weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the product of power or the weighted geometric mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 4

A computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and power product calculating means for generating a second calculation signal representing the product of powers of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted geometric mean of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said power or weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the product of power or the weighted geometric mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 5

A computer implemented hidden Markov model memory apparatus comprising parameter memory means for storing the parameters of a model and characterized by storing the parameters estimated by a computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal by the parameter memory means, employing the hidden Markov model creating apparatus comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and weighted sum calculating means for generating a second calculation signal representing the weighted sum of logarithmic values of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted arithmetic mean of said logarithmic values of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the weighted sum or the weighted arithmetic mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 6

A computer implemented hidden Markov model memory apparatus comprising parameter memory means for storing the parameters of a model and characterized by storing the parameters estimated by a computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal by the parameter memory means, employing the hidden Markov model creating apparatus comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and weighted sum calculating means for generating a second calculation signal representing the weighted sum of logarithmic values of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted arithmetic mean of said logarithmic values of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the weighted sum or the weighted arithmetic mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 7

A computer implemented hidden Markov model memory apparatus comprising parameter memory means for storing the parameters of a model and characterized by storing the parameters estimated by a computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal by the parameter memory means, employing the hidden Markov model creating apparatus comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and power product calculating means for generating a second calculation signal representing the product of powers of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted geometric mean of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said power or weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the product of power or the weighted geometric mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 8

A computer implemented hidden Markov model memory apparatus comprising parameter memory means for storing the parameters of a model and characterized by storing the parameters estimated by a computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal by the parameter memory means, employing the hidden Markov model creating apparatus comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and power product calculating means for generating a second calculation signal representing the product of powers of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted geometric mean of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said power or weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the product of power or the weighted geometric mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t).

Claim 9

A computer implemented likelihood calculating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and weighted sum calculating means for generating a second calculation signal representing the weighted sum of logarithmic values of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted arithmetic mean of said logarithmic values of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the weighted sum or the weighted arithmetic mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t), and a likelihood of a hidden Markov model possessing the parameters stored in the parameter memory means to the observation vector is calculated.

Claim 10

A computer implemented likelihood calculating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and weighted sum calculating means for generating a second calculation signal representing the weighted sum of logarithmic values of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted arithmetic mean of said logarithmic values of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the weighted sum or the weighted arithmetic mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t), and a likelihood of a hidden Markov model possessing the parameters stored in the parameter memory means to the observation vector is calculated.

Claim 11

A computer implemented likelihood calculating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and power product calculating means for generating a second calculation signal representing the product of powers of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted geometric mean of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said power or weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the product of powers or the weighted geometric mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t), and a likelihood of a hidden Markov model possessing the parameters stored in the parameter memory means to the observation vector is calculated.

Claim 12

A computer implemented likelihood calculating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: function value calculating means for generating a first calculation signal corresponding to a mapping of each pair C m and y(t), (C m ,y(t)) into a signal u(y(t),m).di-elect cons.U=[a,b], where m=1, . . . ,M, a,b.di-elect cons.R 1 for 0≦a≦b, C={C 1 , C 2 , . . . , C M }, y(t).di-elect cons.R n , R n is an n-dimensional Euclidean space, signal occurrence probability memory means for storing the occurrence probability of each signal of set C, where said occurrence probabilities of signals in the set C are received and stored, and power product calculating means for generating a second calculation signal representing the product of powers of occurrence probabilities of signals in the set C or generating a third calculation signal representing the weighted geometric mean of occurrence probabilities of signals in the set C, where said occurrence probabilities of signals in the set C are stored in said signal occurrence probability memory means and said power or weighting coefficient for the m-th signal is defined by u(y(t),m), wherein the signal representing the product of powers or the weighted geometric mean is the occurrence degree of the vector signal y(t) at time t, and parameter estimating means is provided for estimating the parameter of the model so that the occurrence degree of the pattern to be modeled, composed of observation vector series y(1) . . . ,y(t), . . . y(T), may be maximum on the basis of said occurrence degree of the vector signal y(t), and a likelihood of a hidden Markov model possessing the parameters stored in the parameter memory means to the observation vector is calculated.

Claim 13

A computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: a feature extraction part for convening a plurality of training speech signals of r=1 to R w for model creation corresponding to recognition unit w(=1, . . . , W) into a feature vector series Y.sup.w(r) =(y.sup.w(r) (1), y.sup.w(r) (2), . . . , y.sup.w(r) (T.sup.w(r))), a recognition unit pattern memory part for storing a plurality of training patterns for model creation corresponding to the recognition unit w(=1, . . . ,W) by R w pieces for w in a form of the feature vector series, a buffer memory temporarily storing patterns Y w (1), . . . ,Y w (R w ) for the recognition unit pattern memory part, a clustering part for clustering the feature vector set from the recognition trait pattern memory part, a centroid memory part for storing centroids of clusters F 1 , . . . ,F M obtained by clustering by the clustering part, a vector membership value calculating memory part for calculating and storing the membership values u(y w (r) (t),1), . . . , u(y w (r) (t),M), each of which is derived for the vector y w (r) (t) composing the training pattern stored in said buffer memory to said cluster F m using centroids stored in said centroid memory part, a parameter estimating part for estimating parameters of model λ w corresponding to recognition unit w, by executing a step of creating the model λ w by giving the occurrence degree ω w 1 (y w (r) (t)) of feature vector y w (r) (t) in state i of model λ w corresponding to the recognition unit in a form of ##EQU62## in the relation between the membership value u(y w (r) (t),m) and occurence probability b w im in state i of cluster F m , a first parameter memory part for temporarily storing the reestimated value of the parameter estimating part, and a second parameter memory part for storing the parameters as the final result of the re-estimation corresponding to the recognition unit w(=1, . . . , W), wherein the parameter estimating part re-estimates parameters of the model λ w by using the value of the first parameter memory part where the parameters are renewed iteratively by the re-estimation and the renewed parameters are used for next re-estimation step.

Claim 14

A computer implemented hidden Markov model creating apparatus for processing an observation vector signal y(t) representing a sound input signal, comprising: a feature extraction part for convening a plurality of training speech signals of r=1 to R w for model creation corresponding to recognition unit w(=1, . . . , W) into a feature vector series Y.sup.w(r) =(y.sup.w(r) (1), y.sup.w(r) (2), . . . , y.sup.w(r) (T.sup.w(r))), a recognition unit pattern memory part for storing a plurality of training patterns for model creation corresponding to the recognition unit w(=1, . . . ,W) by R w pieces for w in a form of the feature vector series, a buffer memory temporarily storing patterns Y w (1), . . . ,Y w (R w ) for the recognition unit pattern memory part, a clustering part for clustering the feature vector set from the recognition trait pattern memory part, a centroid memory part for storing centroids of clusters F 1 , . . . ,F M obtained by clustering by the clustering part, a vector membership value calculating memory part for calculating and storing the membership values u(y w (r) (t),1), . . . , u(y w (r) (t),M), each of which is derived for the vector y w (r) (t) composing the training pattern stored in said buffer memory to said cluster F m using centroids stored in said centroid memory part, a parameter estimating part for estimating parameters of model λ w corresponding to recognition unit w, by executing a step of creating the model λ w by giving the occurrence degree ω w 1 (y w (r) (t)) of feature vector y w (r) (t) in state i of model λ w corresponding to the recognition unit in a form of ##EQU62## in the relation between the membership value u(y w (r) (t),m) and occurence probability b w im in state i of cluster F m , a first parameter memory part for temporarily storing the reestimated value of the parameter estimating part, and a second parameter memory part for storing the parameters as the final result of the re-estimation corresponding to the recognition unit w(=1, . . . , W), wherein the parameter estimating part re-estimates parameters of the model λ w by using the value of the first parameter memory part where the parameters are renewed iteratively by the re-estimation and the renewed parameters are used for next re-estimation step.