Patent US5987404A successfully refreshed from Google Patents.
US5987404A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Statistical natural language understanding using hidden clumpings
Assignee: International Business Machines Corp
•
Filing: 1996-01-29
•
Granted: 1999-11-16
•
Expires: 2016-01-29
Abstract
Abstract The invention proposes using statistical methods to do natural language understanding. The key notion is that there are "strings" of words in the natural language, that correspond to a single semantic concept. One can then define an alignment between an entire semantic meaning (consisting of a set of semantic concepts), and the English. This is modeled using P(E,A|S). One can model p(S) separately. This allows each parameter to be modeled using many different statistical models.
Inventors
- 👤 Stephen Andrew Della Pietra
- 👤 Mark Edward Epstein
- 👤 Martin Franz
- 👤 Joshua David Sherer Koppelman
- 👤 Salim Estephan Roukos
- 👤 Robert Todd Ward
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-30 02:26
CPC Classifications
G; G06; G06F; G06F40/00; G06F40/30; G06F40/40; G06F40/58
Claims Analysis
Total Claims: 17
Claim 1
Independent Claim
150 words
A method for performing natural language understanding, comprising: a training phase, comprising: providing a first sentence E of a given language; providing the meaning S of sentence E; generating a translation model by summing a probability of correctness over one or more possible alignments and clumpings between sentence E and meaning S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters; producing the probability distributions of the alignment and storing the probability distributions; an understanding phase, comprising: inputting a second sentence to be understood; producing, based upon the probability distributions, the meaning of the second sentence, wherein the probability distribution assumes that the sentence E is generated under predetermined semantic rules in non-overlapping substrings such that each substring of the sentence E is generated by one concept in a semantic library, and wherein a set of substrings is called a clumping.
Claim 2
Independent Claim
150 words
A method for performing natural language understanding, comprising: a training phase, comprising: providing a first sentence E of a given language; providing the meaning S of sentence E; generating a translation model by summing a probability of correctness over one or more possible alignments and clumpings between sentence E and meaning S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters; producing the probability distributions of the alignment and storing the probability distributions; an understanding phase, comprising: inputting a second sentence to be understood; producing, based upon the probability distributions, the meaning of the second sentence, wherein the probability distribution assumes that the sentence E is generated under predetermined semantic rules in non-overlapping substrings such that each substring of the sentence E is generated by one concept in a semantic library, and wherein a set of substrings is called a clumping.
Claim 3
Dependent Claim
71 words
The method of claim 1, wherein the training phase assumes that all possible alignments and the clumping is not known, and wherein an expectation maximization ("EM") algorithm is used to train the model parameters to a maximum likelihood value, and wherein the parameters of the translation model are used to predict the probability of the sentence E and a specific alignment of E to S given the semantic meaning S, p(E,A|S).
Claim 4
Dependent Claim
14 words
The method of claim 2, wherein the translation model comprises a hierarchy of sub-models.
Claim 5
Dependent Claim
34 words
The method of claim 3, further comprising constructing a language model to model the probability of the semantic meaning, p(S), and using this model to determine arg max s (p(E|S)p(S)) for a new E.
Claim 6
Dependent Claim
42 words
The method of claim 4, further comprising searching through the set of semantic meanings S to find the one that maximizes p(E|S)p(S), wherein for the maximum likelihood decoder, p(E|S)=the sum over A of p(E,A|S), and for the viterbi decoder p(E|S)=max A p(E,A|S).
Claim 7
Independent Claim
115 words
A method for training a natural language understanding system, comprising: providing a sentence E of a given language; providing meanings S of the sentence E; generating a translation model by summing a probability of correctness over one or more possible alignments and clumpings between E and S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters, and an expectation maximization ("EM") algorithm is used to train the model parameters to a maximum likelihood value, and wherein the parameters of the translation model are used to predict the probability of the sentence E and a specific alignment of E to S given the semantic meaning S, p(E,A|S).
Claim 8
Independent Claim
115 words
A method for training a natural language understanding system, comprising: providing a sentence E of a given language; providing meanings S of the sentence E; generating a translation model by summing a probability of correctness over one or more possible alignments and clumpings between E and S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters, and an expectation maximization ("EM") algorithm is used to train the model parameters to a maximum likelihood value, and wherein the parameters of the translation model are used to predict the probability of the sentence E and a specific alignment of E to S given the semantic meaning S, p(E,A|S).
Claim 9
Dependent Claim
14 words
The method of claim 6, wherein the translation model comprises a hierarchy of sub-models.
Claim 10
Dependent Claim
34 words
The method of claim 6, further comprising constructing a language model to model the probability of the semantic meaning, p(S), and using this model to determine arg max s (p(E|S)p(S) for a new E.
Claim 11
Dependent Claim
42 words
The method of claim 6, further comprising searching through the set of semantic meanings S to find the one that maximizes p(E|S)p(S), wherein for the maximum likelihood decoder, p(E|S)=the sum over A of p(E,A|S), and for the viterbi decoder p(E|S)=max A p(E,A|S).
Claim 12
Independent Claim
123 words
A method for training a natural language understanding system, comprising: providing a sentence E of a given language; providing the meaning S of the sentence E; generating a translation model by summing the probability of correctness over one or more possible alignments and clumpings between E and S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters; producing the probability distributions of the alignment and storing the probability distributions; wherein the probability distribution assumes that the sentence E is generated under predetermined semantic rules in non-overlapping substrings such that each substring of the sentence E is generated by one concept in a semantic library, and wherein a set of substrings is called a clumping.
Claim 13
Independent Claim
123 words
A method for training a natural language understanding system, comprising: providing a sentence E of a given language; providing the meaning S of the sentence E; generating a translation model by summing the probability of correctness over one or more possible alignments and clumpings between E and S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters; producing the probability distributions of the alignment and storing the probability distributions; wherein the probability distribution assumes that the sentence E is generated under predetermined semantic rules in non-overlapping substrings such that each substring of the sentence E is generated by one concept in a semantic library, and wherein a set of substrings is called a clumping.
Claim 14
Independent Claim
148 words
A method for performing natural language understanding, comprising: a training phase, comprising: providing a first sentence E of a given language; providing the meaning S of the sentence E; generating a translation model by summing a probability of correctness over one or more possible alignments and clumpings between E and S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters; producing the probability distributions of the alignment and storing the probability distributions; wherein the probability distribution assumes that the sentence E is generated under predetermined semantic rules in non-overlapping substrings such that each substring of sentence E is generated by one concept in a semantic library, and wherein a set of substrings is called a clumping; an understanding phase, comprising: inputting an English sentence to be understood; producing, based upon the probability distributions, the meaning of the English sentence.
Claim 15
Independent Claim
148 words
A method for performing natural language understanding, comprising: a training phase, comprising: providing a first sentence E of a given language; providing the meaning S of the sentence E; generating a translation model by summing a probability of correctness over one or more possible alignments and clumpings between E and S, wherein the actual alignment and clumping are not known, to produce the probabilities of the translation model's parameters; producing the probability distributions of the alignment and storing the probability distributions; wherein the probability distribution assumes that the sentence E is generated under predetermined semantic rules in non-overlapping substrings such that each substring of sentence E is generated by one concept in a semantic library, and wherein a set of substrings is called a clumping; an understanding phase, comprising: inputting an English sentence to be understood; producing, based upon the probability distributions, the meaning of the English sentence.
Claim 16
Independent Claim
103 words
A method for performing natural language understanding on an input string, comprising: (a) inputting a string to be understood; (b) performing a basic clumping operation on the string to identify clumps in the string; (c) performing a clumping-with-semantic-language-model operation on the string to determine the probability of clumps generated in step (b), given context; (d) generating a general fertility model to model a number of clumps the input string is allowed to generate; (e) generating a distortion model to model distances between clumps of the input string; (f) using the models generated in steps (c), (d), and (e) and, outputting an understanding result.
Claim 17
Independent Claim
103 words
A method for performing natural language understanding on an input string, comprising: (a) inputting a string to be understood; (b) performing a basic clumping operation on the string to identify clumps in the string; (c) performing a clumping-with-semantic-language-model operation on the string to determine the probability of clumps generated in step (b), given context; (d) generating a general fertility model to model a number of clumps the input string is allowed to generate; (e) generating a distortion model to model distances between clumps of the input string; (f) using the models generated in steps (c), (d), and (e) and, outputting an understanding result.
References Cited (Backward Citations)
US4829423A
Cited
US5109509A
Cited
US4829580A
Cited
US4914590A
Cited
US5029085A
Cited
US5369574A
Cited
US5293584A
Cited
US5680511A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-11-15 | 2003-05-17 | 2003-11-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-11-15 | 2007-05-17 | 2007-11-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-11-15 | 2011-05-17 | 2011-11-16 | $7700.00 | Window Closed / Lapsed |