US6188976B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Apparatus and method for building domain-specific language models
Assignee: Nuance Communications Inc
•
Filing: 1998-10-23
•
Granted: 2001-02-13
•
Expires: 2018-10-23
Abstract
Abstract Disclosed is a method and apparatus for building a domain-specific language model for use in language processing applications, e.g., speech recognition. A reference language model is generated based on a relatively small seed corpus containing linguistic units relevant to the domain. An external corpus containing a large number of linguistic units is accessed. Using the reference language model, linguistic units which have a sufficient degree of relevance to the domain are extracted from the external corpus. The reference language model is then updated based on the seed corpus and the extracted linguistic units. The process may be repeated iteratively until the language model is of satisfactory quality. The language building technique may be further enhanced by combining it with mixture modeling or class-based modeling.
Inventors
- 👤 Ganesh N. Ramaswamy
- 👤 Harry W. Printz
- 👤 Ponani S. Gopalakrishnan
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:00
CPC Classifications
G; G10; G10L; G10L15/00; G10L15/08; G10L15/18; G10L15/183
Claims Analysis
Total Claims: 24
Claim 1
Independent Claim
86 words
A method for building a language model specific to a domain, comprising the steps of: a) building a reference language model based on a seed corpus containing linguistic units relevant to said domain; b) accessing an external corpus containing a large number of linguistic units; c) using said reference language model, selectively extracting linguistic units from said external corpus that have a sufficient degree of relevance to said domain; and d) updating said reference language model based on said seed corpus and said extracted linguistic units.
Claim 2
Independent Claim
86 words
A method for building a language model specific to a domain, comprising the steps of: a) building a reference language model based on a seed corpus containing linguistic units relevant to said domain; b) accessing an external corpus containing a large number of linguistic units; c) using said reference language model, selectively extracting linguistic units from said external corpus that have a sufficient degree of relevance to said domain; and d) updating said reference language model based on said seed corpus and said extracted linguistic units.
Claim 3
Dependent Claim
48 words
The method of claim 1 , further comprising the steps of: measuring quality of said updated language model; and, repeating steps b), c) and d) if the measured quality is determined to be below a quality threshold, otherwise defining said updated language model as a final language model.
Claim 4
Dependent Claim
31 words
The method of claim 2 wherein the step of measuring quality comprises calculating perplexity of the updated reference language model using a test corpus containing linguistic units relevant to said domain.
Claim 5
Dependent Claim
38 words
The method of claim 2 wherein the step of measuring quality comprises: providing a test corpus containing linguistic units relevant to said domain; and evaluating speech recognition accuracy for said test corpus using said updated reference language model.
Claim 6
Dependent Claim
40 words
The method of claim 2 wherein the step of measuring quality comprises comparing the size of linguistic units extracted during a current linguistic unit extraction iteration to the size of linguistic units extracted during at least one prior extraction iteration.
Claim 7
Dependent Claim
37 words
The method of claim 1 wherein said step c) is performed by computing perplexity scores for individual linguistic units from said external corpus and selectively extracting those linguistic units having a perplexity score below a perplexity threshold.
Claim 8
Dependent Claim
37 words
The method of claim 6 wherein said perplexity threshold is computed dynamically, and corresponds to a percentile rank of perplexity measures of the linguistic units of said seed corpus, calculated according to the latest reference language model.
Claim 9
Dependent Claim
65 words
The method of claim 1 , further comprising the steps of: forming N subcorpora of linguistic units from said linguistic units extracted from a test corpus, grouped according to degree of relevance to said domain; building N language models based on said seed corpus and said N subcorpora, respectively; and wherein said step of updating said reference language model includes mixing said N language models.
Claim 10
Dependent Claim
19 words
The method of claim 1 wherein said linguistic units of said seed corpus and said external corpus comprise sentences.
Claim 11
Dependent Claim
64 words
The method of claim 1 , further comprising the step of generating word classes from said linguistic units of said seed corpus and said linguistic units extracted from said external corpus; and wherein said step d) of updating said reference language model is performed in accordance with said word classes so as to construct said updated reference language model as a class-based language model.
Claim 12
Dependent Claim
35 words
The method of claim 1 wherein said step d) of updating said reference language model is performed after a predetermined number of linguistic units have been selectively extracted from said external corpus in step c).
Claim 13
Independent Claim
86 words
An apparatus for building a language model for a specific domain, comprising: a seed corpus containing linguistic units relevant to said domain; a language model constructor for building a reference language model from said seed corpus; a corpus extractor operative to access an external corpus and, using said reference language model, to selectively extract linguistic units which have a sufficient degree of relevance to said domain; wherein said language model constructor updates said reference language model based on said seed corpus and said extracted linguistic units.
Claim 14
Independent Claim
86 words
An apparatus for building a language model for a specific domain, comprising: a seed corpus containing linguistic units relevant to said domain; a language model constructor for building a reference language model from said seed corpus; a corpus extractor operative to access an external corpus and, using said reference language model, to selectively extract linguistic units which have a sufficient degree of relevance to said domain; wherein said language model constructor updates said reference language model based on said seed corpus and said extracted linguistic units.
Claim 15
Dependent Claim
83 words
The apparatus of claim 12 , further comprising a model checker for measuring quality of said updated reference language model and defining said updated language model as a final language model if the measured quality is above a quality threshold, otherwise said corpus extractor selectively extracts additional linguistic units from said external corpus and said language model constructor again updates said reference language model based on said seed corpus and cumulative extracted linguistic units, so as to iteratively construct a final language model.
Claim 16
Dependent Claim
99 words
The apparatus of claim 13 , further comprising: a test corpus containing linguistic units relevant to said domain; and said model checker measuring quality of said updated reference language model with at least one of: (i) a speech recognition engine to measure speech recognition accuracy of said reference language model using linguistic units of said test corpus; (ii) a perplexity calculator to calculate perplexity of said updated reference language model using said test corpus; and (iii) an incremental size evaluator for evaluating the number of linguistic units selectively extracted from said external corpus during a current language building iteration.
Claim 17
Dependent Claim
54 words
The apparatus of claim 12 wherein said sufficient degree of relevance is dynamically determined by a threshold parameter generator of said corpus extractor which computes a perplexity threshold corresponding to a percentile rank of perplexity measures of the linguistic units of said seed corpus according to the latest version of said reference language model.
Claim 18
Dependent Claim
101 words
The apparatus of claim 12 , further including a relevant corpus for storing said selectively extracted linguistic units, said relevant corpus comprising a plurality N of subcorpora grouped according to relevance to said domain, each dedicated to storing plural of said selectively extracted linguistic units falling within a certain range of relevance to said domain; wherein said language model constructor is operative to construct N reference language models based on said seed corpus and said N subcorpora, respectively; and said apparatus further includes a language model mixer to mix said N reference language models to form said updated reference language model.
Claim 19
Dependent Claim
99 words
The apparatus of claim 16 , further comprising: a test corpus containing linguistic units relevant to said domain; and a model checker for measuring quality of said updated reference language model with at least one of: (i) a speech engine to measure speech recognition accuracy of said reference language model using linguistic units of said test corpus; (ii) a perplexity calculator to calculate perplexity of said updated reference language model using said test corpus; and (iii) an incremental size evaluator for evaluating the number of linguistic units selectively extracted from said external corpus during a current language building iteration.
Claim 20
Dependent Claim
64 words
The apparatus of claim 12 , further comprising a word class generator for generating word classes from said linguistic units of said seed corpus and said linguistic units extracted from said external corpus; and wherein said language model constructor updates said reference language model in accordance with said word classes so as to construct said updated reference language model as a class-based language model.
Claim 21
Dependent Claim
27 words
The apparatus of claim 12 wherein said sufficient degree of relevance is higher for an initial iteration of external corpus linguistic unit extraction than for subsequent iterations.
Claim 22
Dependent Claim
106 words
A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to provide method steps for building a language model specific to a domain, said method steps comprising: a) building a reference language model based on a seed corpus containing linguistic units relevant to said domain; b) accessing an external corpus containing a large number of linguistic units; c) using said reference language model, selectively extracting linguistic units from said external corpus that have a sufficient degree of relevance to said domain; and d) updating said reference language model based on said seed corpus and said extracted linguistic units.
Claim 23
Dependent Claim
106 words
A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to provide method steps for building a language model specific to a domain, said method steps comprising: a) building a reference language model based on a seed corpus containing linguistic units relevant to said domain; b) accessing an external corpus containing a large number of linguistic units; c) using said reference language model, selectively extracting linguistic units from said external corpus that have a sufficient degree of relevance to said domain; and d) updating said reference language model based on said seed corpus and said extracted linguistic units.
Claim 24
Dependent Claim
37 words
The method of claim 20 wherein said step c) is performed by computing perplexity scores for individual linguistic units from said external corpus and selectively extracting those linguistic units having a perplexity score below a perplexity threshold.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2004-02-13 | 2004-08-14 | 2005-02-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2008-02-13 | 2008-08-14 | 2009-02-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2012-02-13 | 2012-08-14 | 2013-02-13 | $7700.00 | Window Closed / Lapsed |