US5671333A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Training apparatus and method
Assignee: Nokia of America Corp
•
Filing: 1994-04-07
•
Granted: 1997-09-23
•
Expires: 2014-04-07
Abstract
Abstract Apparatus and methods for training classifiers. The apparatus includes a degree of certainty classifier which classifies examples in categories and indicates a degree of certainty regarding the classification and an annotating classifier which receives classified examples with a low degree of certainty from the degree of certainty classifier and annotates them to indicate whether their classifications are correct. The annotated examples are then used to train another classifier. In one version of the invention, the other classifier is a new version of the degree of certainty classifier, and training continues until the degree of certainty classifier has satisfactory performance. The degree of certainty classifier of the embodiment is a probabilistic binary classifier which is trained using relevance feedback. The annotating classifier may include an interactive interface which permits a human user of the system to examine an example and indicate whether it was properly classified.
Inventors
- 👤 Jason A. Catlett
- 👤 William Arthur Gale
- 👤 David Dolan Lewis
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-26 17:25
CPC Classifications
G; G06; G06N; G06N20/00
Claims Analysis
Total Claims: 18
Claim 1
Independent Claim
176 words
An apparatus for interpreting data, comprising: a current first classifier operative to interpret a plurality of actual examples of the data and to output an interpretation of each interpreted example and a certainty value associated with each interpretation wherein the current first classifier comprises a chooser operative to discriminate between certain ones of the outputted interpretations having respective high certainty values and uncertain ones of the outputted interpretations having respective low certainty values and to select and output each of the actual examples associated with a respective uncertain one of the interpretations; a second classifier operative to annotate each of the interpreted examples associated with the selected uncertain ones of the interpretations and to output a preferred interpretation for each interpreted example associated with the selected uncertain ones of the interpretations; and an uncertainty measuring device generator operative to produce a next first classifier by utilizing at least one annotated example and its associated preferred interpretation, the next first classifier capable of interpreting subsequent actual examples of the data more accurately than the current first classifier.
Claim 2
Independent Claim
176 words
An apparatus for interpreting data, comprising: a current first classifier operative to interpret a plurality of actual examples of the data and to output an interpretation of each interpreted example and a certainty value associated with each interpretation wherein the current first classifier comprises a chooser operative to discriminate between certain ones of the outputted interpretations having respective high certainty values and uncertain ones of the outputted interpretations having respective low certainty values and to select and output each of the actual examples associated with a respective uncertain one of the interpretations; a second classifier operative to annotate each of the interpreted examples associated with the selected uncertain ones of the interpretations and to output a preferred interpretation for each interpreted example associated with the selected uncertain ones of the interpretations; and an uncertainty measuring device generator operative to produce a next first classifier by utilizing at least one annotated example and its associated preferred interpretation, the next first classifier capable of interpreting subsequent actual examples of the data more accurately than the current first classifier.
Claim 3
Dependent Claim
27 words
The apparatus set forth in claim 1 further comprising an uncertainty measuring device computes an estimate of the entropy of the classification to each unannotated data portion.
Claim 4
Dependent Claim
18 words
The apparatus set forth in claim 1, wherein: said examples are divided into data portions by a segmenter.
Claim 5
Independent Claim
173 words
An apparatus for interpreting data, comprising: a first classifier operative to interpret a plurality of actual examples of the data and to output an interpretation of each interpreted example and a certainty value associated with each interpretation wherein the first classifier comprises a chooser operative to discriminate between certain ones of the outputted interpretations having respective high certainty values and uncertain ones of the outputted interpretations having respective low certainty values and to select and output each of the interpreted examples associated with a respective uncertain one of the interpretations; a second classifier operative to annotate each of the interpreted examples associated with the selected uncertain ones of the interpretations and to output a preferred interpretation for each interpreted example associated with the selected uncertain ones of the interpretations; and an uncertainty measuring device generator operative to produce a next first classifier by using at least one annotated example and its associated preferred interpretation, the next first classifier capable of interpreting subsequent actual examples of the data more accurately than the first classifier.
Claim 6
Independent Claim
173 words
An apparatus for interpreting data, comprising: a first classifier operative to interpret a plurality of actual examples of the data and to output an interpretation of each interpreted example and a certainty value associated with each interpretation wherein the first classifier comprises a chooser operative to discriminate between certain ones of the outputted interpretations having respective high certainty values and uncertain ones of the outputted interpretations having respective low certainty values and to select and output each of the interpreted examples associated with a respective uncertain one of the interpretations; a second classifier operative to annotate each of the interpreted examples associated with the selected uncertain ones of the interpretations and to output a preferred interpretation for each interpreted example associated with the selected uncertain ones of the interpretations; and an uncertainty measuring device generator operative to produce a next first classifier by using at least one annotated example and its associated preferred interpretation, the next first classifier capable of interpreting subsequent actual examples of the data more accurately than the first classifier.
Claim 7
Dependent Claim
25 words
The apparatus of claim 4 further comprising an uncertainty measuring device computes an estimate of the entropy of the classification to each unannotated data portion.
Claim 8
Dependent Claim
16 words
The apparatus of claim 4, wherein: said examples are divided into data portions by a segmenter.
Claim 9
Independent Claim
180 words
An apparatus for interpreting data, comprising: a first classifier operative to interpret a plurality of actual examples of the data according to a first principle and to output an interpretation of each interpreted example and a certainty value associated with each interpretation wherein the first classifier comprises a chooser operative to discriminate between certain ones of the outputted interpretations having respective high certainty values and uncertain ones of the outputted interpretations having a low certainty values and to select and output each of the interpreted examples associated with a respective uncertain one of the interpretations; a second classifier operative to annotate each of the interpreted examples associated with the selected uncertain ones of the interpretations and to output a preferred interpretation for each interpreted example associated with the selected uncertain ones of the interpretations; and an uncertainty measuring device generator operative to produce a third classifier by utilizing at least one annotated example and its associated preferred interpretation, the third classifier operative to interpret subsequent actual examples of the data according to a second principle different from the first principle.
Claim 10
Independent Claim
180 words
An apparatus for interpreting data, comprising: a first classifier operative to interpret a plurality of actual examples of the data according to a first principle and to output an interpretation of each interpreted example and a certainty value associated with each interpretation wherein the first classifier comprises a chooser operative to discriminate between certain ones of the outputted interpretations having respective high certainty values and uncertain ones of the outputted interpretations having a low certainty values and to select and output each of the interpreted examples associated with a respective uncertain one of the interpretations; a second classifier operative to annotate each of the interpreted examples associated with the selected uncertain ones of the interpretations and to output a preferred interpretation for each interpreted example associated with the selected uncertain ones of the interpretations; and an uncertainty measuring device generator operative to produce a third classifier by utilizing at least one annotated example and its associated preferred interpretation, the third classifier operative to interpret subsequent actual examples of the data according to a second principle different from the first principle.
Claim 11
Dependent Claim
17 words
The apparatus set forth in claim 1, claim 4 or claim 7 wherein, the data is text.
Claim 12
Dependent Claim
17 words
The apparatus set forth in claim 1, claim 4, or claim 7 wherein: the data is text.
Claim 13
Dependent Claim
34 words
The apparatus set forth in claim 1, claim 4, or claim 7 wherein: the second classifier includes interactive means for providing the example to a human judge and receiving the annotation from the judge.
Claim 14
Dependent Claim
30 words
The apparatus set forth in claim 1, claim 4 or claim 7 wherein: the uncertainty measuring device generator includes means for using relevance feedback to produce the next first classifier.
Claim 15
Dependent Claim
71 words
The apparatus set forth in claim 1, claim 4 or claim 7 wherein: the classifier classifies data into classes; the uncertainty measuring device generator includes: means for producing a class membership score for each annotated uncertain example, and means for using logistic regression to produce a parameter for each annotated uncertain example which modifies the class membership score to indicate a probability that the annotated uncertain example belongs to the class.
Claim 16
Dependent Claim
25 words
The apparatus of claim 7 further comprising an uncertainty measuring device computes an estimate of the entropy of the classification to each unannotated data portion.
Claim 17
Dependent Claim
16 words
The apparatus of claim 7, wherein: said examples are divided into data portions by a segmenter.
Claim 18
Dependent Claim
32 words
The apparatus set forth in claim 1, claim 4 or claim 7, wherein each of the respective low certainty values is less than a lowest one of the respective high certainty values.
References Cited (Backward Citations)
US5402519A
Cited
US5251131A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-09-22 | 2001-03-24 | 2001-09-23 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-09-22 | 2005-03-24 | 2005-09-23 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-09-22 | 2009-03-24 | 2009-09-23 | $7700.00 | Window Closed / Lapsed |