US6513025B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Multistage machine learning process
Assignee: Tollgrade Communications Inc
•
Filing: 1999-12-09
•
Granted: 2003-01-28
•
Expires: 2019-12-09
Abstract
Abstract The present invention includes a mechanism for applying expert knowledge and machine-learning routines to a continuous stream of information. The present method comprises learning a set of dependability models, one for each classification model, that characterize the situations in which each of the classification models is able to make correct predictions. At appropriate intervals the method produces new fault localization knowledge in the form of decision-tree based classification and dependability models. Such knowledge is used to enhance the existing classification knowledge already available. Each of these classification models has a particular sub-domain where it is the most reliable, and hence the best choice to use. For future unlabeled examples, these dependability models are consulted to select the most appropriate classification model, and the prediction of that classification model is then accepted.
Inventors
- 👤 Joseph S. Rosen
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 17:19
CPC Classifications
G; G06; G06N; G06N5/00; G06N5/02; G06N5/022; G06N5/025
Claims Analysis
Total Claims: 24
Claim 1
Independent Claim
263 words
A method of operating an expert system to diagnose service affecting conditions on lines in a telecommunications network comprising: providing a set of training data, the data set made up of records with each record containing parameters indicative of measurements made on a line and a state of the line at the time the measurements were made; selecting a plurality of subsets of the training data, wherein at least a first one of the subsets represents training data gathered during a first interval and at least a second one of the subsets representing training data gathered during a second interval after the first interval; building a plurality of classification models, one from each of said plurality of subsets of the training data; building a plurality of dependability models, one for each of said plurality of classification models; providing test data derived from a line in the telecommunication network to each of said dependability models; selecting a dependability model indicating the highest dependability from said plurality of dependability models; and classifying the test data with the classification model associated with the dependability model indicating the highest dependability providing additional training data during a third interval after the second interval; using the additional training data to create additional classification models and additional dependability models; replacing the classification models and dependability models created with a first of the subset of data with the additional classification models and additional dependability models; and thereafter applying test data on a line in the telecommunication network gathered after the third interval to the dependability models to select a classification model.
Claim 2
Independent Claim
263 words
A method of operating an expert system to diagnose service affecting conditions on lines in a telecommunications network comprising: providing a set of training data, the data set made up of records with each record containing parameters indicative of measurements made on a line and a state of the line at the time the measurements were made; selecting a plurality of subsets of the training data, wherein at least a first one of the subsets represents training data gathered during a first interval and at least a second one of the subsets representing training data gathered during a second interval after the first interval; building a plurality of classification models, one from each of said plurality of subsets of the training data; building a plurality of dependability models, one for each of said plurality of classification models; providing test data derived from a line in the telecommunication network to each of said dependability models; selecting a dependability model indicating the highest dependability from said plurality of dependability models; and classifying the test data with the classification model associated with the dependability model indicating the highest dependability providing additional training data during a third interval after the second interval; using the additional training data to create additional classification models and additional dependability models; replacing the classification models and dependability models created with a first of the subset of data with the additional classification models and additional dependability models; and thereafter applying test data on a line in the telecommunication network gathered after the third interval to the dependability models to select a classification model.
Claim 3
Dependent Claim
40 words
The method of claim 1 wherein said step of building a plurality of classification models comprises building classification models of the type selected from the group consisting of a decision tree model, an expert system model and a case-based model.
Claim 4
Independent Claim
114 words
A method of operating an expert system comprising: providing a set of training data; selecting a plurality of subsets of the training data; building a plurality of classification models, one from each of said plurality of subsets of the training data; building a plurality of dependability models, one for each of said plurality of classification models; providing test data to each of said dependability models; selecting a dependability model indicating the highest dependability from said plurality of dependability models; and classifying the test data with the classification model associated with the dependability model indicating the highest dependability; wherein said step of building a plurality of dependability models comprises building decision tree based dependability models.
Claim 5
Independent Claim
114 words
A method of operating an expert system comprising: providing a set of training data; selecting a plurality of subsets of the training data; building a plurality of classification models, one from each of said plurality of subsets of the training data; building a plurality of dependability models, one for each of said plurality of classification models; providing test data to each of said dependability models; selecting a dependability model indicating the highest dependability from said plurality of dependability models; and classifying the test data with the classification model associated with the dependability model indicating the highest dependability; wherein said step of building a plurality of dependability models comprises building decision tree based dependability models.
Claim 6
Dependent Claim
32 words
The method of claim 1 wherein said step of building a plurality of classification models comprises building a plurality of classification model having an output which is an estimated class probability vector.
Claim 7
Dependent Claim
27 words
The method of claim 1 wherein said step of providing a set of training data includes the step of periodically providing a new set of training data.
Claim 8
Dependent Claim
27 words
The method of claim 5 wherein said step of periodically providing a new set of data comprises providing loop failure information data acquired since a previous cycle.
Claim 9
Dependent Claim
30 words
The method of claim 1 wherein said step of providing a set of training data further comprises the step of applying feature mapping rules to an initial set of data.
Claim 10
Dependent Claim
36 words
The method of claim 7 wherein said step of providing a set of training data further comprises the step of augmenting the initial set of data with derived features to promote more accurate classification model construction.
Claim 11
Dependent Claim
33 words
The method of claim 1 wherein said step of providing a set of training data further comprises the step of performing a plausibility analysis which removes or reclassifies data having implausible class labels.
Claim 12
Dependent Claim
20 words
The method of claim 1 wherein providing additional training data comprises providing the new data at a predetermined cycle time.
Claim 13
Dependent Claim
22 words
The method of claim 10 wherein said step of providing a new set of data at a predetermined cycle time is adjustable.
Claim 14
Dependent Claim
24 words
The method of claim 1 wherein said step of providing a set of training data further comprises the step of eliminating misclassified training data.
Claim 15
Dependent Claim
36 words
The method of claim 1 wherein said step of selecting a plurality of subsets of the training data comprises the step of utilizing a randomization technique to select a feature to partition the training data by.
Claim 16
Dependent Claim
32 words
The method of claim 13 wherein said step of utilizing a randomization technique comprises randomly selecting from among all the features having information gain ratio values within a feature-selection-percent of each other.
Claim 17
Dependent Claim
30 words
The method of claim 1 , after said step of building a plurality of classification models, including the step of keeping up to a predetermined maximum number of classification models.
Claim 18
Dependent Claim
24 words
The method of claim 1 wherein said step of providing a set of training data comprises providing a set of subscriber loop test records.
Claim 19
Dependent Claim
39 words
The method of claim 16 wherein said step of building a plurality of dependability models comprises building dependability models which predict whether the class of an unclassified loop test example will be predicted correctly by a corresponding classification model.
Claim 20
Dependent Claim
24 words
The method of claim 1 wherein said step of building a plurality of dependability models includes the step of forming an initial training set.
Claim 21
Independent Claim
121 words
A method of operating an expert system comprising: providing a set of training data; selecting a plurality of subsets of the training data; building a plurality of classification models, one from each of said plurality of subsets of the training data; building a plurality of dependability models, one for each of said plurality of classification models; providing test data to each of said dependability models; selecting a dependability model indicating the highest dependability from said plurality of dependability models; and classifying the test data with the classification model associated with the dependability model indicating the highest dependability; wherein said step of building a plurality of dependability models includes the step of computing a dependability training set from said initial training set.
Claim 22
Independent Claim
121 words
A method of operating an expert system comprising: providing a set of training data; selecting a plurality of subsets of the training data; building a plurality of classification models, one from each of said plurality of subsets of the training data; building a plurality of dependability models, one for each of said plurality of classification models; providing test data to each of said dependability models; selecting a dependability model indicating the highest dependability from said plurality of dependability models; and classifying the test data with the classification model associated with the dependability model indicating the highest dependability; wherein said step of building a plurality of dependability models includes the step of computing a dependability training set from said initial training set.
Claim 23
Dependent Claim
25 words
The method of claim 19 wherein said step building a plurality of dependability models includes learning a dependability model based on said dependability training set.
Claim 24
Dependent Claim
23 words
The method of claim 1 wherein said step of providing a set of training data comprises providing a set of fault localization records.
References Cited (Backward Citations)
US4697242A
Cited
US4845739A
Cited
US4881178A
Cited
US5636344A
Cited
US5317725A
Cited
US5377196A
Cited
US5870730A
Cited
US5781698A
Cited
US6272479B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-01-27 | 2006-07-29 | 2007-01-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-01-27 | 2010-07-29 | 2011-01-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-01-27 | 2014-07-29 | 2015-01-28 | $7700.00 | Window Closed / Lapsed |