US5903884A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method for training a statistical classifier with reduced tendency for overfitting
Assignee: Apple Inc
•
Filing: 1995-08-08
•
Granted: 1999-05-11
•
Expires: 2015-08-08
Abstract
Abstract To prevent overfitting a neural network to a finite set of training samples, random distortions are dynamically applied to the samples each time they are applied to the network during a training session. A plurality of different types of distortions can be applied, which are randomly selected each time a sample is applied to the network. Alternatively, a combination of two or more types of distortion can be applied each time, with the amount of distortion being randomly varied for each type.
Inventors
- 👤 Richard F. Lyon
- 👤 William Stafford
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 17:16
CPC Classifications
G; G06; G06F; G06F18/00; G06F18/20; G06F18/28
Claims Analysis
Total Claims: 28
Claim 1
Independent Claim
85 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, utilizing a set of training samples which are respectively associated with said classes, comprising the following steps which are repeated over a large number of iterations: selecting a training sample from said set of training samples; producing a set of distortion parameters; selectively distorting said training sample in accordance with said distortion parameters to compute a classifier input pattern; and training the classifier using said classifier input pattern.
Claim 2
Independent Claim
85 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, utilizing a set of training samples which are respectively associated with said classes, comprising the following steps which are repeated over a large number of iterations: selecting a training sample from said set of training samples; producing a set of distortion parameters; selectively distorting said training sample in accordance with said distortion parameters to compute a classifier input pattern; and training the classifier using said classifier input pattern.
Claim 3
Dependent Claim
18 words
The training method of claim 1 wherein said distortion parameters are produced according to an effectively random process.
Claim 4
Dependent Claim
17 words
The training method of claim 1 wherein said distortion parameters are produced according to a deterministic pattern.
Claim 5
Dependent Claim
42 words
The training method of claim 1 wherein said distorting step further comprises the steps of: computing said classifier input pattern from said training sample, substantially independent of said distortion parameters; and distorting said classifier input pattern in accordance with said distortion parameters.
Claim 6
Dependent Claim
20 words
The method of claim 4 wherein the step of distorting said classifier input pattern comprises distorting images of said patterns.
Claim 7
Dependent Claim
48 words
The training method of claim 1 wherein said distorting step further comprises the steps of: distorting a first data set representative of said training sample in accordance with said distortion parameters; and converting said first data set to a second data set representative of said classifier input pattern.
Claim 8
Dependent Claim
36 words
The method of claim 6 wherein said first data set comprises x and y coordinates representative of handwritten characters and the step of distorting said first data set comprises distorting said x and y coordinate values.
Claim 9
Dependent Claim
16 words
The method of claim 1 wherein said set of distortion parameters comprise plural types of distortion.
Claim 10
Dependent Claim
26 words
The method of claim 8 wherein said step of producing said set of distortion parameters includes the step of selecting one or more types of distortion.
Claim 11
Dependent Claim
30 words
The method of claim 9 further including the step of randomly selecting an amount of distortion for each type of distortion to be applied to said images during each iteration.
Claim 12
Dependent Claim
20 words
The method of claim 9 wherein one of said types of distortion comprises varying the aspect ratio of an image.
Claim 13
Dependent Claim
16 words
The method of claim 9 wherein one of said types of distortion comprises rotating an image.
Claim 14
Dependent Claim
41 words
The method of claim 1 wherein said training sample is repeatedly used to train the classifier over multiple iterations, and a new set of distortion parameters is produced for each iteration to dynamically compute different classifier input patterns for each iteration.
Claim 15
Dependent Claim
33 words
The method of claim 1 wherein a plurality of training samples are successively selected from said set of training samples, and a new set of distortion parameters is produced for each selected sample.
Claim 16
Independent Claim
112 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, comprising the steps of: (i) generating a set of training sample patterns which are respectively associated with said classes; (ii) processing sample patterns from said set in the classifier to generate output values; (iii) determining error values based on differences between said output values and target values associated with said sample patterns; (iv) adjusting operating parameters of the classifier in accordance with said error values; (v) iteratively repeating steps (ii)-(iv) with the sample patterns from said set; and (vi) modifying said patterns prior to processing them in the classifier, during successive iterations of steps (ii)-(iv).
Claim 17
Independent Claim
112 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, comprising the steps of: (i) generating a set of training sample patterns which are respectively associated with said classes; (ii) processing sample patterns from said set in the classifier to generate output values; (iii) determining error values based on differences between said output values and target values associated with said sample patterns; (iv) adjusting operating parameters of the classifier in accordance with said error values; (v) iteratively repeating steps (ii)-(iv) with the sample patterns from said set; and (vi) modifying said patterns prior to processing them in the classifier, during successive iterations of steps (ii)-(iv).
Claim 18
Dependent Claim
18 words
The method of claim 15 wherein the step of modifying said patterns comprises distorting images of said patterns.
Claim 19
Dependent Claim
15 words
The method of claim 16 wherein plural types of distortion are applied to said images.
Claim 20
Dependent Claim
27 words
The method of claim 17 further including the step of randomly selecting one or more types of distortions to be applied to said images during each iteration.
Claim 21
Dependent Claim
30 words
The method of claim 17 further including the step of randomly selecting an amount of distortion for each type of distortion to be applied to said images during each iteration.
Claim 22
Dependent Claim
20 words
The method of claim 17 wherein one of said types of distortion comprises varying the aspect ratio of an image.
Claim 23
Dependent Claim
16 words
The method of claim 17 wherein one of said types of distortion comprises rotating an image.
Claim 24
Dependent Claim
20 words
The method of claim 17 wherein one of said types of distortion comprises varying the angular orientation of an image.
Claim 25
Independent Claim
135 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, utilizing a set of training samples which are respectively associated with said classes, comprising the following steps: selecting a first training sample from said set of training samples; producing a first set of distortion parameters; selectively distorting said first training sample in accordance with said first set of distortion parameters to dynamically compute a first classifier input pattern; training the classifier using said first classifier input pattern; selecting a second training sample from said set of training samples; producing a second set of distortion parameters; selectively distorting said second training sample in accordance with said second set of distortion parameters to dynamically compute a second classifier input pattern; and training the classifier using said second classifier input pattern.
Claim 26
Independent Claim
135 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, utilizing a set of training samples which are respectively associated with said classes, comprising the following steps: selecting a first training sample from said set of training samples; producing a first set of distortion parameters; selectively distorting said first training sample in accordance with said first set of distortion parameters to dynamically compute a first classifier input pattern; training the classifier using said first classifier input pattern; selecting a second training sample from said set of training samples; producing a second set of distortion parameters; selectively distorting said second training sample in accordance with said second set of distortion parameters to dynamically compute a second classifier input pattern; and training the classifier using said second classifier input pattern.
Claim 27
Independent Claim
130 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, utilizing a set of training samples which are respectively associated with said classes, comprising the following steps: selecting a training sample from said set of training samples; producing a first set of distortion parameters; selectively distorting said training sample in accordance with said first set of distortion parameters to compute a first classifier input pattern; training the classifier using said first classifier input pattern; subsequently selecting said training sample from said set of training samples; producing a second set of distortion parameters; selectively distorting said training sample in accordance with said second set of distortion parameters, to compute a second classifier input pattern; and training the classifier using said second classifier input pattern.
Claim 28
Independent Claim
130 words
A method for training a statistical classifier to recognize input patterns that belong to respective predetermined classes, utilizing a set of training samples which are respectively associated with said classes, comprising the following steps: selecting a training sample from said set of training samples; producing a first set of distortion parameters; selectively distorting said training sample in accordance with said first set of distortion parameters to compute a first classifier input pattern; training the classifier using said first classifier input pattern; subsequently selecting said training sample from said set of training samples; producing a second set of distortion parameters; selectively distorting said training sample in accordance with said second set of distortion parameters, to compute a second classifier input pattern; and training the classifier using said second classifier input pattern.
References Cited (Backward Citations)
US5297237A
Cited
US5299284A
Cited
US5058180A
Cited
US5390285A
Cited
US5546503A
Cited
US5239594A
Cited
US5359699A
Cited
US5276771A
Cited
US5333210A
Cited
US5469528A
Cited
US5590218A
Cited
US5504780A
Cited
US5640468A
Cited
US5572628A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-05-10 | 2002-11-09 | 2003-05-11 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-05-10 | 2006-11-09 | 2007-05-11 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-05-10 | 2010-11-09 | 2011-05-11 | $7700.00 | Window Closed / Lapsed |