Patent US5075868A successfully refreshed from Google Patents.
US5075868A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Memory modification of artificial neural networks
Assignee: United States Department of the Navy
•
Filing: 1989-09-18
•
Granted: 1991-12-24
•
Expires: 2009-09-18
Abstract
Abstract An artificial neural network, which has a plurality of neurons each receiving a plurality of inputs whose effect is determined by adjust able weights at synapses individually connecting the inputs to the neuron to provide a sum signal to a sigmoidal function generator determining the output of the neuron, undergoes memory modification by a steepest-descent method in which individual variations in the outputs of the neurons are successively generated by small perturbations imposed on the sum signals. As each variation is generated on the output of a neuron, an overall error of all the neuron outputs in relation to their desired values is measured and compared to this error prior to the perturbation. The difference in these errors, with adjustments which may be changed as the neuron outputs converge toward their desired values, is used to modify each weight of the neuron presently subjected to the perturbation.
Inventors
- 👤 David K. Andes
- 👤 Robert A. Licklider
- 👤 Donald H. Witcher
- 👤 Richard M. Swenson
- 👤 James F. Barbieri
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-30 17:57
CPC Classifications
G; G06; G06N; G06N3/00; G06N3/02; G06N3/06; G06N3/063
Claims Analysis
Total Claims: 11
Claim 1
Independent Claim
287 words
A memory modification method for use with an artificial neural network having: a plurality of inputs, each input providing a corresponding input signal; and a plurality of neurons having a plurality of synapses individually corresponding to said inputs, each synapse having means for storing a weight individual to the synapse and having means for generating a synapse output signal representing the product of said weight and the input signal corresponding to the synapse, means for generating a sum signal representing the sum of every synapse output signal of said plurality of synapses, and means for generating a neuron output signal determined by a predetermined function of said sum signal, and with teaching means for: receiving every neuron output signal from said plurality of neurons; generating an error signal representing the sum of the differences between every neuron output signal and a desired output signal corresponding thereto; changing selectively said weights of said synapses of each of said plurality of neurons; and changing selectively every sum signal of said plurality of neurons, the memory modification method comprising: said function being a differentiable function, and the method including the steps of: changing successively and by a predetermined perturbation every sum signal of said plurality of neurons; generating a trial such error signal when said sum signal of each one of said neurons is changed by said predetermined perturbation and such weights are the weights resulting in a previously generated such error signal generated when the sum signal of said one of said neurons is not changed by said predetermined perturbation; and changing the weight of every synapse of said one neuron by an adjustment determined by the difference between said previously generated error signal and said trial error signal.
Claim 2
Dependent Claim
287 words
A memory modification method for use with an artificial neural network having: a plurality of inputs, each input providing a corresponding input signal; and a plurality of neurons having a plurality of synapses individually corresponding to said inputs, each synapse having means for storing a weight individual to the synapse and having means for generating a synapse output signal representing the product of said weight and the input signal corresponding to the synapse, means for generating a sum signal representing the sum of every synapse output signal of said plurality of synapses, and means for generating a neuron output signal determined by a predetermined function of said sum signal, and with teaching means for: receiving every neuron output signal from said plurality of neurons; generating an error signal representing the sum of the differences between every neuron output signal and a desired output signal corresponding thereto; changing selectively said weights of said synapses of each of said plurality of neurons; and changing selectively every sum signal of said plurality of neurons, the memory modification method comprising: said function being a differentiable function, and the method including the steps of: changing successively and by a predetermined perturbation every sum signal of said plurality of neurons; generating a trial such error signal when said sum signal of each one of said neurons is changed by said predetermined perturbation and such weights are the weights resulting in a previously generated such error signal generated when the sum signal of said one of said neurons is not changed by said predetermined perturbation; and changing the weight of every synapse of said one neuron by an adjustment determined by the difference between said previously generated error signal and said trial error signal.
Claim 3
Dependent Claim
14 words
The memory modification method of claim 1 wherein said differentiable function is monotonically nondecreasing.
Claim 4
Dependent Claim
59 words
The memory modification method of claim 1 wherein, at each synapse of said one neuron, said adjustment is proportional to the difference between said previously generated error signal and said trial error signal, is proportional to said perturbation, and is in a direction determined by the input signal provided by the one of said inputs corresponding to the synapse.
Claim 5
Dependent Claim
38 words
The memory modification method of claim 3 wherein said input signal provided by the one of said inputs corresponding to the synapse is a signed signal and wherein said direction is determined by the sign of said signal.
Claim 6
Dependent Claim
25 words
The memory modification method of claim 3 wherein said adjustment is also proportional to the input signal provided by the input corresponding to the synapse.
Claim 7
Dependent Claim
46 words
The memory modification method of claim 1 wherein said adjustment is in one direction when said previously generated error signal is greater than said trial error signal and is in the opposite direction when said previously generated error signal is less than said trial error signal.
Claim 8
Dependent Claim
25 words
The memory modification method of claim 1 wherein said adjustment is proportional to a gain factor which is decreased as such an error signal decreases.
Claim 9
Dependent Claim
33 words
The memory modification method of claim 1 wherein said error signal is determined by the sum of the squares of the differences between every neuron output signal and the corresponding desired output signal.
Claim 10
Dependent Claim
17 words
The memory modification method of claim 1 wherein said perturbation is proportional to such an error signal.
Claim 11
Dependent Claim
47 words
The memory modification method of claim 1 wherein the method further comprises changing the weight of every synapse of said one neuron by said adjustment only if the difference between said previously generated error signal and said trial error signal is greater than a predetermined threshold difference.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1994-12-23 | 1995-06-24 | 1995-12-24 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1998-12-23 | 1999-06-24 | 1999-12-24 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2002-12-23 | 2003-06-24 | 2003-12-24 | $7700.00 | Window Closed / Lapsed |