US5371834A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Adaptive neuron model--an architecture for the rapid learning of nonlinear topological transformations
Assignee: United States, ADMINISTRATOR OF NATIONAL AERONAUTICS AND SPACE ADMINISTRATOR, Secretary of
•
Filing: 1992-08-28
•
Granted: 1994-12-06
•
Expires: 2012-08-28
Abstract
Abstract A method and an apparatus for the rapid learning of nonlinear mappings and topological transformations using a dynamically reconfigurable artificial neural network is presented. This fully-recurrent Adaptive Neuron Model (ANM) network has been applied to the highly degenerative inverse kinematics problem in robotics, and its performance evaluation is bench-marked. Once trained, the resulting neuromorphic architecture was implemented in custom analog neural network hardware and the parameters capturing the functional transformation downloaded onto the system. This neuroprocessor, capable of 10 9 ops/sec, was interfaced directly to a three degree of freedom Heathkit robotic manipulator. Calculation of the hardware feed-forward pass for this mapping was benchmarked at ≈10 μsec.
Inventors
- 👤 Raoul Tawel
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-31 17:30
CPC Classifications
B; B25; B25J; B25J9/00; B25J9/16; B25J9/1602; B25J9/1607; G; G06; G06F; G06F18/00; G06F18/20; G06F18/21; G06F18/213; G06F18/2137; G06N; G06N3/00; G06N3/02; G06N3/04; G06N3/044; G06N3/0495; G06N3/0499; G06N3/08; G06N3/09; G05; G05B; G05B2219/00; G05B2219/30; G05B2219/39; G05B2219/39271
Claims Analysis
Total Claims: 20
Claim 1
Independent Claim
163 words
A method of training an analog neural network comprising plural neurons and synapses wherein said neurons are connected together by respective synapses, said neurons comprising respective activity states and adjustable neuron temperatures, said synapses comprising adjustable synapse weights, said method comprising the steps of: defining, relative to an error between activity states of a set of output neurons and a predetermined training vector, predetermined time-dependent behaviors of: (a) said activity states, (b) said neuron temperatures in accordance with a gradient descent of said error in temperature space and (c) said synapse weights in accordance with a gradient descent of said error in weight space, said behaviors governed by (a) an activity state relaxation time, (b) a neuron temperature relaxation time and (c) a synapse weight relaxation time, respectively; continuously updating said neuron activity states, said neuron temperatures and said synapse weights of said analog neural network at respective rates corresponding to said relaxation times until said error is reduced below a predetermined threshold.
Claim 2
Independent Claim
163 words
A method of training an analog neural network comprising plural neurons and synapses wherein said neurons are connected together by respective synapses, said neurons comprising respective activity states and adjustable neuron temperatures, said synapses comprising adjustable synapse weights, said method comprising the steps of: defining, relative to an error between activity states of a set of output neurons and a predetermined training vector, predetermined time-dependent behaviors of: (a) said activity states, (b) said neuron temperatures in accordance with a gradient descent of said error in temperature space and (c) said synapse weights in accordance with a gradient descent of said error in weight space, said behaviors governed by (a) an activity state relaxation time, (b) a neuron temperature relaxation time and (c) a synapse weight relaxation time, respectively; continuously updating said neuron activity states, said neuron temperatures and said synapse weights of said analog neural network at respective rates corresponding to said relaxation times until said error is reduced below a predetermined threshold.
Claim 3
Dependent Claim
54 words
The method of claim 1 wherein said activity state relaxation time is shorter than said neuron temperature relaxation time and shorter than said synapse weight relaxation time, whereby for a given set of values of said neuron temperatures and synapse weights, said neuron activity states equilibrate before changes in said set of values occur.
Claim 4
Dependent Claim
86 words
The method of claim 2 wherein: said behavior of said activity states is an activity state differential equation for each neuron governing a time derivative of a corresponding activity state in terms of a product of a corresponding neuron temperature multiplied by a sum of products of corresponding ones of said synapse weights and corresponding ones of said activity states; and said continuously updating comprises solving said activity state differential equation for said activity state repetitively at a rate corresponding to said activity state relaxation time.
Claim 5
Dependent Claim
91 words
The method of claim 3 wherein: said behavior of said neuron temperatures is a temperature differential equation for each neuron defining a time derivative of a corresponding neuron temperature as a product of an exponential function of said neuron temperature and said error between activity states of the set of output neurons and the predetermined training vector transformed by a matrix of said synapse weights; and said continuously updating step further comprises solving said temperature differential equation for said neuron temperature repetitively at a rate corresponding to said temperature relaxation time.
Claim 6
Dependent Claim
76 words
The method of claim 4 wherein: said behavior of said synapse weights is a weight differential equation for each synapse defining a time derivative of a corresponding synapse weight as a product of an exponential function of the corresponding neuron temperature and a sum of products of said synapse weights; and said continuously updating step further comprises solving said weight differential equation for said synapse weight repetitively at a rate corresponding to said weight relaxation time.
Claim 7
Dependent Claim
60 words
The method of claim 5 wherein said activity state differential equation, temperature differential equation, and weight differential equation are solved in an ascending order associated with an order of ascending relaxation times, such that a result of a solving of a preceding differential equation in the ascending order is employed in solving a next differential equation in the ascending order.
Claim 8
Dependent Claim
15 words
The method of claim 1 wherein said neural network is a fully recurrent neural network.
Claim 9
Dependent Claim
27 words
The method of claim 1 wherein said predetermined training vector corresponds to a classification problem and said temperature relaxation time is shorter than said weight relaxation time.
Claim 10
Dependent Claim
28 words
The method of claim 1 wherein said predetermined training vector corresponds to a continuous mapping relation, and said weight relaxation time is shorter than said temperature relaxation time.
Claim 11
Dependent Claim
165 words
Apparatus for training an analog neural network comprising plural neurons and synapses wherein said neurons are connected together by respective synapses, said neurons comprising respective activity states and adjustable neuron temperatures, said synapses comprising adjustable synapse weights, said apparatus comprising: means for defining and storing, relative to an error between activity states of a set of output neurons and a predetermined training vector, predetermined time-dependent behaviors of: (a) said activity states, (b) said neuron temperatures in accordance with a gradient descent of said error in temperature space and (c) said synapse weights in accordance with a gradient descent of said error in weight space, said behaviors governed by (a) an activity state relaxation time, (b) a neuron temperature relaxation time and (c) a synapse weight relaxation time, respectively; means for continuously updating said neuron activity states, said neuron temperatures and said synapse weights of said analog neural network at respective rates corresponding to said relaxation times until said error is reduced below a predetermined threshold.
Claim 12
Dependent Claim
165 words
Apparatus for training an analog neural network comprising plural neurons and synapses wherein said neurons are connected together by respective synapses, said neurons comprising respective activity states and adjustable neuron temperatures, said synapses comprising adjustable synapse weights, said apparatus comprising: means for defining and storing, relative to an error between activity states of a set of output neurons and a predetermined training vector, predetermined time-dependent behaviors of: (a) said activity states, (b) said neuron temperatures in accordance with a gradient descent of said error in temperature space and (c) said synapse weights in accordance with a gradient descent of said error in weight space, said behaviors governed by (a) an activity state relaxation time, (b) a neuron temperature relaxation time and (c) a synapse weight relaxation time, respectively; means for continuously updating said neuron activity states, said neuron temperatures and said synapse weights of said analog neural network at respective rates corresponding to said relaxation times until said error is reduced below a predetermined threshold.
Claim 13
Dependent Claim
54 words
The apparatus of claim 10 wherein said activity state relaxation time is shorter than said neuron temperature relaxation time and shorter than said synapse weight relaxation time, whereby for a given set of values of said neuron temperatures and synapse weights, said neuron activity states equilibrate before changes in said set of values occur.
Claim 14
Dependent Claim
90 words
The apparatus of claim 10 wherein: said behavior of said activity states is an activity state differential equation for each neuron governing a time derivative of a corresponding activity state in terms of a product of a corresponding neuron temperature multiplied by a sum of products of corresponding ones of said synapse weights and corresponding ones of said activity states; and said means for continuously updating comprises means for solving said activity state differential equation for said activity state repetitively at a rate corresponding to said activity state relaxation time.
Claim 15
Dependent Claim
94 words
The apparatus of claim 12 wherein: said behavior of said neuron temperatures is a temperature differential equation for each neuron governing a time derivative of a corresponding neuron temperature as a product of an exponential function of said neuron temperature and said error between activity states of the set of output neurons and the predetermined training vector transformed by a matrix of said synapse weights; and said means for continuously updating further comprises means for solving said temperature differential equation for said neuron temperature repetitively at a rate corresponding to said temperature relaxation time.
Claim 16
Dependent Claim
79 words
The apparatus of claim 13 wherein: said behavior of said synapse weights is a weight differential equation for each synapse defining a time derivative of a corresponding synapse weight as a product of an exponential function of the corresponding neuron temperature and a sum of products of said synapse weights; and said means for continuously updating further comprises means for solving said weight differential equation for said synapse weight repetitively at a rate corresponding to said weight relaxation time.
Claim 17
Dependent Claim
66 words
The apparatus of claim 14 wherein said means for solving comprises means for solving said activity state differential equation, temperature differential equation, and weight differential equation in an ascending order associated with an order of ascending relaxation times, such that a result of a solving of a preceding differential equation in the ascending order is employed in solving a next differential equation in the ascending order.
Claim 18
Dependent Claim
27 words
The apparatus of claim 15 wherein said predetermined training vector corresponds to a classification problem and said temperature relaxation time is shorter than said weight relaxation time.
Claim 19
Dependent Claim
28 words
The apparatus of claim 15 wherein said predetermined training vector corresponds to a continuous mapping relation, and said weight relaxation time is shorter than said temperature relaxation time.
Claim 20
Dependent Claim
14 words
The method of claim 10 wherein said network is a fully recurrent neural network,
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1997-12-05 | 1998-06-06 | 1998-12-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2001-12-05 | 2002-06-06 | 2002-12-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2005-12-05 | 2006-06-06 | 2006-12-06 | $7700.00 | Window Closed / Lapsed |