US5249259A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Genetic algorithm technique for designing neural networks
Assignee: Massachusetts Institute of Technology
•
Filing: 1992-05-28
•
Granted: 1993-09-28
•
Expires: 2012-05-28
Abstract
Abstract A generic algorithm search is applied to determine an optimum set of values (e.g., interconnection weights in a neural network), each value being associated with a pair of elements drawn from a universe of N elements, N an integer greater than zero, where the utility of any possible set of said values may be measured. An initial possible set of values is assembled, the values being organized in a matrix whose rows and columns correspond to the elements. A genetic algorithm operator is applied to generate successor matrices from said matrix. Matrix computations are performed on the successor matrices to generate measures of the relative utilities of the successor matrices. A surviving matrix is selected from the successor matrices on the basis of the metrics. The steps are repeated until the metric of the surviving matrix is satisfactory.
Inventors
- 👤 Robert L. Harvey
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-26 04:36
CPC Classifications
G; G06; G06N; G06N3/00; G06N3/12; G06N3/126
Claims Analysis
Total Claims: 16
Claim 1
Independent Claim
337 words
A computer method for implementing a neural network for use in a selected application, said neural network being implemented to effect a desired relationship between inputs representing real world phenomena and outputs corresponding to the inputs, said neural network having neurons and neurons interconnections each of said neurons possibly having an interconnection to any other of said neurons, each of said interconnections characterized by an associated weight value, said weight values having potentially any value including zero, positive, or negative, the method comprising (a) by computer, searching possible sets of weight values and selecting a set of weight values for use in the neural network, said step of searching and selecting comprising (i) picking an initial set of weight values; (ii) organizing said initial set of weight values in rows and columns, so the weight values in each of said rows all are associated with interconnections to the same one of said neurons, and the weight values in each of said columns all are associated with interconnections from the same one of said neurons; (iii) generating successor sets of weight values from said initial set of weight values; (iv) for each of said successor sets of weight values, determining test outputs of a neural network having those weights, when activated by predetermined inputs; (v) analyzing the merits of each of said successor sets of weight values based on how well the test outputs conform to predetermined outputs; (vi) selecting a single surviving set of weight values from among said successor sets of weight values based on the relative merits of said surviving set of weight values, said surviving set of weight values serving as an initial set; and (vii) if the merits of said surviving set of weight values meet a threshold, treating said surviving set of weight values as a final set of weight values; otherwise repeating steps (iii)-(vi) using said surviving set of weights as a new initial set of weight values; and (b) implementing said neural network with said final set of weight values.
Claim 2
Dependent Claim
337 words
A computer method for implementing a neural network for use in a selected application, said neural network being implemented to effect a desired relationship between inputs representing real world phenomena and outputs corresponding to the inputs, said neural network having neurons and neurons interconnections each of said neurons possibly having an interconnection to any other of said neurons, each of said interconnections characterized by an associated weight value, said weight values having potentially any value including zero, positive, or negative, the method comprising (a) by computer, searching possible sets of weight values and selecting a set of weight values for use in the neural network, said step of searching and selecting comprising (i) picking an initial set of weight values; (ii) organizing said initial set of weight values in rows and columns, so the weight values in each of said rows all are associated with interconnections to the same one of said neurons, and the weight values in each of said columns all are associated with interconnections from the same one of said neurons; (iii) generating successor sets of weight values from said initial set of weight values; (iv) for each of said successor sets of weight values, determining test outputs of a neural network having those weights, when activated by predetermined inputs; (v) analyzing the merits of each of said successor sets of weight values based on how well the test outputs conform to predetermined outputs; (vi) selecting a single surviving set of weight values from among said successor sets of weight values based on the relative merits of said surviving set of weight values, said surviving set of weight values serving as an initial set; and (vii) if the merits of said surviving set of weight values meet a threshold, treating said surviving set of weight values as a final set of weight values; otherwise repeating steps (iii)-(vi) using said surviving set of weights as a new initial set of weight values; and (b) implementing said neural network with said final set of weight values.
Claim 3
Dependent Claim
22 words
The method of claim 1 further comprising constraining said interconnection weight values to comply with constraints that correspond to said selected application.
Claim 4
Dependent Claim
22 words
The method of claim 2 wherein said constraints include requiring that the interconnection weight values between a neuron and itself be zero.
Claim 5
Dependent Claim
26 words
The method of claim 2 wherein said constraints include requiring that the interconnection weight values leading from a given neuron all be of the same sign.
Claim 6
Dependent Claim
58 words
The method of claim 1 wherein said step of organizing said interconnection weight values comprises organizing said weight values with the interconnection weights leading to a given neuron all appearing in a single row (or a single column), and the interconnection weight values leading from a given neuron all appearing in a single column (or a single row).
Claim 7
Dependent Claim
30 words
The method of claim 5 wherein said rows and columns of interconnection weight values are organized in a number of matrices based on layers of neurons in said neural network.
Claim 8
Dependent Claim
25 words
The method of claim 6 wherein said neural network comprises a layer of input neurons, a layer of hidden neurons, and a single output neuron.
Claim 9
Dependent Claim
18 words
The method of claim 1 wherein said predetermined input comprises binary values which represent edges of an image.
Claim 10
Dependent Claim
28 words
The method of claim 1 wherein said step of generating successor sets of weight values comprises applying a genetic algorithm operator to said initial set of weight values.
Claim 11
Dependent Claim
19 words
The method of claim 9 wherein said step of applying a genetic algorithm operator comprises applying a mutation operator.
Claim 12
Dependent Claim
20 words
The method of claim 10 wherein applying said mutation operator comprises randomly changing a portion of said interconnection weight values.
Claim 13
Dependent Claim
36 words
The method of claim 1 wherein said step of analyzing the relative merits of the successor sets of weight values comprises computing a metric corresponding to the difference between said test outputs and said predetermined outputs.
Claim 14
Dependent Claim
27 words
The method of claim 12 wherein said step of selecting a surviving matrix comprises testing whether said metric lies within a band between upper and lower bounds.
Claim 15
Dependent Claim
27 words
The method of claim 12 wherein said step of selecting a surviving matrix comprises testing whether said metric lies above a passband or below a stop band.
Claim 16
Dependent Claim
19 words
The method of claim 1 wherein said predetermined input comprises gray gradient values which represent edges of an image.
References Cited (Backward Citations)
US4697242A
Cited
US4881178A
Cited
US5155802A
Cited
US4935877A
Cited
US4972363A
Cited
US5140530A
Cited
US5119469A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1996-09-27 | 1997-03-29 | 1997-09-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2000-09-27 | 2001-03-29 | 2001-09-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2004-09-27 | 2005-03-29 | 2005-09-28 | $7700.00 | Window Closed / Lapsed |