US5245696A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Evolution and learning in neural networks: the number and distribution of learning trials affect the rate of evolution
Assignee: Ricoh Co Ltd
•
Filing: 1990-11-21
•
Granted: 1993-09-14
•
Expires: 2010-11-21
Abstract
Abstract The present invention relates to the interrelationships between nature (as mediated by evolution and genetic algorithms) and nurture (as mediated by gradient-descent supervised learning) in a population of neural networks for pattern recognition. The Baldwin effect is demonstrated that learning can change the rate of evolution of the population's genome - a "pseudo-Lamarkian" process, in which information learned is ultimately encoded in the genome by a purely Darwinian process. Selectivity is shown for this effect: too much learning or too little learning in each generation leads to slow evolution of the genome, whereas an intermediate amount leads to most rapid evolution. For a given number of learning trials throughout a population, the most rapid evolution occurs if different individuals each receive a different number of learning trials, rather than the same number. Because all biological networks possess structure due to evolution, it is important that such interactions between learning and evolution be understood. Hybrid systems can take advantage both of gradient descents (learning) and large jumps (genetic algorithms) in very complicated energy landscapes and hence may play an increasingly important role in the design of artificial neural systems.
Inventors
- 👤 David G. Stork
- 👤 Ronald C. Keesing
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-28 04:12
CPC Classifications
G; G06; G06N; G06N3/00; G06N3/02; G06N3/08; G06N3/086; G06F; G06F18/00; G06F18/20; G06F18/21; G06F18/211; G06F18/2111; G06V; G06V10/00; G06V10/40; G06V10/44; G06V10/443; G06V10/449; G06V10/451; G06V10/454
Claims Analysis
Total Claims: 12
Claim 1
Independent Claim
104 words
In a neural network system for use in application tasks including optical character recognition techniques, an evolutionary evaluation and learning method comprising the steps of: forming plural genetic representations of an application task including recognizing a plurality of characters where each character has a bit configuration specifying initial connection strengths between filter lays and category units, converting said plural genetic representations to neural networks, assigning to each network i, k(i) learning trails, chosen randomly between zero and 2K, and training each neural network to optimally perform the application task including optical character recognition where the number of learning trials per network is randomly chosen.
Claim 2
Dependent Claim
104 words
In a neural network system for use in application tasks including optical character recognition techniques, an evolutionary evaluation and learning method comprising the steps of: forming plural genetic representations of an application task including recognizing a plurality of characters where each character has a bit configuration specifying initial connection strengths between filter lays and category units, converting said plural genetic representations to neural networks, assigning to each network i, k(i) learning trails, chosen randomly between zero and 2K, and training each neural network to optimally perform the application task including optical character recognition where the number of learning trials per network is randomly chosen.
Claim 3
Dependent Claim
16 words
The system as in claim 1 wherein said training step includes gradient descent learning utilizing backpropagation.
Claim 4
Dependent Claim
26 words
The system as in claim 2 including the step of constructing a population of genes for said neural networks and converting said genes to neural networks.
Claim 5
Dependent Claim
27 words
The system as in claim 3 wherein the training of each network to perform said application task includes the step of testing each network on the task.
Claim 6
Dependent Claim
46 words
The system as in claim 4 including the step of determining whether the performance is within a predetermined percentage and if not discarding said network and if so utilizing a genetic algorithm process of crossover mutation and replication to determine new genes for the next generation.
Claim 7
Dependent Claim
152 words
The system as in claim 1 having initial parameters of N equals population size, K equals mean number of learning trials/net/generation, s equals portion of nets that survive each generation, c equals portion of surviving nets that crossover, M equals mutation rate, probability/bit/generation, and GenMax equals maximum number of generations, the method including the steps of: initializing the population size having a size N, a mean number of learning trials per net per generation having a value K, a portion of networks that survive each generation having a value S, a portion of the surviving networks that crossover having a value C, a mutation rate, probability/bit/generation having a value M, and a maximum number of generations equal to Genmax, randomly assigning bit values to each gene, constructing N networks from said genes, and presenting input/output data pairs k(i) times to said network i and performing gradient descent learning throughout said learning presentations.
Claim 8
Dependent Claim
26 words
The system as in claim 6 including the step of testing the fitness of each network, including the performance fitness on the problem to be solved.
Claim 9
Dependent Claim
46 words
The system as in claim 7 including the step of choosing the best S×N networks to survive in order to reproduce and randomly mutating genes at rate M by choosing C×S×N networks to crossover mutation so as to chose the cut point randomly in the gene.
Claim 10
Dependent Claim
39 words
The system as in claim 8, including the step of generating a new population of genes Gen where Gen=Gen+1 and if Gen=Genmax, displaying the fittest network in the generation Genmax and if not, constructing N networks from said genes.
Claim 11
Dependent Claim
85 words
In a neural network system for use in application tasks, an evolutionary evaluation and learning method comprising the steps of: forming a genetic representation of an application task having a bit configuration specifying initial connection strengths between filter layers and category units, converting said genetic representation to neural networks, assigning to each network i, k(i) learning trials, chosen randomly between zero and 2K, and training each neural network to optimally perform the application task where the number of learning trials per network is randomly chosen.
Claim 12
Dependent Claim
85 words
In a neural network system for use in application tasks, an evolutionary evaluation and learning method comprising the steps of: forming a genetic representation of an application task having a bit configuration specifying initial connection strengths between filter layers and category units, converting said genetic representation to neural networks, assigning to each network i, k(i) learning trials, chosen randomly between zero and 2K, and training each neural network to optimally perform the application task where the number of learning trials per network is randomly chosen.
References Cited (Backward Citations)
US4935877A
Cited
US5048095A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1996-09-13 | 1997-03-15 | 1997-09-14 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2000-09-13 | 2001-03-15 | 2001-09-14 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2004-09-13 | 2005-03-15 | 2005-09-14 | $7700.00 | Window Closed / Lapsed |