Neutral network with plural weight calculation methods and variation of plural learning parameters

Claim 2

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer including a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit or generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between outputs from said supervising signal generating unit and said second outputs from said output layer; a weight changing unit for changing weights on the basis of the weight changing direction acquired by said weight changing direction deciding unit, wherein said weight changing direction decision unit sets said weight changing direction to either a conjugate gradient direction or the steepest descent direction.

Claim 3

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer including a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit of generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plurality directions, a weight changing direction that minimizes errors between outputs from said supervising signal generating until nd said second outputs from said output layer; a weight changing unit for changing widths on the basis of the weight changing direction acquired by said weight changing direction deciding unit, wherein said weight changing direction decision unit sets said weight changing direction to the steepest direction for a first iteration of learning, and for a second iteration of learning et seq., sets the steepest descent direction for the learning o be carried to the number of iterations equal to the number of weights, and sets the conjugate gradient direction of the learning to be carried out to every other number of iterations.

Claim 4

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer including a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit of generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plurality directions, a weight changing direction that minimizes errors between outputs from said supervising signal generating until nd said second outputs from said output layer; a weight changing unit for changing widths on the basis of the weight changing direction acquired by said weight changing direction deciding unit, wherein said weight changing direction decision unit sets said weight changing direction to the steepest direction for a first iteration of learning, and for a second iteration of learning et seq., sets the steepest descent direction for the learning o be carried to the number of iterations equal to the number of weights, and sets the conjugate gradient direction of the learning to be carried out to every other number of iterations.

Claim 5

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer including a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between outputs from said supervising signal generating unit and said second outputs from said output layer; a weight changing unit for changing weights on the basis of the weight changing direction acquired by said weight changing direction deciding unit, wherein said weight changing direction decision unit sets said weight changing direction to the steepest descent direction for a first iteration of learning, and for a second iteration of learning et seq., sets the steepest descent direction for learning one half to one half the number of iterations equal to the number of weights, and the conjugate gradient direction of the learning to be carried out every other number of iteration.

Claim 6

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer including a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between outputs from said supervising signal generating unit and said second outputs from said output layer; a weight changing unit for changing weights on the basis of the weight changing direction acquired by said weight changing direction deciding unit, wherein said weight changing direction decision unit sets said weight changing direction to the steepest descent direction for a first iteration of learning, and for a second iteration of learning et seq., sets the steepest descent direction for learning one half to one half the number of iterations equal to the number of weights, and the conjugate gradient direction of the learning to be carried out every other number of iteration.

Claim 7

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising of plural multi-input single-outputs circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising plurality multi-input single-output circuits which subject as weighted sum of said output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between said outputs form said supervising signal generating unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plurally learning rates with respect to a weight changing direction; and a weight changing unit for selecting weight changing amounts acquired by said lien searching unit.

Claim 8

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising of plural multi-input single-outputs circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising plurality multi-input single-output circuits which subject as weighted sum of said output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between said outputs form said supervising signal generating unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plurally learning rates with respect to a weight changing direction; and a weight changing unit for selecting weight changing amounts acquired by said lien searching unit.

Claim 9

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 4, wherein said weight changing direction decision unit sets said weight changing direction o the steepest direction for a first iteration of learning, as well as for the second iteration of learning et seq., when the error does not become smaller than the error at a starting point said starting point being after line searching a in a conjugate gradient direction to make the learning rate smaller than a predetermined value, and after the conjugate gradient direction for the learning is set to be carried out for every other number of iteration.

Claim 10

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 4, wherein said weight changing direction decision unit sets said weight changing direction to the steepest direction for a first iteration of learning, as well as for the second iteration of learning et seq., when the error does not become smaller than the error at a starting point said starting point being after line searching, in a conjugate gradient direction to make the weight changing amount smaller than a predetermined value, and after the conjugate gradient direction for the learning is set to be carried out every their number of iteration.

Claim 11

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 4, wherein said weight changing unit changes weights according to a weight changing amount which minimizes an error.

Claim 12

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 4, wherein in producing the weight changing amounts for plurally learning rates with respect to the width changing direction, said line searching unit sets, for a second iteration of the learning et seal, an initial value of the learning rate to the larger between a fixed value and an optimum value produced by the previous iteration of learning.

Claim 13

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between said output from said supervising signal generating unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; a hidden layer product-sum storage unit for storing a product sum of input signals and weight of said hidden layer and a product sum of said input signals and said reaction of changing the weight of said hidden layer; and a weight changing unit for selecting weight changing amounts form th weight changing amounts acquired by said line a searching unit.

Claim 14

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between said output from said supervising signal generating unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; a hidden layer product-sum storage unit for storing a product sum of input signals and weight of said hidden layer and a product sum of said input signals and said reaction of changing the weight of said hidden layer; and a weight changing unit for selecting weight changing amounts form th weight changing amounts acquired by said line a searching unit.

Claim 15

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, w a weight changing direction that minimizes errors between outputs form said supervising signal generating unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to said weight changing direction; a weight changing unit for selecting weight changing amounts from weight changing amounts acquired by said line searching unit. a hidden layer product-sum storage unit for storing a product sum of input signals and weights of said hidden layer and a product sum of said input signal and said direction of changing th weight of said hidden layer; and a non-effective learning state detection unit or detecting that the learning has fallen into a non-effective state where further iteration does not effectively reduce an error.

Claim 16

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for selecting and deciding, from plural directions, w a weight changing direction that minimizes errors between outputs form said supervising signal generating unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to said weight changing direction; a weight changing unit for selecting weight changing amounts from weight changing amounts acquired by said line searching unit. a hidden layer product-sum storage unit for storing a product sum of input signals and weights of said hidden layer and a product sum of said input signal and said direction of changing th weight of said hidden layer; and a non-effective learning state detection unit or detecting that the learning has fallen into a non-effective state where further iteration does not effectively reduce an error.

Claim 17

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction to the steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer; a lien searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; and a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit.

Claim 18

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction to the steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer; a lien searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; and a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit.

Claim 19

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 11, wherein said weight changing unit changes weights according to a weight changing amount which minimizes an error.

Claim 20

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 11, wherein in producing weight changing amounts for plural learning rates with respect to the weight changing direction, said lien searching unit nests, for a second iteration of the learning et seq., an initial learning rate to the larger value of a fixed value and an optimum value produced by the previous iteration of learning.

Claim 21

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted su of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide record output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction tot eh steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect o a weight changing direction; a hidden layer product-sum storage unit for storing a product sum of input signal sand weights of said hidden layer and a product sum of said input signals and said direction of changing ht weight of said hidden layer; and a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit.

Claim 22

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted su of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide record output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction tot eh steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect o a weight changing direction; a hidden layer product-sum storage unit for storing a product sum of input signal sand weights of said hidden layer and a product sum of said input signals and said direction of changing ht weight of said hidden layer; and a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit.

Claim 23

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturating characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signal form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide record output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction to the steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer a line searching unit for producing weight changing amounts for plurality learning rates with respect to a weight changing direction; a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit. a hidden layer product-sum storage unit for storing a product sum of input signals and weights of said hidden layer and a product sum of said input signals and said direction of changing the weight of said hidden layer; and a non-effective learning state detecting unit for detecting that the learning is in a non-effective state where further iterations do not effectively reduce the error.

Claim 24

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturating characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signal form said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide record output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction to the steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer a line searching unit for producing weight changing amounts for plurality learning rates with respect to a weight changing direction; a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit. a hidden layer product-sum storage unit for storing a product sum of input signals and weights of said hidden layer and a product sum of said input signals and said direction of changing the weight of said hidden layer; and a non-effective learning state detecting unit for detecting that the learning is in a non-effective state where further iterations do not effectively reduce the error.

Claim 25

A learning machine with multi-in-ut single-output circuits connected in a hierarchical structure according to claim 15, wherein aid non-effective state detection unit detects a non-effective learning state by detecting that a decreasing error rate is smaller tan a predetermined value and the weight changing direction is the steepest descent direction, and initializes each weight by a random number in order to restart the learning.

Claim 26

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 15, wherein said non-effective state direction unit detects a non-effective learning state by detecting that a decreasing error rate is smaller tan a predetermined value and the weight changing direction is the steepest descent direction, and compresses equally all said weight sin order to restart the learning.

Claim 27

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing reaction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between outputs from said supervising signal generation unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; a weight changing unit for selecting weight changing amounts acquired by said line searching unit; and a non-effective learning state detection unit for detecting that the learning has fallen into a non-effective state where further iteration does not effectively reduce an error.

Claim 28

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide first output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide second output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing reaction decision unit for selecting and deciding, from plural directions, a weight changing direction that minimizes errors between outputs from said supervising signal generation unit and said second outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; a weight changing unit for selecting weight changing amounts acquired by said line searching unit; and a non-effective learning state detection unit for detecting that the learning has fallen into a non-effective state where further iteration does not effectively reduce an error.

Claim 29

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said non-effective state detection unit detects a non-effective learning state by detecting that a decreasing error rate is smaller than a predetermined value and the weight changing direction is the steepest descent direction.

Claim 30

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said non-effective state detection unit detects a non-effective learning state by detecting that a decreasing error rate is smaller than a predetermined value and the weight changing direction is the steepest descent direction, and initializes each weight by a random number in order to restart the learning.

Claim 31

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said non-effective state detection unit detects a non-effective learning state by detecting that a decreasing error rate is smaller than a predetermined value and the weight changing direction is th steepest descent direction, and compresses equally all said weights in order to restart the learning.

Claim 32

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said non-effective state defection unit detects a non-effective learning state by detecting that an absolute value of each weight is larger tan a predetermined value.

Claim 33

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said non-effective state detection unit detects a non-effective learning state by detecting that an absolute value of each weight is larger than a predetermined value and initializes each weight by a random number in order to restart the learning.

Claim 34

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said no-effective state detection unit detects a non-effective learning state by detecting that an absolute value of each weight is larger than a predetermined value and compresses equally all weights in order to restart the learning.

Claim 35

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 18, wherein said non-effective state detection unit detects a non-effective learning state by detecting that an absolute value of each weight is larger than a predetermine value and compresses only that weight having the largest absolute value in order to restart the learning.

Claim 36

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide fist output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide record output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction to the steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit; and a non-effective learning state detecting unit for detecting that the learning is in a non-effective state where further iterations do not effectively reduce the error.

Claim 37

A learning machine with multi-input single-output circuits connected in a hierarchical structure, comprising: a hidden layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of input signals to non-linear processing by a characteristic function having a saturation characteristic in order to provide fist output signals; an output layer comprising a plurality of multi-input single-output circuits which subject a weighted sum of said first output signals from said hidden layer to non-linear processing by a characteristic function having a saturation characteristic in order to provide record output signals; a supervising signal generating unit for generating supervising signals for said output layer; a weight changing direction decision unit for setting a weight changing direction to the steepest descent direction that minimizes errors between outputs from said supervising signal generating unit and outputs from said output layer; a line searching unit for producing weight changing amounts for plural learning rates with respect to a weight changing direction; a weight changing unit for selecting weight changing amounts from the weight changing amounts acquired by said line searching unit; and a non-effective learning state detecting unit for detecting that the learning is in a non-effective state where further iterations do not effectively reduce the error.

Claim 38

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 26, wherein said non-effective state detection unit detects a non-effective learning state by detecting that a decreasing error rate is smaller than a predetermined value and the weight changing direction is the steepest descent direction.

Claim 39

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 26, wherein said non-effective state detection unit detects a none-effective learning state by detecting that an absolute value of each weight is larger than a predetermined value.

Claim 40

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 26, wherein said non-effective state detection unit detects a non-effective learning state by detecting that an absolute value of each weight is larger tan a predetermined value and initializes each weight by a random number in order to restart the learning.

Claim 41

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 26, wherein said non-effective state detection unit detects a non-effective learning state by detecting that an absolute value of each weight is layer than a predetermined value and compresses equally all weights in order to restart the learning.

Claim 42

A learning machine with multi-input single-output circuits connected in a hierarchical structure according to claim 26, wherein said non-effective state detection unit detects a non-effective learning state by detecting that an absolute value of each weight said larger than a predetermined value and compresses only that weight having the largest absolute value in order to restart the learning.