Neural network with selective error reduction to increase learning speed

Claim 2

An iterative learning machine comprising: an output signal computing means comprising: a plurality of multi-input/single-output signal processing means connected to form a network of a hierarchical structure such that said multi-input/single-output signal processing means in each hierarchy have no mutual coupling therebetween and signals propagate only in the direction of a higher hierarchy; a weight coefficient updating means for updating weight coefficients for said output signal computing means on the basis of an output signal therefrom; each said multi-input/single-output signal processing means comprising: memory means for storing a plurality of weight coefficients; a plurality of input means for receiving a plurality of data; multiplying means for weighting input data from said input means according to the weight coefficients stored in said memory means; adder means for adding together data weighted by said multiplying means; threshold value processing means for limiting an output from said adder means to a value within a predetermined range; said coefficient updating means comprising: supervisory signal generator means for generating a signal reference value for an output signal from said output signal computing means; error signal computing means for computing the difference between the output signal of said output signal computing means and said reference signal; weight change quantity computing means for computing change quantities for weight coefficients stored in said memory, said computing being performed in accordance with an error output of said error signal computing means; weight coefficient change quantity control means for controlling change quantities for said weight coefficients in accordance with an error output of said error signal computing means; error signal determination means for ascertaining whether the output level of said error signal computing means exceeds a threshold value that is 2/3 the absolute value of a maximum error; and weight change quantity control means for multiplying weight coefficient change quantities in the highest hierarchy in said output signal computing means by a factor m, when said error signal determination means determines that an output value of said error signal computing means exceeds a threshold value.

Claim 3

An iterative learning machine comprising: an output signal computing means comprising: a plurality of multi-input/single-output signal processing means connected to form a network of a hierarchical structure such that said multi-input/single-output signal processing means in each hierarchy have no mutual coupling therebetween and signals propagate only in the direction of a higher hierarchy; a weight coefficient updating means for updating weight coefficients for said output signal computing means on the basis of an output signal therefrom; each said multi-input/single-output signal processing means comprising: memory means for storing a plurality of weight coefficients; a plurality of input means for receiving a plurality of data; multiplying means for weighting input data from said input means according to the weight coefficients stored in said memory means; adder means for adding together data weighted by said multiplying means; threshold value processing means for limiting an output from said adder means to a value within a predetermined range; said coefficient updating means comprising: supervisory signal generator means for generating a signal reference value for an output signal from said output signal computing means; error signal computing means for computing the difference between the output signal of said output signal computing means and said reference signal; weight change quantity computing means for computing change quantities for weight coefficients stored in said memory, said computing being performed in accordance with an error output of said error signal computing means; weight coefficient change quantity control means for controlling change quantities for said weight coefficients in accordance with an error output of said error signal computing means, said weight coefficient change quantity control means comprising: error signal determination means for ascertaining whether the output of said error signal computing unit does not exceed a threshold value; learning advancement determination means for determining the degree of advancement of learning on the basis of an output of said error signal computing unit; threshold value control means for iteratively reducing said threshold value in accordance with the result of determination by said learning advancement determination means; and second weight change quantity control means for changing weight coefficient change quantities to zero whenever said error signal determination means determines that the error output of said error signal computing means does not exceed said threshold value.

Claim 4

An iterative learning machine comprising: an output signal computing means comprising: a plurality of multi-input/single-output signal processing means connected to form a network of a hierarchical structure such that said multi-input/single-output signal processing means in each hierarchy have no mutual coupling therebetween and signals propagate only in the direction of a higher hierarchy; a weight coefficient updating means for updating weight coefficients for said output signal computing means on the basis of an output signal therefrom; each said multi-input/single-output signal processing means comprising: memory means for storing a plurality of weight coefficients; a plurality of input means for receiving a plurality of data; multiplying means for weighting input data from said input means according to the weight coefficients stored in said memory means; adder means for adding together data weighted by said multiplying means; threshold value processing means for limiting an output from said adder means to a value within a predetermined range; said coefficient updating means comprising: supervisory signal generator means for generating a signal reference value for an output signal from said output signal computing means; error signal computing means for computing the difference between the output signal of said output signal computing means and said reference signal; weight change quantity computing means for computing change quantities for weight coefficients stored in said memory, said computing being performed in accordance with an error output of said error signal computing means; weight coefficient change quantity control means for controlling change quantities for said weight coefficients in accordance with an error output of said error signal computing means, said weight coefficient change quantity control means comprising: error signal determination means for ascertaining whether the output of said error signal computing unit does not exceed a threshold value; learning advancement determination means for determining the degree of advancement of learning on the basis of an output of said error signal computing unit; threshold value control means for iteratively reducing said threshold value in accordance with the result of determination by said learning advancement determination means; and second weight change quantity control means for changing weight coefficient change quantities to zero whenever said error signal determination means determines that the error output of said error signal computing means does not exceed said threshold value.

Claim 5

A learning machine according to claim 2, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with the total sum of errors of said multi-input/single-output signal processing means comprising the highest hierarchy in said output signal computing means.

Claim 6

A learning machine according to claim 2, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with the number of iterations of learning.

Claim 7

A learning machine according to claim 2, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with the number of multi-input/single-output signal processing means in the highest hierarchy in said output signal computing means whose output errors exceed said threshold value.

Claim 8

A learning machine according to claim 2, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with a maximum output value produced by said error signal computing means in a single iteration of learning.

Claim 9

An iterative learning machine comprising: an output signal computing means comprising: a plurality of multi-input/single-output signal processing means connected to form a network of a hierarchical structure such that said multi-input/single-output a signal processing means in each hierarchy have no mutual coupling therebetween and signals propagate only in the direction of a higher hierarchy; a weight coefficient updating means for updating weight coefficients for said output signal computing means on the basis of an output signal therefrom; each said multi-input/single-output signal processing means comprising: memory means for storing a plurality of weight coefficients; a plurality of input means for receiving a plurality of data; multiplying means for weighting input data from said input means according to the weight coefficients stored in said memory means; adder means for adding together data weighted by said multiplying means; threshold value processing means for limiting an output from said adder means to a value within a predetermined range; said coefficient updating means comprising: supervisory signal generator means for generating a signal reference value for an output signal from said output signal computing means; error signal computing means for computing the difference between the output signal of said output signal computing means and said reference signal; weight change quantity computing means for computing change quantities for weight coefficients stored in said memory, said computing being performed in accordance with an error output of said error signal computing means; weight coefficient change quantity control means for controlling change quantities for said weight coefficients in accordance with an error output of said error signal computing means, said weight coefficient change quantity control means comprising: error signal determination means for ascertaining whether the error output of said error signal computing means does not exceed a threshold value; jump determination means for outputting a jump signal when, as a result of counting output signals from said error signal determination means, said jump determination means determines that the error output of said error signal computing means does not exceed said threshold value; learning advancement determination means for determining the degree of advancement of learning based on the output of said error signal computing means; threshold value control means for iteratively reducing said threshold value in accordance with the result of determination by said learning advancement determination means; and weight change quantity control means for skipping a weight changing operation of said weight change computing means in accordance with a jump signal outputted from said jump determination means.

Claim 10

An iterative learning machine comprising: an output signal computing means comprising: a plurality of multi-input/single-output signal processing means connected to form a network of a hierarchical structure such that said multi-input/single-output a signal processing means in each hierarchy have no mutual coupling therebetween and signals propagate only in the direction of a higher hierarchy; a weight coefficient updating means for updating weight coefficients for said output signal computing means on the basis of an output signal therefrom; each said multi-input/single-output signal processing means comprising: memory means for storing a plurality of weight coefficients; a plurality of input means for receiving a plurality of data; multiplying means for weighting input data from said input means according to the weight coefficients stored in said memory means; adder means for adding together data weighted by said multiplying means; threshold value processing means for limiting an output from said adder means to a value within a predetermined range; said coefficient updating means comprising: supervisory signal generator means for generating a signal reference value for an output signal from said output signal computing means; error signal computing means for computing the difference between the output signal of said output signal computing means and said reference signal; weight change quantity computing means for computing change quantities for weight coefficients stored in said memory, said computing being performed in accordance with an error output of said error signal computing means; weight coefficient change quantity control means for controlling change quantities for said weight coefficients in accordance with an error output of said error signal computing means, said weight coefficient change quantity control means comprising: error signal determination means for ascertaining whether the error output of said error signal computing means does not exceed a threshold value; jump determination means for outputting a jump signal when, as a result of counting output signals from said error signal determination means, said jump determination means determines that the error output of said error signal computing means does not exceed said threshold value; learning advancement determination means for determining the degree of advancement of learning based on the output of said error signal computing means; threshold value control means for iteratively reducing said threshold value in accordance with the result of determination by said learning advancement determination means; and weight change quantity control means for skipping a weight changing operation of said weight change computing means in accordance with a jump signal outputted from said jump determination means.

Claim 11

A learning machine according to claim 7, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with the total sum of errors of all said multi-input/single-output signal processing means comprising the highest hierarchy in said output signal computing means.

Claim 12

A learning machine according to claim 7, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with the number of iterations of learning.

Claim 13

A learning machine according to claim 7, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with the number of multi-input/single-output signal processing means in the highest hierarchy in said output signal computing means whose output errors exceed said threshold value.

Claim 14

A learning machine according to claim 7, wherein said learning advancement determination means determines the degree of advancement of learning in accordance with a maximum output value produced by said error signal computing means in a single iteration of learning.