Neural-network content-addressable memory

Claim 1

A neural network content-addressable error-correcting memory system of the type used to provide an error-corrected output signal in response to an error-containing input signal, said system comprising: (a) a plurality of visible input/output processing units each having an input terminal and an output terminal, said visible processing units adapted to provide an output signal V i in response to input signal conditions: (b) a plurality of hidden processing units each having an input terminal and an output terminal, said hidden processing units adapted to provide an output signal V i in response to input signal conditions; (c) a plurality of linear circuit connection elements, each having an impedance value, said connection units connecting the outputs and inputs of selected visible and hidden processing units, wherein the impedance value of said connection elements is selected as a means for storing a presecribed set of error-free codewords; and (d) said plurality of connection elements and said visible and hidden processing units forming a network wherein; (i) substantially no connections are present between the output of a processing unit and its input; (ii) the impedance value of all connecting elements between a first processing unit and a second processing unit are substantially equal; (iii) the output and input of each visible processing unit is connected to the input and output respectively of each other visible processing unit and to the input and output respectively of each hidden processing unit; and (iv) there are substantially no connections between hidden processing units.

Claim 8

A method of operating a neural-network content-addressable error-correcting memory system of the type used to provide an error-corrected output signal in response to an error-containing input signal, wherein said network comprises: (a) a set of visible processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; (b) a set of hidden processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; and (c) a network interconnection matrix comprised of linear connection elements connecting the output and input terminals of selected processors to the input and output terminal of other selected processors, each linear connection element having an impedance value selected to facilitate storing of a prescribed set of error-free codewords, said interconnection matrix being constructed in accordance with the following network architecture, wherein the symbol W ij represents the conductance of the linear connection elements connecting the output of processor j to the input of processor i; (i) substantially all W ii =O; (ii) substantially all W ij =W ji ; (iii) substantially all visible processors are interconnected; (iv) substantially all visible processors are interconnected to each hidden processor; (v) substantially all hidden processor to hidden processor connections are absent; said method of operating comprising the steps of: (1) providing a learning algorithm to select the impedance value of the particular linear connection elements and to thereby store the bit patterns representative of the codewords, wherein the learning algorithm comprises the steps of: (a) selecting an initial value for each connection element W ij ; (b) clamping the output signal values V i of the visible processors to a particular bit pattern representing one of the codewords to be stored; (c) providing an updated output signal value V i for each of the unclamped processors according to the following equation: V.sub.i =σ(.sub.j ΣW.sub.ij V.sub.j /T) wherein the symbol σ represents the transfer function of the processor, the parameter T is an arbitrarily selected scalar, and V j is the output of all processors to which processor i has connections; (d) gradually reducing the value of parameter T and calculating an updated value for each unclamped V i for each value of T; (e) unclamping a portion of the visible processors and repeating steps (c) and (d) for each unclamped processor to arrive at a final output signal value V i ' for each unclamped processor; and (f) revising the values W ij in accordance with the following equation: ΔW.sub.ij =η(V.sub.i V.sub.j -V.sub.i 'V.sub.j ') wherein the value of parameter η is a constant, prescribed learning rate value; and (2) providing a retrieval algorithm to facilitate retrieval of a prescribed stored codeword in response to an error-containing input condition.

Claim 9

A method of operating a neural-network content-addressable error-correcting memory system of the type used to provide an error-corrected output signal in response to an error-containing input signal, wherein said network comprises: (a) a set of visible processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; (b) a set of hidden processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; and (c) a network interconnection matrix comprised of linear connection elements connecting the output and input terminals of selected processors to the input and output terminal of other selected processors, each linear connection element having an impedance value selected to facilitate storing of a prescribed set of error-free codewords, said interconnection matrix being constructed in accordance with the following network architecture, wherein the symbol W ij represents the conductance of the linear connection elements connecting the output of processor j to the input of processor i; (i) substantially all W ii =O; (ii) substantially all W ij =W ji ; (iii) substantially all visible processors are interconnected; (iv) substantially all visible processors are interconnected to each hidden processor; (v) substantially all hidden processor to hidden processor connections are absent; said method of operating comprising the steps of: (1) providing a learning algorithm to select the impedance value of the particular linear connection elements and to thereby store the bit patterns representative of the codewords, wherein the learning algorithm comprises the steps of: (a) selecting an initial value for each connection element W ij ; (b) clamping the output signal values V i of the visible processors to a particular bit pattern representing one of the codewords to be stored; (c) providing an updated output signal value V i for each of the unclamped processors according to the following equation: V.sub.i =σ(.sub.j ΣW.sub.ij V.sub.j /T) wherein the symbol σ represents the transfer function of the processor, the parameter T is an arbitrarily selected scalar, and V j is the output of all processors to which processor i has connections; (d) gradually reducing the value of parameter T and calculating an updated value for each unclamped V i for each value of T; (e) unclamping a portion of the visible processors and repeating steps (c) and (d) for each unclamped processor to arrive at a final output signal value V i ' for each unclamped processor; and (f) revising the values W ij in accordance with the following equation: ΔW.sub.ij =η(V.sub.i V.sub.j -V.sub.i 'V.sub.j ') wherein the value of parameter η is a constant, prescribed learning rate value; and (2) providing a retrieval algorithm to facilitate retrieval of a prescribed stored codeword in response to an error-containing input condition.

Claim 14

A method of operating a neural-network content-addressable error-correcting memory system of the type used to provide an error-corrected output signal in response to an error-containing input signal, wherein said network comprises: (a) a set of visible processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; (b) a set of hidden processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; and (c) a network interconnection matrix comprised of linear connection elements connecting the output and input terminals of selected processors to the input and output terminal of other selected processors, each linear connection element having an impedance value selected to facilitate storing of a prescribed set of error-free codewords, said interconnection matrix being constructed in accordance with the following network architecture, wherein the symbol W ij represent the conductance of the linear connection elements connecting the output of processor j to the input of processor i; (i) substantially all W ii =O; (ii) substantially all W ij =W ji ; (iii) substantially all visible processors are interconnected; (iv) substantially all visible processors are interconnected to each hidden processor; (v) substantially all hidden processor to hidden processor connections are absent; said method of operating comprising the steps of: (1) providing a learning algorithm to select the impedance value of the particular linear connection elements and to thereby store the bit pattern representative of the codewords; and (2) providing a retrieval algorithm to facilitate retrieval of a prescribed stored codeword in response to an error-containing input condition, wherein the retrieval algorithm comprises the steps of: (a) clamping the output signal values of the visible processors to the value of an error-containing input bit pattern; (b) providing an updated output signal value V i for each of the unclamped processors according to the following equation: V.sub.i =σ(.sub.j ΣW.sub.ij V.sub.j /T) wherein σ represents the transfer function of the processors and the parameter T is an arbitrarily selected scalar; (c) unclamping the output signal values of the visible processors; (d) repeating step (b) for all unclamped processors while gradually reducing the value of scalar T; and (e) providing the final value of outputs V i of the visible units as an error-corrected version of the input bit pattern.

Claim 15

A method of operating a neural-network content-addressable error-correcting memory system of the type used to provide an error-corrected output signal in response to an error-containing input signal, wherein said network comprises: (a) a set of visible processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; (b) a set of hidden processors adapted to provide an output signal V i in response to input signal conditions, each processor having an input terminal and an output terminal; and (c) a network interconnection matrix comprised of linear connection elements connecting the output and input terminals of selected processors to the input and output terminal of other selected processors, each linear connection element having an impedance value selected to facilitate storing of a prescribed set of error-free codewords, said interconnection matrix being constructed in accordance with the following network architecture, wherein the symbol W ij represent the conductance of the linear connection elements connecting the output of processor j to the input of processor i; (i) substantially all W ii =O; (ii) substantially all W ij =W ji ; (iii) substantially all visible processors are interconnected; (iv) substantially all visible processors are interconnected to each hidden processor; (v) substantially all hidden processor to hidden processor connections are absent; said method of operating comprising the steps of: (1) providing a learning algorithm to select the impedance value of the particular linear connection elements and to thereby store the bit pattern representative of the codewords; and (2) providing a retrieval algorithm to facilitate retrieval of a prescribed stored codeword in response to an error-containing input condition, wherein the retrieval algorithm comprises the steps of: (a) clamping the output signal values of the visible processors to the value of an error-containing input bit pattern; (b) providing an updated output signal value V i for each of the unclamped processors according to the following equation: V.sub.i =σ(.sub.j ΣW.sub.ij V.sub.j /T) wherein σ represents the transfer function of the processors and the parameter T is an arbitrarily selected scalar; (c) unclamping the output signal values of the visible processors; (d) repeating step (b) for all unclamped processors while gradually reducing the value of scalar T; and (e) providing the final value of outputs V i of the visible units as an error-corrected version of the input bit pattern.