Computer utilizing neural network and method of using same

Claim 2

A computing device comprising: a microprocessor for executing general purpose programs, wherein the microprocessor has at least one output port for transmitting data and at least one input port for receiving data; and a neural network co-processor, operatively coupled to the at least one input port and the at least one output port, for computing at least one mathematical function that includes a polynomial expansion having a form ##EQU4## wherein y represents an output of said neural network co-processor w i-1 represents a weight value, x 1 , x 2 ,. . . , x n represent inputs to said neural network co-processor, g 1i , . . . , g ni represent gating functions and m and n are integers.

Claim 3

The computing device recited in claim 1 wherein said computing device is contained on an integrated circuit.

Claim 4

The computing device recited in claim 1 wherein said computing device is contained on a printed circuit board.

Claim 5

The computing device recited in claim 1 wherein said microprocessor iteratively executes a program, and wherein said co-processor comprises at least one input and at least one output, said program transferring data to said at least one input of said co-processor and said program transferring data from said at least one output of said co-processor.

Claim 6

A computing device comprising: a microprocessor for executing general purpose programs, wherein said microprocessor has at least one output port for transmitting data and at least one input port for receiving data; a neural network co-processor for performing mathematical computations, wherein said co-processor is responsive to said at least one microprocessor output port and generates at least one output, and wherein said at least one microprocessor input port is responsive to said at least one co-processor output; and DMA control means coupled to said neural network co-processor, for transferring successive ones of a plurality of inputs to said neural network co-processor; said neural network co-processor comprising: an input port for receiving successive ones of said plurality of inputs and generating corresponding outputs; a plurality of neurons, each neuron having at least one input and generating an output; means for distributing said input port outputs to said at least one input of each of said plurality of neurons; and a plurality of summers each summing the outputs of predetermined ones of said neurons and generating said at least one co-processor output; each of said neurons comprising: first memory means, responsive to the output of said input port, for storing at least one weight value, wherein each of said at least one weight values is accessible by said microprocessor, and wherein each of said at least one weight values is accessible by said DMA control means; and second memory means, responsive to the output of said input port, for storing a plurality of gating functions, wherein each of said gating functions is accessible by said microprocessor, and wherein each of said gating functions is accessible by said DMA control means.

Claim 7

A computing device comprising: a microprocessor for executing general purpose programs, wherein said microprocessor has at least one output port for transmitting data and at least one input port for receiving data; a neural network co-processor for performing mathematical computations, wherein said co-processor is responsive to said at least one microprocessor output port and generates at least one output, and wherein said at least one microprocessor input port is responsive to said at least one co-processor output; and DMA control means coupled to said neural network co-processor, for transferring successive ones of a plurality of inputs to said neural network co-processor; said neural network co-processor comprising: an input port for receiving successive ones of said plurality of inputs and generating corresponding outputs; a plurality of neurons, each neuron having at least one input and generating an output; means for distributing said input port outputs to said at least one input of each of said plurality of neurons; and a plurality of summers each summing the outputs of predetermined ones of said neurons and generating said at least one co-processor output; each of said neurons comprising: first memory means, responsive to the output of said input port, for storing at least one weight value, wherein each of said at least one weight values is accessible by said microprocessor, and wherein each of said at least one weight values is accessible by said DMA control means; and second memory means, responsive to the output of said input port, for storing a plurality of gating functions, wherein each of said gating functions is accessible by said microprocessor, and wherein each of said gating functions is accessible by said DMA control means.

Claim 8

The computing device recited in claim 5 wherein said neurons comprise a linear transfer function.

Claim 9

The computing device recited in claim 5 wherein the outputs of at least some of said neurons may be summed by more than one of said summers.

Claim 10

The computing device recited in claim 5 wherein said co-processor comprises at least one input, and wherein said microprocessor iteratively executes a program, said program transferring data to said at least one input of said co-processor, and said program transferring data from said at least one output of said co-processor.

Claim 11

The computing device recited in claim 5 wherein said microprocessor executes a program, said program transferring said at least one weight value to said first memory means.

Claim 12

The computing device recited in claim 5 wherein said microprocessor executes a program, said program transferring said plurality of gating functions to said second memory means.

Claim 13

The computing device recited in claim 5 wherein said computing device is contained on an integrated circuit.

Claim 14

The computing device recited in claim 5 wherein said computing device is contained on a printed circuit board.

Claim 15

The computing device recited in claim 5 wherein said neural network co-processor computes a polynomial expansion.

Claim 16

The computing device recited in claim 13 wherein said polynomial expansion has the form: ##EQU5## wherein y represents the output of said neural network co-processor; wherein w i-1 represents the weight value for the ith neuron; wherein x 1 , x 2 , . . . , x n represent inputs to said neural network co-processor; wherein g 1i , . . . , g ni represent gating functions for the ith neuron which are applied to said inputs; and wherein m and n are integers.

Claim 17

The computing device recited in claim 14 wherein each x i is represented by the function x i =f i (z j ), wherein z j is any arbitrary variable, and wherein the indices i and j are any positive integers.

Claim 18

The computing device recited in claim 14 wherein said neural network co-processor computes a truncated version of said polynomial expansion.

Claim 19

The computing device recited in claim 13 wherein said polynomial expansion has the form: ##EQU6## wherein y represents the output of said neural network co-processor; wherein w i-1 represents the weight value for the ith neuron; wherein x i , x j , x k represent inputs to said neural network co-processor; wherein n is a positive integer; and wherein f 1 (i) through f6.sub.(i,j,k) are index-generating functions.

Claim 20

The computing device recited in claim 13 wherein said polynomial expansion has the form: ##EQU7## wherein y represents the output of the neural network co-processor; wherein w 0 , w 1 , . . . , w N-1 represent weight values, N being a positive integer representing the Nth neuron, and the term w i in general representing the weight value associated with the i+1 neuron; wherein x 1 , x 2 , . . . , x n represent inputs to said neural network co-processor, n being a positive integer; and wherein g 1N , . . . , g nN represent gating functions for the Nth neuron which are applied to said inputs.

Claim 21

The computing device recited in claim 5 wherein said neural network co-processor performs computations based upon an orthogonal function.

Claim 22

The computing device recited in claim 19 wherein said orthogonal function is a sine, cosine, exponential, logarithmic, Fourier transformation, Legendre polynomial, or a non-linear function such as a Volterra function or a radial basis function.

Claim 23

The computing device recited in claim 5 wherein said neural network co-processor computes a combination of polynomial expansion and orthogonal functions.

Claim 24

In a computing device including a microprocessor for executing general purpose programs, said microprocessor having at least one output port for transmitting data and at least one input port for receiving data, and at least one neural network co-processor for performing mathematical computations, said co-processor being responsive to said at least one output port and generating at least one co-processor output, said at least one microprocessor input port being responsive to said at least one co-processor output, said co-processor comprising first memory means for storing gating functions and second memory means for storing weight values, a method of operating said computing device, said method comprising the following steps: (a) said microprocessor transmitting gating functions to said first memory means; (b) said microprocessor transmitting weight values to said second memory means; (c) said microprocessor transmitting at least one co-processor input to said at least one neural network co-processor; and (d) said co-processor computing a mathematical function, utilizing said gating functions, said weight values, and said at least one co-processor input, said co-processor generating said at least one co-processor output as a result of computing said mathematical function.

Claim 25

In a computing device including a microprocessor for executing general purpose programs, said microprocessor having at least one output port for transmitting data and at least one input port for receiving data, and at least one neural network co-processor for performing mathematical computations, said co-processor being responsive to said at least one output port and generating at least one co-processor output, said at least one microprocessor input port being responsive to said at least one co-processor output, said co-processor comprising first memory means for storing gating functions and second memory means for storing weight values, a method of operating said computing device, said method comprising the following steps: (a) said microprocessor transmitting gating functions to said first memory means; (b) said microprocessor transmitting weight values to said second memory means; (c) said microprocessor transmitting at least one co-processor input to said at least one neural network co-processor; and (d) said co-processor computing a mathematical function, utilizing said gating functions, said weight values, and said at least one co-processor input, said co-processor generating said at least one co-processor output as a result of computing said mathematical function.

Claim 26

The method recited in claim 22 and further comprising the following steps: (e) said microprocessor determining whether said at least one co-processor output is ready to be read; (i) if so, proceeding to step (f); (ii) if not, returning to step (c); (f) said microprocessor reading said at least one co-processor output; and (g) returning to step (c).

Claim 27

The method recited in claim 22, wherein said computing device further comprises a DMA control means, and wherein steps (a) through (c) and steps (f) and (g) are performed by said DMA control means rather than said microprocessor.

Claim 28

The method recited in claim 22, wherein said first memory means includes a plurality of gating function memories, each of said plurality of gating function memories being capable of storing a plurality of said gating functions.

Claim 29

The method recited in claim 25, wherein in step (a) said gating functions are transmitted by substeps (a1) through (a7) as follows: (a1) said microprocessor determining whether said gating functions are to be transmitted to said at least one co-processor using either global mode, block mode, or sequential mode; (i) if global mode, proceeding to step (a2); (ii) if block mode, skipping steps (a2) and (a3), and proceeding to step (a4); (iii) if sequential mode, skipping steps (a2) through (a5), and proceeding to step (a6); (a2) said microprocessor broadcasting a single gating function to said at least one co-processor; (a3) said at least one co-processor storing said single gating function in each of said plurality of gating function memories and skipping steps (a4) through (a6); (a4) said microprocessor broadcasting said gating functions to said at least one neural network co-processor; (a5) said at least one neural network co-processor receiving said gating functions, and transmitting said gating functions to predetermined ones of said plurality of gating function memories and skipping step (a6); (a6) said microprocessor transmitting said gating functions to a predetermined one of said plurality of gating function memories; (a7) determining whether another of said gating functions is to be transmitted; (i) if so, returning to step (a1).

Claim 30

The method recited in claim 26, wherein said computing device further comprises a DMA control means, and wherein steps (a1) and (a2) and steps (a4) and (a6) are performed by said DMA control means rather than said microprocessor.

Claim 31

The method recited in claim 22, wherein said second memory means includes a plurality of weight value memories, each of said plurality of weight value memories being capable of storing at least one weight value.

Claim 32

The method recited in claim 28, wherein in step (b) said weight values are transmitted by substeps (b1) through (b7) as follows: (b1) said microprocessor determining whether said weight values are to be transmitted to said at least one co-processor using either global mode, block mode, or sequential mode; (i) if global mode, proceeding to step (b2); (ii) if block mode, skipping steps (b2) and (b3), and proceeding to step (b4); (iii) if sequential mode, skipping steps (b2) through (b5) and proceeding to step (b6); (b2) said microprocessor broadcasting a single weight value to said at least one co-processor; (b3) said at least one co-processor storing said single weight value in each of said plurality of weight value memories and skipping steps (b4) through (b6); (b4) said microprocessor broadcasting said weight values to said at least one neural network co-processors; (b5) said at least one neural network co-processor receiving said weight values and transmitting said weight values to predetermined ones of said plurality of weight value memories and skipping step (b6); (b6) said microprocessor transmitting said weight values to a predetermined one of said plurality of weight value memodes; (b7) determining whether another of said weight values is to be transmitted; (i) if so, returning to step (b1).

Claim 33

The method recited in claim 29, wherein said computing device further comprises a DMA control means, wherein steps (b1) and (b2) and steps (b4) and (b6) are performed by said DMA control means rather than said microprocessor.

Claim 34

The method of operating a computing device recited in claim 22 wherein the operation of said computing device is based on a polynomial expansion.

Claim 35

The method of operating a computing device recited in claim 31 wherein said polynomial expansion has the form: ##EQU8## wherein y represents the output of said neural network co-processor; wherein w i-1 represents the weight value for the ith neuron; wherein x 1 , x 2 , . . . , x n represent inputs to said neural network co-processor; wherein g 1i , . . . , g ni represent gating functions for the ith neuron which are applied to said inputs; and wherein m and n are integers.

Claim 36

The method of operating a computing device recited in claim 32 wherein each x i is represented by the function x i =f i (z j ), wherein z j is any arbitrary variable, and wherein the indices i and j are any positive integers.

Claim 37

The method of operating a computing device recited in claim 31 wherein the operation of said computing device is based on a truncated version of said polynomial expansion.

Claim 38

The method of operating a computing device recited in claim 31 wherein said polynomial expansion has the form: ##EQU9## wherein y represents the output of said neural network co-processor; wherein w i-1 represents the weight value for the ith neuron; wherein x i , x j , x k represent inputs to said neural network co-processor; wherein n is a positive integer; and wherein f 1 (i) through f6.sub.(i,j,k) are index-generating functions.

Claim 39

The method of operating a computing device recited in claim 31 wherein said polynomial expansion has the form: ##EQU10## wherein y represents the output of the neural network co-processor; wherein w 0 , w 1 , . . . , w N-1 represent weight values, N being a positive integer representing the Nth neuron, and the term w i in general representing the weight value associated with the i+1 neuron; wherein x 1 , x 2 , . . . , x n represent inputs to said neural network co-processor, n being a positive integer; and wherein g 1N , . . . , g nN represent gating functions for the Nth neuron which are applied to said inputs.

Claim 40

The method of operating a computing device recited in claim 22 wherein the operation of said computing device is based upon an orthogonal function.

Claim 41

The method of operating a computing device recited in claim 37 wherein said orthogonal function is a sine, cosine, exponential, logarithmic, Fourier transformation, Legendre polynomial, or a non-linear function such as a Volterra function or a radial basis function.

Claim 42

The method of operating a computing device recited in claim 22 wherein the operation of said computing device is based on a combination of polynomial expansion and orthogonal functions.

Claim 43

The method of operating a computing device recited in claim 22 wherein said computing device is contained on at least one integrated circuit.

Claim 44

The method of operating a computing device recited in claim 22 wherein said computing device is contained on at least one printed circuit board.