Intelligent controller with neural network and reinforcement learning

Claim 2

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and further wherein the plant control signal is generated in further accordance with unsupervised learning; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant.

Claim 3

A plant controller as recited in claim 1, wherein the action network generates a continuous plant control signal.

Claim 4

A plant controller as recited in claim 1, wherein the action network comprises an unsupervised learning based neural network.

Claim 5

A plant controller as recited in claim 3, wherein the unsupervised learning based neural network comprises first, second and third neuron layers with first and second pluralities of inter-neuron weights data stored in memory and associated therewith, and wherein the first and second pluralities of inter-neuron weights data are selectively modified in accordance with the reinforcement signal.

Claim 6

A plant controller as recited in claim 1, wherein the critic network comprises a neural network.

Claim 7

A plant controller as recited in claim 5, wherein the neural network comprises first, second and third neuron layers with first, second and third pluralities of inter-neuron weights data stored in memory and associated therewith, and wherein the first, second and third pluralities of inter-neuron weights data are selectively modified in accordance with the plant performance signal.

Claim 8

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant, and further wherein the reinforcement signal is outputted in further accordance with a distance parameter which represents a difference between the actual and desired states of the quantitative plant performance parameter.

Claim 9

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant, and further wherein the reinforcement signal is outputted in further accordance with a distance parameter which represents a difference between the actual and desired states of the quantitative plant performance parameter.

Claim 10

A plant controller as recited in claim 7, wherein the action network generates a continuous plant control signal.

Claim 11

A plant controller as recited in claim 7, wherein the action network comprises a neural network.

Claim 12

A plant controller as recited in claim 9, wherein the neural network comprises first, second and third neuron layers with first and second pluralities of inter-neuron weights data stored in memory and associated therewith, and wherein the first and second pluralities of inter-neuron weights data are selectively modified in accordance with the reinforcement signal.

Claim 13

A plant controller as recited in claim 9, wherein the critic network comprises a neural network.

Claim 14

A plant controller as recited in claim 11, wherein the neural network comprises first, second and third neuron layers with first, second and third pluralities of inter-neuron weights data stored in memory and associated therewith, and wherein the first, second and third pluralities of inter-neuron weights data are selectively modified in accordance with the plant performance signal.

Claim 15

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant; generating a reinforcement signal in accordance with the plant state signal and the plant performance signal, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal and in further accordance with unsupervised learning, wherein the plant control signal represents a desired state of the quantitative plant performance parameter.

Claim 16

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant; generating a reinforcement signal in accordance with the plant state signal and the plant performance signal, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal and in further accordance with unsupervised learning, wherein the plant control signal represents a desired state of the quantitative plant performance parameter.

Claim 17

A plant control method as recited in claim 13, wherein the step of generating the plant control signal in accordance with the plant state signal and the reinforcement signal comprises generating a continuous plant control signal.

Claim 18

A plant control method as recited in claim 13, wherein the step of generating the plant control signal in accordance with the plant state signal and the reinforcement signal comprises generating the plant control signal with an unsupervised learning based neural network which includes first and second pluralities of inter-neuron weights data associated therewith.

Claim 19

A plant control method as recited in claim 15, wherein the step of generating the plant control signal with an unsupervised learning based neural network comprises selectively modifying the first and second pluralities of inter-neuron weights data in accordance with the reinforcement signal.

Claim 20

A plant control method as recited in claim 13, wherein the step of generating the reinforcement signal in accordance with the plant state signal and the plant performance signal comprises generating the reinforcement signal with a neural network.

Claim 21

A plant control method as recited in claim 17, wherein the step of generating the reinforcement signal with a neural network comprises selectively modifying first, second and third pluralities of inter-neuron weights data in accordance with the plant performance signal.

Claim 22

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant, generating a reinforcement signal in accordance with the plant state signal and the plant performance signal and in further accordance with a distance parameter which represents a difference between the actual state and a desired state of the quantitative plant performance parameter, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal, wherein the plant control signal represents the desired state of the quantitative plant performance parameter.

Claim 23

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant, generating a reinforcement signal in accordance with the plant state signal and the plant performance signal and in further accordance with a distance parameter which represents a difference between the actual state and a desired state of the quantitative plant performance parameter, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal, wherein the plant control signal represents the desired state of the quantitative plant performance parameter.

Claim 24

A plant control method as recited in claim 19, wherein the step of generating the plant control signal in accordance with the plant state signal and the reinforcement signal comprises generating a continuous plant control signal.

Claim 25

A plant control method as recited in claim 19, wherein the step of generating the plant control signal in accordance with the plant state signal and the reinforcement signal comprises generating the plant control signal with a neural network which includes first and second pluralities of inter-neuron weights data associated therewith.

Claim 26

A plant control method as recited in claim 21, wherein the step of generating the plant control signal with a neural network comprises selectively modifying the first and second pluralities of inter-neuron weights data in accordance with the reinforcement signal.

Claim 27

A plant control method as recited in claim 19, wherein the step of generating the reinforcement signal in accordance with the plant state signal and the plant performance signal comprises generating the reinforcement signal with a neural network.

Claim 28

A plant control method as recited in claim 23, wherein the step of generating the reinforcement signal with a neural network comprises selectively modifying first, second and third pluralities of inter-neuron weights data in accordance with the plant performance signal.

Claim 29

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the plant control signal is generated in further accordance with unsupervised learning, and further wherein the action network comprises an unsupervised learning based neural network, and still further wherein the unsupervised learning based neural network is configured to perform Kohonen feature mapping; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant.

Claim 30

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the plant control signal is generated in further accordance with unsupervised learning, and further wherein the action network comprises an unsupervised learning based neural network, and still further wherein the unsupervised learning based neural network is configured to perform Kohonen feature mapping; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant.

Claim 31

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the plant control signal is generated in further accordance with unsupervised learning, and further wherein the action network generates the plant control signal in still further accordance with a plurality of fuzzy logic rules; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant.

Claim 32

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the plant control signal is generated in further accordance with unsupervised learning, and further wherein the action network generates the plant control signal in still further accordance with a plurality of fuzzy logic rules; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant.

Claim 33

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the action network comprises a neural network, and further wherein the neural network is configured to perform Kohonen feature mapping; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein, the plant performance signal represents a qualitative plant performance parameter for the plant, and further wherein the reinforcement signal is outputted in further accordance with a distance parameter which represents a difference between the actual and desired states of the quantitative plant performance parameter.

Claim 34

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the action network comprises a neural network, and further wherein the neural network is configured to perform Kohonen feature mapping; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein, the plant performance signal represents a qualitative plant performance parameter for the plant, and further wherein the reinforcement signal is outputted in further accordance with a distance parameter which represents a difference between the actual and desired states of the quantitative plant performance parameter.

Claim 35

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the action network generates the plant control signal in still further accordance with a plurality of fuzzy logic rules; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant, and further wherein the reinforcement signal is outputted in further accordance with a distance parameter which represents a difference between the actual and desired states of the quantitative plant performance parameter.

Claim 36

A plant controller using reinforcement learning for controlling a plant by generating a control signal therefor, the plant controller comprising: an action network for coupling to a plant and receiving therefrom a plant state signal, receiving a reinforcement signal and in accordance therewith generating a plant control signal, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant, the reinforcement signal represents a desired state of a performance control parameter for the action network, and the plant control signal represents a desired state of the quantitative plant performance parameter, and wherein the action network generates the plant control signal in still further accordance with a plurality of fuzzy logic rules; and a critic network, coupled to the action network, for coupling to the plant and receiving therefrom the plant state signal and a plant performance signal and in accordance therewith outputting the reinforcement signal to the action network, wherein the plant performance signal represents a qualitative plant performance parameter for the plant, and further wherein the reinforcement signal is outputted in further accordance with a distance parameter which represents a difference between the actual and desired states of the quantitative plant performance parameter.

Claim 37

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant; generating a reinforcement signal in accordance with the plant state signal and the plant performance signal, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal and in further accordance with unsupervised learning and Kohonen feature mapping with an unsupervised learning based neural network which includes first and second pluralities of inter-neuron weights data associated therewith, wherein the plant control signal represents a desired state of the quantitative plant performance parameter.

Claim 38

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant; generating a reinforcement signal in accordance with the plant state signal and the plant performance signal, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal and in further accordance with unsupervised learning and Kohonen feature mapping with an unsupervised learning based neural network which includes first and second pluralities of inter-neuron weights data associated therewith, wherein the plant control signal represents a desired state of the quantitative plant performance parameter.

Claim 39

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant; generating a reinforcement signal in accordance with the plant state signal and the plant performance signal, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal and in further accordance with unsupervised learning and in still further accordance with a plurality of fuzzy logic rules, wherein the plant control signal represents a desired state of the quantitative plant performance parameter.

Claim 40

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant; generating a reinforcement signal in accordance with the plant state signal and the plant performance signal, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal and the reinforcement signal and in further accordance with unsupervised learning and in still further accordance with a plurality of fuzzy logic rules, wherein the plant control signal represents a desired state of the quantitative plant performance parameter.

Claim 41

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant, generating a reinforcement signal in accordance with the plant state signal and the plant performance signal and in further accordance with a distance parameter which represents a difference between the actual state and a desired state of the quantitative plant performance parameter, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal, the reinforcement signal and Kohonen feature mapping with a neural network which includes first and second pluralities of inter-neuron weights data associated therewith, wherein the plant control signal represents the desired state of the quantitative plant performance parameter.

Claim 42

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant, generating a reinforcement signal in accordance with the plant state signal and the plant performance signal and in further accordance with a distance parameter which represents a difference between the actual state and a desired state of the quantitative plant performance parameter, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal, the reinforcement signal and Kohonen feature mapping with a neural network which includes first and second pluralities of inter-neuron weights data associated therewith, wherein the plant control signal represents the desired state of the quantitative plant performance parameter.

Claim 43

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant, generating a reinforcement signal in accordance with the plant state signal and the plant performance signal and in further accordance with a distance parameter which represents a difference between the actual state and a desired state of the quantitative plant performance parameter, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal, the reinforcement signal and a plurality of fuzzy logic rules, wherein the plant control signal represents the desired state of the quantitative plant performance parameter.

Claim 44

A plant control method using reinforcement learning for controlling a plant by generating a control signal therefor, comprising the steps of: receiving a plant state signal and a plant performance signal from a plant, wherein the plant state signal represents an actual state of a quantitative plant performance parameter for the plant and the plant performance signal represents a qualitative plant performance parameter for the plant, generating a reinforcement signal in accordance with the plant state signal and the plant performance signal and in further accordance with a distance parameter which represents a difference between the actual state and a desired state of the quantitative plant performance parameter, wherein the reinforcement signal represents a desired state of a performance control parameter; receiving the reinforcement signal; and generating a plant control signal in accordance with the plant state signal, the reinforcement signal and a plurality of fuzzy logic rules, wherein the plant control signal represents the desired state of the quantitative plant performance parameter.