Claims Analysis
Total Claims: 7
Claim 1
Independent Claim
249 words
A neural network model having an input line for receiving state information for a plurality of states, and an output generator for controlling the movement of an object along a path of selected states among said plurality of states, said neural network model comprising: a satisfaction unit, comprising: a satisfaction index; means for detecting a first state, wherein said first state is a current state; first determining means for determining that said current state is a non-goal state; first modifying means, responsive to said first determining means, for modifying said satisfaction index to indicate a reduced level of satisfaction; second determining means for determining that said current state is a goal state; second modifying means, responsive to said second determining means, for modifying said satisfaction index to indicate an increased level of satisfaction; at least three action units corresponding to at least three directions of movement, each of said action units comprising: means for increasing a randomness factor if said satisfaction index indicates a low level of satisfaction; means for decreasing said randomness factor if said satisfaction index indicates a high level of satisfaction; means for randomly selecting by said randomness factor a temporary weight from a temporary weight range; means for adding a permanent weight to said temporary weight to achieve an effective weight; and sending means for sending an indication to move said object in the direction of movement that corresponds to said action unit to said output generator if said effective weight exceeds a predetermined value.
Claim 2
Dependent Claim
249 words
A neural network model having an input line for receiving state information for a plurality of states, and an output generator for controlling the movement of an object along a path of selected states among said plurality of states, said neural network model comprising: a satisfaction unit, comprising: a satisfaction index; means for detecting a first state, wherein said first state is a current state; first determining means for determining that said current state is a non-goal state; first modifying means, responsive to said first determining means, for modifying said satisfaction index to indicate a reduced level of satisfaction; second determining means for determining that said current state is a goal state; second modifying means, responsive to said second determining means, for modifying said satisfaction index to indicate an increased level of satisfaction; at least three action units corresponding to at least three directions of movement, each of said action units comprising: means for increasing a randomness factor if said satisfaction index indicates a low level of satisfaction; means for decreasing said randomness factor if said satisfaction index indicates a high level of satisfaction; means for randomly selecting by said randomness factor a temporary weight from a temporary weight range; means for adding a permanent weight to said temporary weight to achieve an effective weight; and sending means for sending an indication to move said object in the direction of movement that corresponds to said action unit to said output generator if said effective weight exceeds a predetermined value.
Claim 3
Dependent Claim
44 words
The neural network model of claim 1, wherein said first modifying means further comprises: means for using a decay constant that indicates relative importance of previously selected states versus said current state as a factor in said first modifying means modifying said satisfaction index.
Claim 4
Dependent Claim
44 words
The neural network model of claim 1, wherein said second modifying means further comprises: means for using a decay constant that indicates relative importance of previously selected states versus said current state as a factor in said second modifying means modifying said satisfaction index.
Claim 5
Dependent Claim
35 words
The neural network model of claim 1, wherein said output generator further comprises: means for moving said current state of said object from said first state to a second state responsive to said sending means.
Claim 6
Dependent Claim
70 words
The neural network model of claim 4, wherein each of said action units further comprises: means for concluding that moving said object from said first state to said second state was better than moving said object from said first state to any previously selected state; and means, responsive to said concluding means, for storing said effective weight as said permanent weight for said first state in a permanent weight vector.
Claim 7
Dependent Claim
97 words
The neural network model of claim 5, wherein said analyzing means further comprises: means for incrementing a history value for said first state as said object is moved from said first state to said second state; means for incrementing a reinforcement accumulator for said first state by an amount indicative of a reward if said second state is a goal state; means for deciding that said reinforcement accumulator divided by said history value exceeds a reinforcement baseline for said first state; and means for replacing said reinforcement baseline by said reinforcement accumulator divided by said history value.