US6760645B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Training of autonomous robots
Assignee: Sony France SA
•
Filing: 2002-04-29
•
Granted: 2004-07-06
•
Expires: 2022-04-29
Abstract
Abstract A clicker-training technique developed for animal training is adapted for training robots, notably autonomous animal-like robots. In this robot-training method, a behaviour (for example, (DIG)) is broken down into smaller achievable responses ((SIT)-(HELLO)-(DIG)) that will eventually lead to the desired final behaviour. The robot is guided progressively to the correct behaviour through the use, normally the repeated use, of a secondary reinforcer. When the correct behaviour has been achieved, a primary reinforcer is applied so that the desired behaviour can be "captured". This method can be used for training a robot to perform, on command, rare behaviours or a sequence of behaviours (typically actions). This method can also be used to ensure that a robot is focusing its attention upon a desired object.
Inventors
- 👤 Frédéric Kaplan
- 👤 Pierre-Yves Oudeyer
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-25 08:00
CPC Classifications
A; A63; A63H; A63H11/00; B; B25; B25J; B25J9/00; B25J9/16; B25J9/1656; G; G05; G05B; G05B19/00; G05B19/02; G05B19/42; A63H2200/00
Claims Analysis
Total Claims: 18
Claim 1
Independent Claim
85 words
A method of programming a robot to perform a desired behaviour, the method comprising the steps of: providing a robot for recognizing at least one stimulus as a primary reinforcer and; conditioning the robot to recognize at least one further stimulus as a secondary reinforcer; guiding the robot to the desired behaviour by presenting the robot with a secondary reinforcer when the robot exhibits a behaviour approaching the desired behaviour and presenting the robot with a primary reinforcer when the robot exhibits the desired behaviour.
Claim 2
Independent Claim
85 words
A method of programming a robot to perform a desired behaviour, the method comprising the steps of: providing a robot for recognizing at least one stimulus as a primary reinforcer and; conditioning the robot to recognize at least one further stimulus as a secondary reinforcer; guiding the robot to the desired behaviour by presenting the robot with a secondary reinforcer when the robot exhibits a behaviour approaching the desired behaviour and presenting the robot with a primary reinforcer when the robot exhibits the desired behaviour.
Claim 3
Dependent Claim
66 words
The robot programming method of claim 1 , wherein the providing step comprises providing a robot which, in use, in response to perception of a secondary reinforcer, exhibits a behaviour related to the exhibited behaviour that prompted appearance of the secondary reinforcer and, in response to perception of a primary reinforcer, registers one or more of the exhibited behaviours that prompted appearance of the primary reinforcer.
Claim 4
Dependent Claim
61 words
The robot programming method of claim 2 , wherein the providing step comprises providing a robot which, in use, in response to perception of a primary reinforcer, repeats the one or more registered behaviours, and the method further comprises the step of presenting the robot with a primary reinforcer if the repeated one or more behaviours corresponds to the desired behaviour.
Claim 5
Dependent Claim
59 words
A method according to claim 1 , for programming a robot to perform a sequence of desired behaviours, the method comprising the step of presenting the robot with a secondary reinforcer when the robot exhibits a desired behaviour of said sequence and presenting the robot with a primary reinforcer after the robot has exhibited the sequence of desired behaviours.
Claim 6
Dependent Claim
71 words
The robot programming method of claim 1 , wherein the desired behaviour is the directing of the robot's attention on a particular subject, and the guiding step comprises presenting the robot with a secondary reinforcer as the robot directs the visual apparatus thereof more and more precisely towards said particular subject and presenting the robot with a primary reinforcer when the robot directs the visual apparatus thereof at said particular subject.
Claim 7
Dependent Claim
41 words
The robot programming method of claim 1 , wherein the step of conditioning the robot to recognize at least one further stimulus as a secondary reinforcer comprises repeatedly presenting the robot with said further stimulus in association with a primary reinforcer.
Claim 8
Dependent Claim
27 words
The robot programming method of claim 1 , and comprising the step of providing the robot with a command indication immediately after provision of a primary reinforcer.
Claim 9
Dependent Claim
28 words
The robot programming method of claim 7 , wherein the providing step comprises providing a robot which, in use, provides feedback enabling the command indication to be confirmed.
Claim 10
Dependent Claim
19 words
The robot programming method of claim 7 , wherein the command indication is a spoken word or hand signal.
Claim 11
Dependent Claim
66 words
The robot programming method of claim 1 , wherein the providing step comprises providing a robot which, in use, undergoes a transition from one behaviour to another behaviour thereof according to a respective probability, wherein the probability of a transition taking place between a particular pair of behaviours is increased if the exhibition of said transition occurs and prompts appearance of a secondary or primary reinforcer.
Claim 12
Dependent Claim
43 words
An autonomous robot programmable by a method according to claim 1 , wherein the robot comprises: means for recognizing at least one stimulus as a primary reinforcer, and means for enabling at least one further stimulus to be identified as a secondary reinforcer.
Claim 13
Dependent Claim
59 words
The autonomous robot according to claim 11 , which, in use, in response to perception of a secondary reinforcer, exhibits a behaviour related to to the exhibited behaviour that prompted appearance of the secondary reinforcer and, in response to perception of a primary reinforcer, registers one or more of the exhibited behaviours that prompted appearance of the primary reinforcer.
Claim 14
Dependent Claim
44 words
The autonomous robot according to claim 12 , which, in use, in response to perception of a primary reinforcer, repeats the one or more registered behaviours, and confirms registration of said one or more behaviours if the repetition prompts appearance of a primary reinforcer.
Claim 15
Dependent Claim
39 words
The autonomous robot according to claim 11 , wherein said enabling means for recognizing at least one further stimulus as a secondary reinforcer when the robot perceives said further stimulus repeatedly presented thereto in association with a primary stimulus.
Claim 16
Dependent Claim
27 words
The autonomous robot according to claim 11 , which, in use, in response to perception of a primary reinforcer, awaits presentation of, and registers, a command indication.
Claim 17
Dependent Claim
20 words
The autonomous robot according to claim 15 , which, in use, provides feedback enabling the command indication to be confirmed.
Claim 18
Dependent Claim
58 words
The autonomous robot according to claim 11 , which, in use, undergoes a transition from one behaviour to another behaviour thereof according to a respective probability, wherein the probability of a transition taking place between a particular pair of behaviours is increased if the exhibition of said transition occurs and prompts appearance of a secondary or primary reinforcer.
References Cited (Backward Citations)
US4657104A
Cited
US6058385A
Cited
US6275773B1
Cited
US5963712A
Cited
US6038493A
Cited
US6321140B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-07-06 | 2008-01-05 | 2008-07-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-07-06 | 2012-01-05 | 2012-07-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-07-06 | 2016-01-05 | 2016-07-06 | $7700.00 | Window Closed / Lapsed |