Instruction recognition system having gesture recognition function

Claim 2

An instruction recognition system having a gesture recognition function for recognizing a user's gesture, the system comprising: a nodding action recognition section which detects the user's nodding action on the basis of a photographed image of a user's head; a line-of-sight detection section for detecting the line of a user's sight; a display section for indicating image data; a determination section which determines whether or not the nodding action recognition section has detected nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time and the line of sight detected by the line-of-sight detection section during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; an execution section which performs predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when the determination section determines that the nodding action recognition section has detected nodding action and that the line of sight detected by the line-of-sight detection section during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section.

Claim 3

The instruction recognition system as defined in claim 1 , wherein the detected line of sight is determined to be directed toward the candidate for selection on condition that the line of sight detected by the line-of-sight detection section corresponds to a gazing state where the eyes remain fixed in one orientation for a predetermined period of time.

Claim 4

The instruction recognition system as defined in claim 2 , wherein the gazing state corresponds to a state in which the line of sight is directed toward a predetermined candidate for selection from the beginning of the nodding action to the end thereof.

Claim 5

The instruction recognition system as defined in claim 1 , wherein the nodding action recognition section detects nodding action according to the amount of vertical oscillation of the user's head.

Claim 6

The instruction recognition system as defined in claim 1 , wherein the nodding action recognition section recognizes nodding action when a certain portion of the user's head moves vertically within a predetermined time period and when the amount of vertical movement exceeds predetermined amounts.

Claim 7

The instruction recognition system as defined in claim 1 , further comprising a photography section for photographing the user's head; wherein the nodding action recognition section detects nodding action from the image data photographed by the photography section; and the line-of-sight detection section detects a line of sight from the image data produced through photography by the photography section.

Claim 8

The instruction recognition system as defined in claim 6 , wherein the nodding action recognition section detects nodding action through use of at least data pertaining to the positions of the user's eyes and/or mouth of the image data produced by the photography section.

Claim 9

The instruction recognition system as defined in claim 6 , wherein the nodding action recognition section detects nodding action through use of at least data pertaining to the center of gravity of a flesh-colored region on the user's head of the image data obtained by the photography section.

Claim 10

An instruction recognition system having a gesture recognition function for recognition the user's gesture, the system comprising: a nodding action recognition means for detecting the user's nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time on the basis of a photographed image of the user's head; a line-of-sight detection means for detecting the line of the user's sight; a display means for indicating image data; a determination means which determines whether or not the nodding action recognition means has detected nodding action and the line of sight detected by the line-of-sight detection means during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display means; an execution means which performs predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when the determination means determines that the nodding action recognition means has detected nodding action and that the line of sight detected by the line-of-sight detection means during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display means.

Claim 11

An instruction recognition system having a gesture recognition function for recognition the user's gesture, the system comprising: a nodding action recognition means for detecting the user's nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time on the basis of a photographed image of the user's head; a line-of-sight detection means for detecting the line of the user's sight; a display means for indicating image data; a determination means which determines whether or not the nodding action recognition means has detected nodding action and the line of sight detected by the line-of-sight detection means during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display means; an execution means which performs predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when the determination means determines that the nodding action recognition means has detected nodding action and that the line of sight detected by the line-of-sight detection means during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display means.

Claim 12

An instruction recognition system having a gesture recognition function for recognizing a user's gesture, the system comprising: a photographing device for photographing a user's head; a display device for displaying image information; and a controller performing a nodding action recognition operation for detecting the user's nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time on the basis of the image photographed by the photographing device; a line-of-sight detection operation for detecting the line of a user's sight on the basis of the image photographed by the photographing device; a determination operation for determining whether or not the nodding action has been detected by the nodding action recognition operation and the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; an execution operation for performing predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when it is determined that nodding action has been detected by the nodding action recognition operation and that the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section.

Claim 13

An instruction recognition system having a gesture recognition function for recognizing a user's gesture, the system comprising: a photographing device for photographing a user's head; a display device for displaying image information; and a controller performing a nodding action recognition operation for detecting the user's nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time on the basis of the image photographed by the photographing device; a line-of-sight detection operation for detecting the line of a user's sight on the basis of the image photographed by the photographing device; a determination operation for determining whether or not the nodding action has been detected by the nodding action recognition operation and the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; an execution operation for performing predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when it is determined that nodding action has been detected by the nodding action recognition operation and that the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section.

Claim 14

The instruction recognition system according to claim 10 , wherein the controller determines that the detected line of sight is directed toward the candidate for selection on condition that the line of sight detected by the line-of-sight detection operation corresponds to a gazing state where the eyes remain fixed in one orientation for a predetermined period of time.

Claim 15

The instruction recognition system according to claim 11 , wherein the gazing state corresponds to a state in which the line of sight is directed toward a predetermined candidate for selection from the beginning of the nodding action to the end thereof.

Claim 16

The instruction recognition system according to claim 10 , wherein during the nodding action detection operation, the controller detects nodding action according to the amount of vertical oscillation of the user's head.

Claim 17

The instruction recognition system according to claim 10 , wherein during the nodding action detection operation, the controller recognizes nodding action when a certain portion of the user's head moves vertically within a predetermined time period and when the amount of vertical movement exceeds predetermined amounts.

Claim 18

The instruction recognition system according to claim 10 , wherein the controller detects nodding action through use of at least data pertaining to the positions of the user's eyes and/or mouth of the image data produced by the photographing device.

Claim 19

The instruction recognition system according to claim 10 , wherein the controller detects nodding action through use of at least data pertaining to the center of gravity of a flesh-colored region on the user's head of the image data obtained by the photographing device.

Claim 20

An instruction recognition system having a gesture recognition function for recognizing a user's gesture, the system comprising: a photographing device for photographing a user's head; a display device for displaying image information; and a controller performing a nodding action recognition operation for detecting the user's nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time on the basis of the image photographed by the photographing device; a line-of-sight detection operation for detecting the line of a user's sight on the basis of the image photographed by the photographing device; a determination operation for determining whether or not the nodding action has been detected by the nodding action recognition operation and the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; an execution operation for performing predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when it is determined that nodding action has been detected by the nodding action recognition operation and that the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; wherein the controller detects nodding action through use of at least data pertaining to the positions of the user's eyes and/or mouth of the image data produced by the photographing device.

Claim 21

An instruction recognition system having a gesture recognition function for recognizing a user's gesture, the system comprising: a photographing device for photographing a user's head; a display device for displaying image information; and a controller performing a nodding action recognition operation for detecting the user's nodding action defined as one of a down then up and an up then down head movement performed in continuum within a specified period of time on the basis of the image photographed by the photographing device; a line-of-sight detection operation for detecting the line of a user's sight on the basis of the image photographed by the photographing device; a determination operation for determining whether or not the nodding action has been detected by the nodding action recognition operation and the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; an execution operation for performing predetermined processing, on the assumption that the candidate toward which the line of the user's sight is directed will be selected, when it is determined that nodding action has been detected by the nodding action recognition operation and that the line of sight detected by the line-of-sight detection operation during the nodding action is directed toward any of the candidates for selection, while the system awaits the user's instruction and one or a plurality of candidates for selection are displayed on the display section; wherein the controller detects nodding action through use of at least data pertaining to the positions of the user's eyes and/or mouth of the image data produced by the photographing device.