Patent US6498628B2 successfully refreshed from Google Patents.
US6498628B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Motion sensing interface
Assignee: Sony Corp
•
Filing: 1998-11-17
•
Granted: 2002-12-24
•
Expires: 2018-11-17
Abstract
Abstract An electronic appliance remote controller which includes a display screen (which may be part of the appliance, e.g. a TV screen) for displaying icons representing possible operations of the electronic appliance, and a motion detector circuit for detecting a motion within a field of view of the motion detector circuit. The motion detector circuit either detects an image of the user's hand or a predetermined motion of a moving hand within the field of view as an indication that a remote control operation is to be started and, thereafter, tracks the movement of the hand. The motion detector circuit outputs a cursor control signal representative of the motion of the hand. A control circuit, connected to the display screen, the electronic appliance, and the motion detector circuit and supplied with the cursor control signal, controls the display screen to display a movable visual indicator, e.g. a cursor, whose own motion tracks the movement of the moving hand and the electronic appliance to perform operations corresponding to the icons selected by the user using the visual indicator. In one embodiment, two cameras allow three dimensional movements of the user's hand to control the electronic appliance.
Inventors
- 👤 Ryuichi Iwamura
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:02
CPC Classifications
G; G08; G08C; G08C23/00; G06; G06F; G06F3/00; G06F3/002; G06F3/005; G06F3/01; G06F3/017; G06F3/03; G06F3/0304; G08C2201/00; G08C2201/30; G08C2201/32; H; H04; H04N; H04N21/00; H04N21/40; H04N21/47
Claims Analysis
Total Claims: 46
Claim 1
Independent Claim
144 words
A remote controller for an electronic appliance, comprising: a display screen for displaying icons representing possible operations of the electronic appliance; a motion detector for detecting a hand motion within a field of view of the motion detector, the motion detector first detecting a predetermined motion of a user's hand within the field of view to initiate remote control operation and, thereafter, tracking the movement of the hand and outputting a cursor control signal representative of the motion of the hand; and a control circuit, connected to the display screen, the electronic appliance, and the motion detector and supplied with the cursor control signal, for controlling the display screen to display a movable visual indicator which tracks the movement of the moving hand and for controlling the electronic appliance to perform operations corresponding to the icons selected by the user using the visual indicator.
Claim 2
Dependent Claim
144 words
A remote controller for an electronic appliance, comprising: a display screen for displaying icons representing possible operations of the electronic appliance; a motion detector for detecting a hand motion within a field of view of the motion detector, the motion detector first detecting a predetermined motion of a user's hand within the field of view to initiate remote control operation and, thereafter, tracking the movement of the hand and outputting a cursor control signal representative of the motion of the hand; and a control circuit, connected to the display screen, the electronic appliance, and the motion detector and supplied with the cursor control signal, for controlling the display screen to display a movable visual indicator which tracks the movement of the moving hand and for controlling the electronic appliance to perform operations corresponding to the icons selected by the user using the visual indicator.
Claim 3
Dependent Claim
44 words
A remote controller according to claim 1 , wherein the motion detector detects the selection of an icon by the user by detecting a predetermined motion pattern of the hand when the visual indicator is coincident on the display screen with a particular icon.
Claim 4
Dependent Claim
41 words
A remote controller according to claim 2 , wherein the predetermined motion pattern is a cessation of movement of the hand for a predetermined period of time after the visual indicator is coincident on the display screen with the particular icon.
Claim 5
Dependent Claim
41 words
A remote controller according to claim 2 , wherein the predetermined motion pattern is a movement of the hand along an axis perpendicular to the display screen after the visual indicator is coincident on the display screen with the particular icon.
Claim 6
Dependent Claim
18 words
A remote controller according to claim 1 , wherein the motion detector includes at least one video camera.
Claim 7
Dependent Claim
84 words
A remote controller according to claim 5 , wherein the motion detector detects motion by comparing the pixel values of each corresponding macro block between two time sequential video frames in a video output signal from the video camera and determining that motion has taken place with respect to a macro block if the difference in pixel values between the two video frames for that macro block exceeds a predetermined threshold value and, if so, determining that macro block to be an active region.
Claim 8
Dependent Claim
55 words
A remote controller according to claim 6 , wherein in detecting the user's hand, the motion detector checks each macro block of the two time sequential video frames to determine if a grouping of physically proximate macro blocks in a single video frame exhibits a predetermined pattern of becoming active regions and then non-active regions.
Claim 9
Dependent Claim
91 words
A remote controller according to claim 7 , wherein in detecting the user's hand, the motion detector, after detecting that a grouping of physically proximate macro blocks in a single video frame exhibits a predetermined pattern of becoming active regions and then non-active regions, compares a portion of the video image contained in the current active region with a stored video image and determines that the current active region of the video image output by the camera is the user's hand if there is a match with the stored video image.
Claim 10
Dependent Claim
64 words
A remote controller according to claim 5 , wherein in detecting the user's hand, the motion detector compares a user selected portion of the video image output by the camera with a stored video image and determines that the user selected portion of the video image output by the camera is the user's hand if there is a match with the stored video image.
Claim 11
Dependent Claim
15 words
A remote controller according to claim 1 , wherein the visual indicator is a cursor.
Claim 12
Dependent Claim
16 words
A remote controller according to claim 1 , wherein the electronic appliance is a television set.
Claim 13
Dependent Claim
17 words
A remote controller according to claim 1 , wherein the display screen is a cathode ray tube.
Claim 14
Dependent Claim
47 words
A remote controller according to claim 1 , further comprising an automatic cursor sensitivity control means for automatically adjusting a length of movement of the visual indicator on the display screen relative to a corresponding length of movement of the moving hand detected by the motion detector.
Claim 15
Dependent Claim
49 words
A remote controller according to claim 13 , wherein the automatic cursor sensitivity control means adjusts the length of movement of the visual indicator on the display screen to move a fixed, predetermined distance on the display screen in response to the length of the detected first predetermined motion.
Claim 16
Dependent Claim
27 words
A remote controller according to claim 1 , wherein the motion detector includes two video cameras placed perpendicularly with respect to each other to view the user.
Claim 17
Dependent Claim
83 words
A remote controller according to claim 15 , wherein the motion detector detects motion by comparing the pixel values of each macro block between two time sequential video frames in a video output signal from each video camera and determining that motion has taken place with respect to a macro block if the difference in pixel values between the two video frames for that macro block exceed a predetermined threshold value and, if so, determining that macro block to be an active region.
Claim 18
Dependent Claim
58 words
A remote controller according to claim 16 , wherein in detecting the user's hand, the motion detector checks each macro block of the two time sequential video frames of each camera to determine if a grouping of physically proximate macro blocks in a single video frame exhibits a predetermined pattern of becoming active regions and then non-active regions.
Claim 19
Dependent Claim
67 words
A remote controller according to claim 17 , wherein in detecting the user's hand, the motion detector, after detecting that a grouping of corresponding physically proximate macro blocks in a single video frame of the output of each camera exhibits a predetermined pattern of becoming active regions and then non-active regions, locks onto the active regions of the video images of both cameras as the active regions.
Claim 20
Dependent Claim
98 words
A remote controller according to claim 17 , wherein in detecting the user's hand, the motion detector, after detecting that a grouping of corresponding physically proximate macro blocks in a single video frame of the output of each camera exhibits a predetermined pattern of becoming active regions and then non-active regions, compares a portion of the video image contained in the current active region with a stored video image and determines that the current active region of the video image output by the cameras is the user's hand if there is a match with the stored video image.
Claim 21
Dependent Claim
44 words
A remote controller according to claim 15 , wherein the motion detector detects the selection of an icon by the user by detecting a predetermined motion pattern of the hand when the visual indicator is coincident on the display screen with a particular icon.
Claim 22
Dependent Claim
41 words
A remote controller according to claim 20 , wherein the predetermined motion pattern is a cessation of movement of the hand for a predetermined period of time after the visual indicator is coincident on the display screen with the particular icon.
Claim 23
Dependent Claim
41 words
A remote controller according to claim 20 , wherein the predetermined motion pattern is a movement of the hand along an axis perpendicular to the display screen after the visual indicator is coincident on the display screen with the particular icon.
Claim 24
Independent Claim
122 words
A method of remote controlling an electronic appliance, comprising the steps of: displaying icons on a display screen representing possible operations of the electronic appliance; detecting a hand motion within a field of view of the motion detector by the steps of first detecting a predetermined motion of a user's hand within the field of view to initiate remote control operation and, thereafter, tracking the movement of the hand and outputting a cursor control signal representative of the motion of the hand; and controlling the display screen to display a movable visual indicator which tracks the movement of the moving hand and for controlling the electronic appliance to perform operations corresponding to the icons selected by the user using the visual indicator.
Claim 25
Independent Claim
122 words
A method of remote controlling an electronic appliance, comprising the steps of: displaying icons on a display screen representing possible operations of the electronic appliance; detecting a hand motion within a field of view of the motion detector by the steps of first detecting a predetermined motion of a user's hand within the field of view to initiate remote control operation and, thereafter, tracking the movement of the hand and outputting a cursor control signal representative of the motion of the hand; and controlling the display screen to display a movable visual indicator which tracks the movement of the moving hand and for controlling the electronic appliance to perform operations corresponding to the icons selected by the user using the visual indicator.
Claim 26
Dependent Claim
48 words
A method of remote controlling according to claim 23 , wherein the motion detecting step includes detecting the selection of an icon by the user by detecting a predetermined motion pattern of the hand when the visual indicator is coincident on the display screen with a particular icon.
Claim 27
Dependent Claim
43 words
A method of remote controlling according to claim 23 , wherein the predetermined motion pattern is a cessation of movement of the hand for a predetermined period of time after the visual indicator is coincident on the display screen with the particular icon.
Claim 28
Dependent Claim
43 words
A method of remote controlling according to claim 23 , wherein the predetermined motion pattern is a movement of the hand along an axis perpendicular to the display screen after the visual indicator is coincident on the display screen with the particular icon.
Claim 29
Dependent Claim
33 words
A method of remote controlling according to claim 23 , wherein the motion detecting step includes the step of using a video camera to generate a video image of the field of view.
Claim 30
Dependent Claim
90 words
A method of remote controlling according to claim 27 , wherein the motion detecting step includes the steps of detecting motion by comparing the pixel values of each macro block between two time sequential video frames in a video output signal from the video camera and determining that motion has taken place with respect to a macro block if the difference in pixel values between the two video frames for that macro block exceed a predetermined threshold value and, if so, determining that macro block to be an active region.
Claim 31
Dependent Claim
62 words
A method of remote controlling according to claim 28 , wherein in detecting the user's hand, the motion detecting step includes the steps of checking each macro block of the two time sequential video frames to determine if a grouping of physically proximate macro blocks in a single video frame exhibits a predetermined pattern of becoming active regions and then non-active regions.
Claim 32
Dependent Claim
102 words
A remote controller according to claim 28 , wherein in detecting the user's hand, the motion detecting step includes the steps, after detecting that a grouping of corresponding physically proximate macro blocks in a single video frame of the output of each camera exhibits a predetermined pattern of becoming active regions and then non-active regions, comparing a portion of the video image contained in the current active region with a stored video image and determining that the current active region of the video image output by the cameras is the user's hand if there is a match with the stored video image.
Claim 33
Dependent Claim
98 words
A method of remote controlling according to claim 30 , wherein in detecting the user's hand, the motion detecting step includes the steps of, after detecting that a grouping of physically proximate macro blocks in a single video frame exhibits a predetermined pattern of becoming active regions and then non-active regions, comparing a portion of the video image contained in the current active region with a stored video image and determining that the current active region of the video image output by the camera is the user's hand if there is a match with the stored video image.
Claim 34
Dependent Claim
17 words
A method of remote controlling according to claim 23 , wherein the visual indicator is a cursor.
Claim 35
Dependent Claim
18 words
A method of remote controlling according to claim 23 , wherein the electronic appliance is a television set.
Claim 36
Dependent Claim
19 words
A method of remote controlling according to claim 23 , wherein the display screen is a cathode ray tube.
Claim 37
Dependent Claim
46 words
A method of remote controlling according to claim 23 , further comprising the steps of automatically adjusting a length of movement of the visual indicator on the display screen relative to a corresponding length of movement of the moving hand detected by the motion detecting step.
Claim 38
Dependent Claim
49 words
A method of remote controlling according to claim 35 , further comprising the steps of adjusting the length of movement of the visual indicator on the display screen to move a fixed, predetermined distance on the display screen in response to the length of the detected first predetermined motion.
Claim 39
Dependent Claim
42 words
A method of remote controlling according to claim 23 , wherein the motion detecting step includes generating two simultaneous video images of the field of view by using two video cameras placed perpendicularly with respect to each other to view the user.
Claim 40
Dependent Claim
90 words
A method of remote controlling according to claim 37 , wherein the motion detecting step includes the steps of detecting motion by comparing the pixel values of each macro block between two time sequential video frames in a video output signal from each video camera and determining that motion has taken place with respect to a macro block if the difference in pixel values between the two video frames for that macro block exceed a predetermined threshold value and, if so, determining that macro block to be an active region.
Claim 41
Dependent Claim
65 words
A method of remote controlling according to claim 32 , wherein in detecting the user's hand, the motion detecting step includes the steps of checking each macro block of the two time sequential video frames of each camera to determine if a grouping of physically proximate macro blocks in a single video frame exhibits a predetermined pattern of becoming active regions and then non-active regions.
Claim 42
Dependent Claim
105 words
A method of remote controlling according to claim 39 , wherein in detecting the user's hand, the motion detecting step includes the steps of, after detecting that a grouping of corresponding physically proximate macro blocks in a single video frame of the output of each camera exhibits a predetermined pattern of becoming active regions and then non-active regions, comparing a portion of the video image contained in the current active region with a stored video image and determining that the current active region of the video image output by the cameras is the user's hand if there is a match with the stored video image.
Claim 43
Dependent Claim
71 words
A method of remote controlling according to claim 37 , wherein in detecting the user's hand, the motion detecting step includes the steps of comparing a user selected portion of the video image output by the cameras with a stored video image and determining that the user selected portion of the video image output by the cameras is the user's hand if there is a match with the stored video image.
Claim 44
Dependent Claim
51 words
A method of remote controlling according to claim 37 , wherein the motion detecting step includes the steps of detecting the selection of an icon by the user by detecting a predetermined motion pattern of the hand when the visual indicator is coincident on the display screen with a particular icon.
Claim 45
Dependent Claim
43 words
A method of remote controlling according to claim 42 , wherein the predetermined motion pattern is a cessation of movement of the hand for a predetermined period of time after the visual indicator is coincident on the display screen with the particular icon.
Claim 46
Dependent Claim
43 words
A method of remote controlling according to claim 42 , wherein the predetermined motion pattern is a movement of the hand along an axis perpendicular to the display screen after the visual indicator is coincident on the display screen with the particular icon.
References Cited (Backward Citations)
US4988981A
Cited
US4988981B1
Cited
US5444791A
Cited
US5808678A
Cited
US5617312A
Cited
US5473364A
Cited
US5594469A
Cited
US6191773B1
Cited
US5991428A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-12-23 | 2006-06-24 | 2006-12-24 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-12-23 | 2010-06-24 | 2010-12-24 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-12-23 | 2014-06-24 | 2014-12-24 | $7700.00 | Window Closed / Lapsed |