Patent US5168531A successfully refreshed from Google Patents.
US5168531A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Real-time recognition of pointing information from video
Assignee: Hewlett Packard Development Co LP
•
Filing: 1991-06-27
•
Granted: 1992-12-01
•
Expires: 2011-06-27
Abstract
Abstract An occurrence of a predefined object in an image is recognized by receiving image data, convolving the image data with a set of predefined functions to analyze the image data into occurrences of predefined elementary features, and examining the occurrences for an occurrence of a predefined combination of the elementary features that is characteristic of the predefined object. Preferably the image data are convolved directly with a first predefined function to determine blob responses, and a second predefined function to determine ganglia responses indicating edges of objects. Then the ganglia responses are convolved with a third predefined function to determine simple responses indicating lines in the image, and the simple responses are combined with the ganglia responses to determine complex responses indicating terminated line segments in the image. A pointing finger, for example, is recognized from the combination of a blob response and a complex response. The method, for example, permits a data input terminal to recognize in real time the presence, position, and orientation of a pointing finger, to eliminate the need for data input devices such as "mice" or "joysticks." Therefore a user can direct an application program in the most natural way, without the distraction of manipulating a data input device.
Inventors
- 👤 Claude Sigel
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-30 09:35
CPC Classifications
G; G06; G06F; G06F3/00; G06F3/01; G06F3/011; G06F3/017; G06F3/03; G06F3/041; G06F3/042; G06F3/0425; G06V; G06V10/00; G06V10/40; G06V10/44; G06V10/443
Claims Analysis
Total Claims: 33
Claim 1
Independent Claim
184 words
A method of operating a digital computer to recognize an occurrence of a predefined object in an image, said method comprising the steps of: a) receiving image data representing samples of said image at spaced locations in said image; b) convolving said image data with a set of predefined functions to analyze the image data into occurrences of predefined elementary features; and c) examining the occurrences of the predefined elementary features for an occurrence of a predefined combination of occurrences of said elementary features that is characteristic of the predefined object; wherein said predefined elementary features include blobs, simples and complexes, wherein the blobs indicate regions of similar luminosity, simples indicate lines in said image, and complexes indicate terminated line segments in said image; and wherein the image data are convolved with a first one of said functions to provide blob responses, the image data are convolved with a second one of said functions to provide ganglia responses indicating edges of objects, the ganglia responses are convolved to provide simple responses, and the simple responses and ganglia responses are combined to provide complex responses.
Claim 2
Independent Claim
184 words
A method of operating a digital computer to recognize an occurrence of a predefined object in an image, said method comprising the steps of: a) receiving image data representing samples of said image at spaced locations in said image; b) convolving said image data with a set of predefined functions to analyze the image data into occurrences of predefined elementary features; and c) examining the occurrences of the predefined elementary features for an occurrence of a predefined combination of occurrences of said elementary features that is characteristic of the predefined object; wherein said predefined elementary features include blobs, simples and complexes, wherein the blobs indicate regions of similar luminosity, simples indicate lines in said image, and complexes indicate terminated line segments in said image; and wherein the image data are convolved with a first one of said functions to provide blob responses, the image data are convolved with a second one of said functions to provide ganglia responses indicating edges of objects, the ganglia responses are convolved to provide simple responses, and the simple responses and ganglia responses are combined to provide complex responses.
Claim 3
Dependent Claim
24 words
The method as claimed in claim 1, wherein the simples include respective indications of lines at six different angular orientations at 30 degree increments.
Claim 4
Dependent Claim
22 words
The method as claimed in claim 1, wherein said predefined combination of said elementary features includes a complex extending from a blob.
Claim 5
Dependent Claim
19 words
The method as claimed in claim 3, wherein said predefined object is a human hand having a pointing finger.
Claim 6
Dependent Claim
34 words
The method as claimed in claim 1, further comprising the step of determining an angular orientation for an occurrence of said predefined combination of said elementary features that is characteristic of the predefined object.
Claim 7
Dependent Claim
20 words
The method as claimed in claim 5, wherein said predefined elementary features include lines at a plurality of angular orientations.
Claim 8
Dependent Claim
31 words
The method as claimed in claim 6, wherein said step of determining an angular orientation for an occurrence includes computing an angle from convolutions for lines at two neighboring angular orientations.
Claim 9
Dependent Claim
41 words
The method as claimed in claim 1, further comprising the step of computing a position for said occurrence of said predefined object by interpolating between positions of occurrences of said predefined combination which occur at neighboring spaced locations in said image.
Claim 10
Dependent Claim
57 words
The method as claimed in claim 1, further comprising the step d) of computing a response for a plurality of occurrences of said predefined combination of said elementary features that is characteristic of the predefined object, and determining a location for the predefined object from one of said occurrences of said predefined combination having a maximum response.
Claim 11
Dependent Claim
19 words
The method as claimed in claim 1, further comprising the step of suppressing features that remain constant with time.
Claim 12
Dependent Claim
67 words
The method as claimed in claim 1, wherein for at least one of said predefined functions, said convolving is performed by computing, for each spaced location in said image, a vector scalar product of said image data and said one of said predefined functions after shifting said one of said predefined functions so that said one of said predefined functions is centered upon said each spaced location.
Claim 13
Independent Claim
142 words
A method of operating a digital computer to recognize an occurrence of a predefined object in an image, said method comprising the steps of: a) receiving image data representing samples of said image at spaced locations in said image; b) convolving said image data with a first predefined function to determine blob responses indicating regions of similar luminance in said image; c) convolving said image data with a second predefined function to determine ganglia responses indicating edges of objects; d) convolving said ganglia responses with a third predefined function to determine simple responses indicating lines in said image; e) combining the simple responses and the ganglia responses to determine complex responses indicating terminated line segments in said image; and f) examining combinations of blob responses and complex responses for an occurrence of a predefined combination that is characteristic of the predefined object.
Claim 14
Independent Claim
142 words
A method of operating a digital computer to recognize an occurrence of a predefined object in an image, said method comprising the steps of: a) receiving image data representing samples of said image at spaced locations in said image; b) convolving said image data with a first predefined function to determine blob responses indicating regions of similar luminance in said image; c) convolving said image data with a second predefined function to determine ganglia responses indicating edges of objects; d) convolving said ganglia responses with a third predefined function to determine simple responses indicating lines in said image; e) combining the simple responses and the ganglia responses to determine complex responses indicating terminated line segments in said image; and f) examining combinations of blob responses and complex responses for an occurrence of a predefined combination that is characteristic of the predefined object.
Claim 15
Dependent Claim
68 words
The method as claimed in claim 12 wherein said simple responses include responses indicating lines in said image at a plurality of angular orientations, and said method further comprises the step g) of determining angular orientation for the occurrence of said predefined combination that is characteristic of the predefined object by computing an angle from two of said simple responses indicating lines having different and neighboring angular orientations.
Claim 16
Dependent Claim
42 words
The method as claimed in claim 12, wherein said predefined object is a human hand, and said predefined combination is a complex response and a blob response indicating a terminated line segment extending from a region of similar luminance in said image.
Claim 17
Dependent Claim
38 words
The method as claimed in claim 12, wherein one of said simple responses is combined with ganglia responses to provide a complex response only when said one of said simple responses has a value exceeding a predetermined threshold.
Claim 18
Dependent Claim
52 words
The method as claimed in claim 12, further comprising the step of computing a position for said occurrence of said predefined object by interpolating between positions of occurrences of said predefined combination which occur at neighboring spaced locations in said image, based upon responses computed for said occurrences of said predefined combination.
Claim 19
Dependent Claim
55 words
The method as claimed in claim 12, wherein step f) further comprises the step of computing responses for a plurality of combinations of blob responses and complex responses that are characteristic of the predefined object, and determining a location for the predefined object from the combination of blob and complex responses having a maximum response.
Claim 20
Dependent Claim
20 words
The method as claimed in claim 12, further comprising the step of suppressing responses that do not change with time.
Claim 21
Dependent Claim
68 words
The method as claimed in claim 12, wherein for each of said first and second predefined functions, said convolving is performed by computing, for each spaced location in said image, a vector scalar product of said image data and said each of said predefined functions after shifting said each of said predefined functions so that said each of said predefined functions is centered upon said each spaced location.
Claim 22
Independent Claim
215 words
A method of operating a digital computer to receive information from a human operator at a computer terminal in response to a pointing finger of the human operator, said method comprising the step of: a) scanning a region of space near said computer terminal to obtain image data representing samples of an image at spaced locations near said computer terminal; and b) analyzing said image data to recognize when the pointing finger is present in the image and to determine the position of the pointing finger in the image when the pointing finger is present, wherein said analyzing includes convolving said image data to resolve the image data into blobs characteristic of a fist of the human operator and complexes characteristic of the pointing finger, and examining the blobs and the complexes for a combination of a complex extending from a blob to recognize the presence of the pointing finger, and wherein said convolving said image data includes: convolving said image data with a first predefined function to determine blob responses; convolving said image data with a second predefined function to determine ganglia responses indicating edges of said finger; convolving said ganglia responses with a third predefined function to determine simple responses; and combining the simple responses and the ganglia responses to determine complex responses.
Claim 23
Independent Claim
215 words
A method of operating a digital computer to receive information from a human operator at a computer terminal in response to a pointing finger of the human operator, said method comprising the step of: a) scanning a region of space near said computer terminal to obtain image data representing samples of an image at spaced locations near said computer terminal; and b) analyzing said image data to recognize when the pointing finger is present in the image and to determine the position of the pointing finger in the image when the pointing finger is present, wherein said analyzing includes convolving said image data to resolve the image data into blobs characteristic of a fist of the human operator and complexes characteristic of the pointing finger, and examining the blobs and the complexes for a combination of a complex extending from a blob to recognize the presence of the pointing finger, and wherein said convolving said image data includes: convolving said image data with a first predefined function to determine blob responses; convolving said image data with a second predefined function to determine ganglia responses indicating edges of said finger; convolving said ganglia responses with a third predefined function to determine simple responses; and combining the simple responses and the ganglia responses to determine complex responses.
Claim 24
Dependent Claim
23 words
The method as claimed in claim 20, further comprising the step of determining an angular orientation of said finger from said image data.
Claim 25
Dependent Claim
35 words
The method as claimed in claim 21, wherein said step of determining an angular orientation of said finger includes computing an angle from convolutions of the image data with predetermined functions having different angular orientations.
Claim 26
Dependent Claim
25 words
The method as claimed in claim 20, wherein said step b) is not responsive to finger-like features of said image that remain constant with time.
Claim 27
Dependent Claim
244 words
A data input terminal for a digital computer, said data input terminal comprising, in combination: a video display; a video camera mounted for scanning a region of space near said video display to provide a video signal representing an image of said region of space; an analog-to-digital converter receiving said video signal and providing digital samples of image data; and digital computing means for receiving said digital samples of image data and analyzing said image data to recognize when a pointing finger of a human operator is present in said image and to determine the location of the pointing finger in the image when the pointing finger is present; wherein said digital computing means includes means for convolving said image data to resolve the image data into blobs characteristic of a fist of the human operator and complexes characteristic of the pointing finger, and examining the blobs and the complexes for a combination f a complex extending from a blob to recognize the presence of the pointing finger, and wherein said means for convolving said image data includes means for convolving said image data with a first predefined function to determine blob responses, means for convolving said image data with a second predefined function to determine ganglia responses indicating edges of the pointing finger, means for convolving said ganglia responses with a third predefined function to determine simple responses, and means for combining the simple responses and the ganglia responses to determine complex responses.
Claim 28
Dependent Claim
244 words
A data input terminal for a digital computer, said data input terminal comprising, in combination: a video display; a video camera mounted for scanning a region of space near said video display to provide a video signal representing an image of said region of space; an analog-to-digital converter receiving said video signal and providing digital samples of image data; and digital computing means for receiving said digital samples of image data and analyzing said image data to recognize when a pointing finger of a human operator is present in said image and to determine the location of the pointing finger in the image when the pointing finger is present; wherein said digital computing means includes means for convolving said image data to resolve the image data into blobs characteristic of a fist of the human operator and complexes characteristic of the pointing finger, and examining the blobs and the complexes for a combination f a complex extending from a blob to recognize the presence of the pointing finger, and wherein said means for convolving said image data includes means for convolving said image data with a first predefined function to determine blob responses, means for convolving said image data with a second predefined function to determine ganglia responses indicating edges of the pointing finger, means for convolving said ganglia responses with a third predefined function to determine simple responses, and means for combining the simple responses and the ganglia responses to determine complex responses.
Claim 29
Dependent Claim
50 words
The data input terminal as claimed in claim 24, wherein said digital computing means includes means for computing responses for a plurality of combinations of complexes extending from blobs, and means for determining a location for the pointing finger from one of said plurality of combinations having a maximum response.
Claim 30
Dependent Claim
30 words
The data input terminal as claimed in claim 24, wherein said digital computing means further includes means for determining an angular orientation of the pointing finger from the image data.
Claim 31
Dependent Claim
39 words
The data input terminal as claimed in claim 26, wherein said means for determining an angular orientation of said finger includes means for computing an angle from convolutions of the image data with predetermined functions having different angular orientations.
Claim 32
Dependent Claim
49 words
The data input terminal as claimed in claim 24, wherein said digital computing means further comprises means for computing the location of said pointing finger by interpolating between positions of a complex of an occurrence of said pointing finger and a complex at a neighboring location in said image.
Claim 33
Dependent Claim
25 words
The data input terminal as claimed in claim 24, wherein said digital computing means further comprises means for suppressing complexes that remain constant with time.
References Cited (Backward Citations)
US3701095A
Cited
US4468694A
Cited
US5059959A
Cited
US4783833A
Cited
US4905296A
Cited
US4884225A
Cited
US5014327A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1995-12-01 | 1996-06-01 | 1996-12-01 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1999-12-01 | 2000-06-01 | 2000-12-01 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2003-12-01 | 2004-06-01 | 2004-12-01 | $7700.00 | Window Closed / Lapsed |