US6151403A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method for automatic detection of human eyes in digital images
Assignee: Monument Peak Ventures LLC
•
Filing: 1997-08-29
•
Granted: 2000-11-21
•
Expires: 2017-08-29
Abstract
Abstract A computer program product for locating first and second objects, each having substantially the same physical characteristics, and the ratio of the distance between the first and second objects and the size of each object is substantially invariant, the computer program product comprises: a computer readable storage medium having a computer program stored thereon for performing the steps of determining potential flesh regions in an image; determining valley regions in an image; performing template matching for determining a plurality of locations that give a desirable match of the object relative to the template; and performing verification for determining the likelihood of pairs of potential eye candidates at the locations determined in the performing template matching step.
Inventors
- 👤 Jiebo Luo
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-06 13:04
CPC Classifications
G; G06; G06V; G06V40/00; G06V40/10; G06V40/18
Claims Analysis
Total Claims: 55
Claim 1
Independent Claim
271 words
A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.
Claim 2
Dependent Claim
271 words
A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.
Claim 3
Dependent Claim
64 words
The computer program product as in claim 1 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.
Claim 4
Dependent Claim
44 words
The computer program product as in claim 1 further comprising forming a mask image for searching for the first and second human eye objects; the locations where a search is to be performed are determined by the valley regions within the determined flesh region.
Claim 5
Dependent Claim
30 words
The computer program product as in claim 1, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.
Claim 6
Dependent Claim
40 words
The computer program product as in claim 1, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.
Claim 7
Dependent Claim
30 words
The computer program product as in claim 1, wherein computing the symmetry includes computing the symmetry between first and second halves of a face window determined by the eye locations.
Claim 8
Dependent Claim
37 words
The computer program product as in claim 1, wherein computing the centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.
Claim 9
Dependent Claim
21 words
The computer program product as in claim 1 further comprising the step of normalizing the template for maximizing robustness of matching.
Claim 10
Dependent Claim
26 words
The computer program product as in claim 1 further comprising the step of creating an image block of a same size and orientation as the template.
Claim 11
Dependent Claim
22 words
The computer program product as in claim 1 further comprising the step of normalizing the image block for maximizing robustness of matching.
Claim 12
Dependent Claim
31 words
The computer program product as in claim 9 further comprising the step of extracting individual sub-zones in both the template and the image block, and performing cross-correlation for each corresponding sub-zone.
Claim 13
Dependent Claim
31 words
The computer product as in claim 11 further comprising the step of computing an overall cross-correlation score for the image block if the cross-correlation score for each sub-zone exceeds a threshold.
Claim 14
Dependent Claim
263 words
A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for symmetry includes computing the symmetry between first and second halves of a face window determined by the locations of the human eye objects.
Claim 15
Dependent Claim
263 words
A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for symmetry includes computing the symmetry between first and second halves of a face window determined by the locations of the human eye objects.
Claim 16
Dependent Claim
64 words
The computer program product as in claim 13 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.
Claim 17
Dependent Claim
44 words
The computer program product as in claim 13 further comprising forming a mask image for searching for the first and second human eye objects; the locations where a search is to be performed are determined by the valley regions within the determined flesh region.
Claim 18
Dependent Claim
42 words
The computer program product as in claim 13, wherein computing the orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.
Claim 19
Dependent Claim
30 words
The computer program product as in claim 13, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.
Claim 20
Dependent Claim
40 words
The computer program product as in claim 13, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.
Claim 21
Dependent Claim
37 words
The computer program product as in claim 13, wherein computing the centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.
Claim 22
Dependent Claim
21 words
The computer program product as in claim 13 further comprising the step of normalizing the template for maximizing robustness of matching.
Claim 23
Dependent Claim
26 words
The computer program product as in claim 13 further comprising the step of creating an image block of a same size and orientation as the template.
Claim 24
Dependent Claim
22 words
The computer program product as in claim 13 further comprising the step of normalizing the image block for maximizing robustness of matching.
Claim 25
Dependent Claim
31 words
The computer program product as in claim 21 further comprising the step of extracting individual sub-zones in both the template and the image block, and performing cross-correlation for each corresponding sub-zone.
Claim 26
Dependent Claim
32 words
The computer program product as in claim 23 further comprising the step of computing an overall cross-correlation score for the image block if the cross-correlation score for each sub-zone exceeds a threshold.
Claim 27
Dependent Claim
266 words
A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.
Claim 28
Dependent Claim
266 words
A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising: a computer readable storage medium having a computer program stored thereon for performing the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.
Claim 29
Dependent Claim
64 words
The computer program product as in claim 25 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.
Claim 30
Dependent Claim
44 words
The computer program product as in claim 25 further comprising forming a mask image for searching for the first and second human eye objects; the locations where a search is to be performed are determined by the valley regions within the determined flesh region.
Claim 31
Dependent Claim
42 words
The computer program product as in claim 25, wherein computing the orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.
Claim 32
Dependent Claim
30 words
The computer program product as in claim 25, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.
Claim 33
Dependent Claim
40 words
The computer program product as in claim 25, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.
Claim 34
Dependent Claim
30 words
The computer program product as in claim 25, wherein computing the symmetry includes computing the symmetry between first and second halves of a face window determined by the eye locations.
Claim 35
Dependent Claim
21 words
The computer program product as in claim 25 further comprising the step of normalizing the template for maximizing robustness of matching.
Claim 36
Dependent Claim
26 words
The computer program product as in claim 25 further comprising the step of creating an image block of a same size and orientation as the template.
Claim 37
Dependent Claim
22 words
The computer program product as in claim 25 further comprising the step of normalizing the image block for maximizing robustness of matching.
Claim 38
Dependent Claim
31 words
The computer program product as in claim 33 further comprising the step of extracting individual sub-zones in both the template and the image block, and performing cross-correlation for each corresponding sub-zone.
Claim 39
Dependent Claim
32 words
The computer program product as in claim 35 further comprising the step of computing an overall cross-correlation score for the image block if the cross-correlation score for each sub-zone exceeds a threshold.
Claim 40
Independent Claim
254 words
A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.
Claim 41
Independent Claim
254 words
A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.
Claim 42
Dependent Claim
62 words
The method as in claim 37 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.
Claim 43
Dependent Claim
42 words
The method as in claim 37 further comprising forming a mask image for searching for the first and second human eye objects; the locations where a search is to be performed are determined by the valley regions within the determined flesh region.
Claim 44
Dependent Claim
28 words
The method as in claim 37, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.
Claim 45
Dependent Claim
38 words
The method as in claim 37, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.
Claim 46
Dependent Claim
28 words
The method as in claim 37, wherein computing the symmetry includes computing the symmetry between first and second halves of a face window determined by the eye locations.
Claim 47
Dependent Claim
35 words
The method as in claim 37, wherein computing the centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.
Claim 48
Independent Claim
245 words
A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit symmetry includes computing the symmetry between first and second halves of a face window determined by the locations of the human eye objects.
Claim 49
Independent Claim
245 words
A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of: (a) determining a potential flesh region in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit symmetry includes computing the symmetry between first and second halves of a face window determined by the locations of the human eye objects.
Claim 50
Dependent Claim
61 words
The method as in claim 44 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.
Claim 51
Dependent Claim
41 words
The method in claim 44 further comprising forming a mask image for searching for the first and second human eye objects; the locations where a search is to be performed are determined by the valley regions within the determined flesh region.
Claim 52
Independent Claim
247 words
A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ration of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of: (a) determining a potential flesh regions in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit centrality includes computing a distance from a middle point between the first and second human objects to a major axis of the determined flesh region.
Claim 53
Independent Claim
247 words
A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ration of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of: (a) determining a potential flesh regions in an intensity image; (b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image; (c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and (d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit centrality includes computing a distance from a middle point between the first and second human objects to a major axis of the determined flesh region.
Claim 54
Dependent Claim
61 words
The method as in claim 47 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.
Claim 55
Dependent Claim
42 words
The method as in claim 47 further comprising forming a mask image for searching for the first and second human eye objects; the locations where a search is to be performed are determined by the valley regions within the determined flesh region.
References Cited (Backward Citations)
US5103484A
Cited
US5128711A
Cited
US5063603A
Cited
US5719951A
Cited
US5164992A
Cited
US5450504A
Cited
US5430809A
Cited
US5412738A
Cited
US5689575A
Cited
US5572596A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-11-21 | 2004-05-22 | 2004-11-21 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-11-21 | 2008-05-22 | 2008-11-21 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-11-21 | 2012-05-22 | 2012-11-21 | $7700.00 | Window Closed / Lapsed |