Patent US7116820B2 successfully refreshed from Google Patents.
US7116820B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Detecting and correcting red-eye in a digital image
Assignee: Hewlett Packard Development Co LP
•
Filing: 2003-04-28
•
Granted: 2006-10-03
•
Expires: 2023-04-28
Abstract
Abstract A preliminary set of candidate red-eye pixel areas is identified based on computed pixel redness measures. Candidate red-eye pixel areas having computed redness contrasts relative to respective neighboring pixel areas less than a prescribed redness contrast threshold are filtered from the preliminary set. Candidate red-eye pixel areas located in areas of the digital image having computed grayscale contrasts relative to respective neighboring pixel areas less than a prescribed grayscale contrast threshold also are filtered from the preliminary set. In another aspect, a final pixel mask identifying red-eye pixels and non-red-eye pixels in the candidate red-eye area is generated. Pixels in the candidate red-eye pixel area identified as red-eye pixels in the final pixel mask are desaturated from their original color values by respective scaling factors that vary depending on luminance of the pixels in the candidate red-eye pixel area and proximity of the pixels to boundaries between red-eye pixels and non-red-eye pixels.
Inventors
- 👤 Huitao Luo
- 👤 Jonathan Yen
- 👤 Daniel Tretter
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:03
CPC Classifications
H; H04; H04N; H04N1/00; H04N1/46; H04N1/56; H04N1/60; H04N1/62; G; G06; G06V; G06V40/00; G06V40/10; G06V40/18; G06V40/193; H04N1/624
Claims Analysis
Total Claims: 64
Claim 1
Independent Claim
110 words
A method of processing a digital image containing pixels with original color values, comprising: computing measures of pixel redness in the digital image; identifying a preliminary set of candidate red-eye pixel areas based on the measures of pixel redness; filtering from the preliminary set each candidate red-eye pixel area having a computed redness contrast relative to at least one respective neighboring pixel area less than a prescribed redness contrast threshold; and filtering from the preliminary set each candidate red-eye pixel area located in an area of the digital image having a computed grayscale contrast relative to at least one respective neighboring pixel area less than a prescribed grayscale contrast threshold.
Claim 2
Independent Claim
110 words
A method of processing a digital image containing pixels with original color values, comprising: computing measures of pixel redness in the digital image; identifying a preliminary set of candidate red-eye pixel areas based on the measures of pixel redness; filtering from the preliminary set each candidate red-eye pixel area having a computed redness contrast relative to at least one respective neighboring pixel area less than a prescribed redness contrast threshold; and filtering from the preliminary set each candidate red-eye pixel area located in an area of the digital image having a computed grayscale contrast relative to at least one respective neighboring pixel area less than a prescribed grayscale contrast threshold.
Claim 3
Dependent Claim
33 words
The method of claim 1 , wherein pixel redness measures are computed based on a ratio of a measure of a red component of pixel energy to a measure of total pixel energy.
Claim 4
Dependent Claim
69 words
The method of claim 2 , wherein pixel redness measures (R) are computed based on the following ratio: R = α · r + β · g + γ · b r + g + b + d where r, g, and b are red, green, and blue component pixel values of the digital image, respectively, α, β and γ are weighting factors, and d is a prescribed constant.
Claim 5
Dependent Claim
55 words
The method of claim 1 , wherein identifying the preliminary set of candidate red-eye pixel areas comprises applying a two-dimensional redness filter to the computed pixel redness measures, wherein the redness filter is operable to compute a redness score based on a central kernel pixel area and a pixel area surrounding the kernel pixel area.
Claim 6
Dependent Claim
18 words
The method of claim 4 , wherein the kernel pixel area and the surrounding pixel area are concentric.
Claim 7
Dependent Claim
48 words
The method of claim 4 , wherein the kernel pixel area corresponds to a square pixel area of side length dl and the surrounding pixel area corresponds to a rectangular path defined between square pixel area of side length d2 and the central kernel pixel area, wherein d2<d2.
Claim 8
Dependent Claim
39 words
The method of claim 4 , wherein the computed redness score includes a component measuring average redness of the kernel pixel area and a component measuring average contrast of the kernel pixel area relative to the surrounding pixel area.
Claim 9
Dependent Claim
18 words
The method of claim 4 , wherein redness scores are computed for multiple different kernel pixel area sizes.
Claim 10
Dependent Claim
23 words
The method of claim 4 , further comprising applying a prescribed threshold to the measures of pixel redness to identify candidate red-eye pixels.
Claim 11
Dependent Claim
23 words
The method of claim 9 , wherein computing the preliminary set of candidate red-eye pixel areas comprises segmenting candidate red-eye pixels into groups.
Claim 12
Dependent Claim
54 words
The method of claim 10 , further comprising computing a measure of average pixel redness for each segmented pixel group, and maximizing each segmented pixel group to a candidate red-eye pixel area having an average pixel redness measure at least as large as the average pixel redness measure computed for the segmented pixel group.
Claim 13
Dependent Claim
24 words
The method of claim 1 , further comprising filtering candidate red-eye pixel areas from the preliminary set based on candidate red-eye pixel area shape.
Claim 14
Dependent Claim
41 words
The method of claim 1 , wherein each candidate red-eye pixel area having a computed redness contrast relative to each of a set of corresponding surrounding pixel areas less than the prescribed redness contrast threshold is filtered from the preliminary set.
Claim 15
Dependent Claim
22 words
The method of claim 13 , wherein each surrounding pixel area is equal in size to a corresponding candidate red-eye pixel area.
Claim 16
Dependent Claim
57 words
The method of claim 1 , further comprising computing measures of pixel grayscale in the digital image and computing for a given candidate red-eye pixel area a candidate eyeball area centered at the given candidate red-eye pixel area and having a size maximizing grayscale contrast between the candidate eyeball area and areas surrounding the candidate eyeball area.
Claim 17
Dependent Claim
52 words
The method of claim 15 , further comprising computing a measure of grayscale contrast between the candidate eyeball area and at least a portion of the areas surrounding the candidate eyeball area and applying a threshold to the measure of grayscale contrast to filter candidate red-eye pixel areas from the preliminary set.
Claim 18
Dependent Claim
24 words
The method of claim 1 , further comprising pairing candidate red-eye pixel areas, and filtering unpaired candidate red-eye pixel areas from the preliminary set.
Claim 19
Dependent Claim
27 words
The method of claim 1 , further comprising filtering from the preliminary set candidate red-eye pixel areas merged with adjacent skin tone regions in the digital image.
Claim 20
Dependent Claim
44 words
The method of claim 1 , further comprising filtering candidate red-eye pixel areas from the preliminary set based on proportions of non-skin tone pixels in the candidate red-eye pixel areas and proportions of skin tone pixels in areas surrounding the candidate red-eye pixel areas.
Claim 21
Dependent Claim
37 words
The method of claim 1 , further comprising mapping digital image pixels in candidate red-eye pixel areas into YCbCr color space, and applying thresholds to mapped YCbCr color space pixel values to filter candidate red-eye pixel areas.
Claim 22
Dependent Claim
42 words
The method of claim 1 , further comprising computing a ratio of YCbCr color space chrominance components for each pixel in image areas respectively encompassing candidate red-eye pixels areas and applying a threshold to the computed ratios to identify candidate red-eye regions.
Claim 23
Dependent Claim
23 words
The method of claim 21 , further comprising identifying in the candidate red-eye regions final red-eye pixel regions corresponding to isolated pixel islands.
Claim 24
Dependent Claim
14 words
The method of claim 1 , further comprising correcting red-eye in the digital image.
Claim 25
Dependent Claim
52 words
The method of claim 23 , wherein correcting red-eye comprises classifying pixels in each candidate red-eye pixel area based on redness and skin tone coloration and, for each candidate red-eye pixel area, generating an initial pixel mask identifying all pixels in the candidate red-eye pixel area classified as red and non-skin tone.
Claim 26
Dependent Claim
26 words
The method of claim 24 , further comprising modifying an initial pixel mask to achieve a mask size and location matching a corresponding red-eye pixel area.
Claim 27
Dependent Claim
31 words
The method of claim 25 , further comprising generating a final pixel mask by increasing the modified pixel mask in size to accommodate pink areas surrounding the corresponding red-eye pixel area.
Claim 28
Dependent Claim
25 words
The method of claim 26 , further comprising correcting original color values of pixels corresponding to the final pixel mask by desaturating original color values.
Claim 29
Dependent Claim
33 words
The method of claim 27 , wherein original color values identified as red in the final pixel mask are desaturated by respective amounts that vary with pixel location in the final pixel mask.
Claim 30
Dependent Claim
25 words
The method of claim 27 , wherein original color values of a given pixel are desaturated based on a luminance level of the given pixel.
Claim 31
Dependent Claim
31 words
The method of claim 27 , wherein original color values of a given pixel are desaturated based on a number of pixels in the final pixel mask neighboring the given pixel.
Claim 32
Dependent Claim
26 words
The method of claim 27 , further comprising correcting original color values of pixels corresponding to the final pixel mask by darkening the original color values.
Claim 33
Dependent Claim
28 words
The method of claim 31 , wherein original color values of a given pixel are darkened by an amount decreasing with a luminance level of the given pixel.
Claim 34
Independent Claim
111 words
A method of processing a digital image containing pixels with original color values, comprising: detecting a candidate red-eye pixel area in the digital image; generating a final pixel mask identifying red-eye pixels and non-red-eye pixels in the candidate red-eye pixel area; and desaturating pixels in the candidate red-eye pixel area identified as red-eye pixels in the final pixel mask from their original color values by respective scaling factors that vary depending on luminance of the identified red-eye pixels in the candidate red-eye pixel area and proximity of the identified red-eve pixels in the candidate red-eve pixel area to boundaries between red-eye pixels and non-red-eye pixels identified in the final pixel mask.
Claim 35
Independent Claim
111 words
A method of processing a digital image containing pixels with original color values, comprising: detecting a candidate red-eye pixel area in the digital image; generating a final pixel mask identifying red-eye pixels and non-red-eye pixels in the candidate red-eye pixel area; and desaturating pixels in the candidate red-eye pixel area identified as red-eye pixels in the final pixel mask from their original color values by respective scaling factors that vary depending on luminance of the identified red-eye pixels in the candidate red-eye pixel area and proximity of the identified red-eve pixels in the candidate red-eve pixel area to boundaries between red-eye pixels and non-red-eye pixels identified in the final pixel mask.
Claim 36
Dependent Claim
77 words
The method of claim 33 , further comprising: classifying pixels in the candidate red-eye pixel area as one of red and non-red; classifying pixels in the candidate red-eye pixel area as one of skin tone and non-skin tone; generating an initial pixel mask identifying all pixels in the candidate red-eye pixel area classified as red and non-skin tone; and modifying the initial pixel mask to achieve a mask size and location matching the candidate red-eye pixel area.
Claim 37
Dependent Claim
31 words
The method of claim 34 , further comprising generating the final pixel mask by increasing the modified pixel mask in size to accommodate pink areas surrounding the candidate red-eye pixel area.
Claim 38
Dependent Claim
63 words
The method of claim 33 , wherein each red-eye pixel in the final pixel mask has a respective number of neighboring red-eye pixels, and the desaturating comprises desaturating pixels in the candidate red-eye pixel area identified as red-eye pixels in the final pixel mask from their original color values by respective scale factors that vary depending their respective numbers of neighboring red-eye pixels.
Claim 39
Dependent Claim
28 words
The method of claim 33 , wherein original color values of a given pixel are darkened by an amount increasing with a darkness level of the given pixel.
Claim 40
Independent Claim
116 words
A system for processing a digital image containing pixels with original color values, comprising a global screening module operable to: compute measures of pixel redness in the digital image; identify a preliminary set of candidate red-eye pixel areas based on the measures of pixel redness; filter from the preliminary set each candidate red-eye pixel area having a computed redness contrast relative to at least one respective neighboring pixel area less than a prescribed redness contrast threshold; and filter from the preliminary set each candidate red-eye pixel area located in an area of the digital image having a computed grayscale contrast relative to at least one respective neighboring pixel area less than a prescribed grayscale contrast threshold.
Claim 41
Independent Claim
116 words
A system for processing a digital image containing pixels with original color values, comprising a global screening module operable to: compute measures of pixel redness in the digital image; identify a preliminary set of candidate red-eye pixel areas based on the measures of pixel redness; filter from the preliminary set each candidate red-eye pixel area having a computed redness contrast relative to at least one respective neighboring pixel area less than a prescribed redness contrast threshold; and filter from the preliminary set each candidate red-eye pixel area located in an area of the digital image having a computed grayscale contrast relative to at least one respective neighboring pixel area less than a prescribed grayscale contrast threshold.
Claim 42
Dependent Claim
33 words
The system of claim 38 , wherein pixel redness measures are computed based on a ratio of a measure of a red component of pixel energy to a measure of total pixel energy.
Claim 43
Dependent Claim
55 words
The system of claim 38 , wherein identifying the preliminary set of candidate red-eye pixel areas comprises applying a two-dimensional redness filter to the computed pixel redness measures, wherein the redness filter is operable to compute a redness score based on a central kernel pixel area and a pixel area surrounding the kernel pixel area.
Claim 44
Dependent Claim
30 words
The system of claim 38 , wherein the global screening module is operable to filter candidate red-eye pixel areas from the preliminary set based on candidate red-eye pixel area shape.
Claim 45
Dependent Claim
41 words
The system of claim 38 , wherein each candidate red-eye pixel area having a computed redness contrast relative to each of a set of corresponding surrounding pixel areas less than the prescribed redness contrast threshold is filtered from the preliminary set.
Claim 46
Dependent Claim
63 words
The system of claim 38 , wherein the global screening module is operable to compute measures of pixel grayscale in the digital image and computing for a given candidate red-eye pixel area a candidate eyeball area centered at the given candidate red-eye pixel area and having a size maximizing grayscale contrast between the candidate eyeball area and areas surrounding the candidate eyeball area.
Claim 47
Dependent Claim
30 words
The system of claim 38 , further comprising a local verification module operable to pair candidate red-eye pixel areas and filter unpaired candidate red-eye pixel areas from the preliminary set.
Claim 48
Dependent Claim
33 words
The system of claim 38 , further comprising a local verification module operable to filter from the preliminary set candidate red-eye pixel areas merged with adjacent skin tone regions in the digital image.
Claim 49
Dependent Claim
50 words
The system of claim 38 , further comprising a local verification module operable to filter candidate red-eye pixel areas from the preliminary set based on proportions of non-skin tone pixels in the candidate red-eye pixel areas and proportions of skin tone pixels in areas surrounding the candidate red-eye pixel areas.
Claim 50
Dependent Claim
43 words
The system of claim 38 , further comprising a local verification module operable to map digital image pixels in candidate red-eye pixel areas into YCbCr color space, and applying thresholds to mapped YCbCr color space pixel values to filter candidate red-eye pixel areas.
Claim 51
Dependent Claim
48 words
The system of claim 38 , further comprising a local verification module operable to compute a ratio of YCbCr color space chrominance components for each pixel in image areas respectively encompassing candidate red-eye pixels areas and applying a threshold to the computed ratios to identify candidate red-eye regions.
Claim 52
Dependent Claim
29 words
The system of claim 48 , wherein the local verification module is operable to identify in the candidate red-eye regions final red-eye pixel regions corresponding to isolated pixel islands.
Claim 53
Dependent Claim
56 words
The system of claim 38 , further comprising a red-eye segmentation module operable to classify pixels in each candidate red-eye pixel area based on redness and skin tone coloration and, for each candidate red-eye pixel area, generate an initial pixel mask identifying all pixels in the candidate red-eye pixel area classified as red and non-skin tone.
Claim 54
Dependent Claim
31 words
The method of claim 50 , wherein the red-eye segmentation module operable to modify an initial pixel mask to achieve a mask size and location matching a corresponding red-eye pixel area.
Claim 55
Dependent Claim
36 words
The method of claim 51 , wherein the red-eye segmentation module operable to generate a final pixel mask by increasing the modified pixel mask in size to accommodate pink areas surrounding the corresponding red-eye pixel area.
Claim 56
Dependent Claim
20 words
The system of claim 38 , further comprising a red-eye correction module operable to correct red-eye in the digital image.
Claim 57
Dependent Claim
30 words
The system of claim 53 , wherein the red-eye correction module operable to correct original color values of pixels corresponding to the final pixel mask by desaturating original color values.
Claim 58
Dependent Claim
25 words
The system of claim 54 , wherein original color values of a given pixel are desaturated based on a luminance level of the given pixel.
Claim 59
Dependent Claim
35 words
The system of claim 54 , wherein original color values of pixels identified as red in the final pixel mask are desaturated by respective amounts that vary with pixel location in the final pixel mask.
Claim 60
Dependent Claim
31 words
The system of claim 54 , wherein original color values of a given pixel are desaturated based on a number of pixels in the final pixel mask neighboring the given pixel.
Claim 61
Dependent Claim
31 words
The system of claim 54 , wherein the red-eye correction module operable to correct original color values of pixels corresponding to the final pixel mask by darkening the original color values.
Claim 62
Dependent Claim
28 words
The system of claim 58 , wherein original color values of a given pixel are darkened by an amount decreasing with a luminance level of the given pixel.
Claim 63
Independent Claim
129 words
A system for processing a digital image containing pixels with original color values, comprising: a red-eye detection module operable to detect a candidate red-eye pixel area in the digital image; a red-eye segmentation module operable to generate a final pixel mask identifying red-eye pixels and non-red-eye pixels in the candidate red-eye pixel area; and a red-eye correction module operable to desaturate pixels in the candidate red-eye pixel area identified as red-eye pixels in the final pixel mask from their original color values by respective scaling factors that vary depending on luminance of the identified red-eye pixels in the candidate red-eye pixel area and proximity of the identified red-eye pixels in the candidate red-eye pixel area to boundaries between red-eye pixels and non-red-eye pixels identified in the final pixel mask.
Claim 64
Independent Claim
129 words
A system for processing a digital image containing pixels with original color values, comprising: a red-eye detection module operable to detect a candidate red-eye pixel area in the digital image; a red-eye segmentation module operable to generate a final pixel mask identifying red-eye pixels and non-red-eye pixels in the candidate red-eye pixel area; and a red-eye correction module operable to desaturate pixels in the candidate red-eye pixel area identified as red-eye pixels in the final pixel mask from their original color values by respective scaling factors that vary depending on luminance of the identified red-eye pixels in the candidate red-eye pixel area and proximity of the identified red-eye pixels in the candidate red-eye pixel area to boundaries between red-eye pixels and non-red-eye pixels identified in the final pixel mask.
References Cited (Backward Citations)
US5432863A
Cited
US6307954B1
Cited
EP0884694A1
Cited
US6009209A
Cited
US6252976B1
Cited
US6292574B1
Cited
WO1999017254A1
Cited
US6016354A
Cited
WO2001071421A1
Cited
US6728401B1
Cited
US20020136450A1
Cited
US20020176623A1
Cited
US6873743B2
Cited
US20030044042A1
Cited
US20030007687A1
Cited
US20040184670A1
Cited
US20030202105A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-10-02 | 2010-04-03 | 2010-10-03 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-10-02 | 2014-04-03 | 2014-10-03 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-10-02 | 2018-04-03 | 2018-10-03 | $7700.00 | Window Closed / Lapsed |