← Back to Patent Explorer
US6134339A Expired / Lapsed US ⏱️ 0.0 years remaining

Method and apparatus for determining the position of eyes and for correcting eye-defects in a captured frame

Assignee: Monument Peak Ventures LLC
•
Filing: 1998-09-17
•
Granted: 2000-10-17
•
Expires: 2018-09-17
📄 Original PDF 🌐 Google Patents 🔄 Sync Now

Abstract

Abstract The present apparatus and method captures at least a pair of images in relatively quick succession with an illumination source being fired during the capture of one of the images. The illumination source is located adjacent to the optical lens or close to the optical axis of the camera. Next the illumination level of the images is compensated to make them equal in illumination. Following that an analysis of the images is performed to determine the difference between the illumination compensated images. After the illumination compensation the difference has to be caused by eye-defects or specular reflections from glasses. The pixel coordinates of the transitory eye-defect regions are recorded and analyzed to determine the spacing of the potential eye locations and/or the spatial structure of the eyes for plausibility. Once confirmed the coordinates are output and recorded.

Inventors

  • 👤 Jiebo Luo

Bibliographic Details

Application Number —
Jurisdiction US
Data Source google_patents
Last Synchronized 2026-10-30 11:13
CPC Classifications
G; G06; G06V; G06V40/00; G06V40/10; G06V40/18; G06V40/193; G06T; G06T7/00; G06T7/97; G06T7/70; G06T7/73; G06T7/74; G06V40/19; G06T2207/00; G06T2207/30; G06T2207/30216

Claims Analysis

Total Claims: 46
Claim 1
Independent Claim 151 words
A method for determining the position of eyes within a frame of captured image data comprising the steps of: capturing two color frames of an image scene wherein one frame is captured with an ambient illumination and the other frame is captured with an additional illumination; storing said frames of said image scene in a memory; compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; and outputting the data of pixel pair coordinates wherein each coordinate corresponds to a center of each region of high intensity pixel value in said color channel within said captured image scene.
Claim 2
Independent Claim 151 words
A method for determining the position of eyes within a frame of captured image data comprising the steps of: capturing two color frames of an image scene wherein one frame is captured with an ambient illumination and the other frame is captured with an additional illumination; storing said frames of said image scene in a memory; compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; and outputting the data of pixel pair coordinates wherein each coordinate corresponds to a center of each region of high intensity pixel value in said color channel within said captured image scene.
Claim 3
Dependent Claim 76 words
The method according to claim 1, wherein said compensating step comprises: a) converting the value of each pixel in said captured frames into log metric; b) calculating the mean log intensity value of each of said converted images; c) adding the difference between said mean log intensity values to the converted image corresponding to the subtrahend of said difference; and d) converting said compensated log image to the inverse log metric to obtain a compensated frame.
Claim 4
Dependent Claim 34 words
The method according to claim 1, wherein said determined pairs of regions of high intensity pixel value, are processed in the red channel for human eyes, according to color, size, shape structure, and spacing.
Claim 5
Dependent Claim 34 words
The method according to claim 1, wherein said determined pairs of regions of high intensity pixel value, are processed in the green channel for animal eyes, according to color, size, shape structure, and spacing.
Claim 6
Dependent Claim 39 words
The method according to claim 1, wherein said determined pairs of regions of high intensity pixel value, preferably in the red and green channels for human and animal eyes, are processed according to color, size, shape structure, and spacing.
Claim 7
Dependent Claim 25 words
The method according to claim 1 further comprising the step of: analyzing the spacing of a potential eye location and the spatial structure for plausibility.
Claim 8
Dependent Claim 26 words
The method according to claim 1 wherein the outputting step comprises recording the pixel coordinates of the center of said regions of high intensity pixel value.
Claim 9
Dependent Claim 21 words
The method according to claim 1 wherein said frames of said image scene are consecutive pictures of a video image stream.
Claim 10
Independent Claim 190 words
An apparatus for determining the position of eyes within a frame of captured image data comprising: means for capturing at least two color frames of an image scene, said means is provided with an optical lens defining an optical axis; means for storing the data of at least two consecutive captured frames of an image scene; an illumination source for illuminating one of said captured frames of said image scene wherein said illumination source is positioned as close as possible to said optical axis; and electronic means: a) for compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; b) for determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; c) for scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; and d) for outputting the data of pixel pair coordinates wherein each coordinate corresponds to a center of each region of high intensity pixel value in said color channel within said captured image scene.
Claim 11
Independent Claim 190 words
An apparatus for determining the position of eyes within a frame of captured image data comprising: means for capturing at least two color frames of an image scene, said means is provided with an optical lens defining an optical axis; means for storing the data of at least two consecutive captured frames of an image scene; an illumination source for illuminating one of said captured frames of said image scene wherein said illumination source is positioned as close as possible to said optical axis; and electronic means: a) for compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; b) for determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; c) for scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; and d) for outputting the data of pixel pair coordinates wherein each coordinate corresponds to a center of each region of high intensity pixel value in said color channel within said captured image scene.
Claim 12
Dependent Claim 21 words
Apparatus according to claim 9 wherein the capturing means is a video camera for capturing consecutive frames of an image scene.
Claim 13
Dependent Claim 33 words
Apparatus according to claim 9 wherein the illumination source is a flash light emitting source for triggering direct illumination in synchronization with the capturing of the second image frame of said image scene.
Claim 14
Dependent Claim 26 words
Apparatus according to claim 9 wherein the electronic means is a computer to which a keyboard, a monitor, an external storage, and a printer are connected.
Claim 15
Independent Claim 169 words
A method, used in a digital camera, for locating eye-defect regions comprises the steps of: selecting a function of the digital camera for eye-defect free picture taking; capturing two color frames of an image scene with a minimum delay in between to minimize spatial registration error, wherein one frame is captured with an ambient illumination and the other frame is captured with an additional camera triggered illumination; storing data, representing said two captured frames in a memory; compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; and obtaining the data of pixel pair coordinates of said regions of bright intensity pixel value wherein each coordinate corresponds to a center of each of said eye-defect regions.
Claim 16
Independent Claim 169 words
A method, used in a digital camera, for locating eye-defect regions comprises the steps of: selecting a function of the digital camera for eye-defect free picture taking; capturing two color frames of an image scene with a minimum delay in between to minimize spatial registration error, wherein one frame is captured with an ambient illumination and the other frame is captured with an additional camera triggered illumination; storing data, representing said two captured frames in a memory; compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; and obtaining the data of pixel pair coordinates of said regions of bright intensity pixel value wherein each coordinate corresponds to a center of each of said eye-defect regions.
Claim 17
Dependent Claim 76 words
The method according to claim 13, wherein said compensating step comprises: a) converting the value of each pixel in said captured frames into log metric; b) calculating the mean log intensity value of each of said converted images; c) adding the difference between said mean log intensity values to the converted image corresponding to the subtrahend of said difference; and d) converting said compensated log image to the inverse log metric to obtain a compensated frame.
Claim 18
Dependent Claim 25 words
The method according to claim 13 further comprising the step of: analyzing the spacing of a potential eye-defect region and the spatial structure for plausibility.
Claim 19
Dependent Claim 22 words
The method according to claim 13 wherein the outputting step comprises recording the pixel coordinates of the center of said eye-defect regions.
Claim 20
Dependent Claim 34 words
The method according to claim 15, wherein said determined eye- defect regions of high intensity pixel value, are processed in the red channel for human eyes, according to color, size, shape structure, and spacing.
Claim 21
Dependent Claim 34 words
The method according to claim 15, wherein said determined eye- defect regions of high intensity pixel value, are processed in the green channel for animal eyes, according to color, size, shape structure, and spacing.
Claim 22
Dependent Claim 39 words
The method according to claim 15, wherein said determined eye- defect regions of high intensity pixel value, preferably in the red and green channels for human and animal eyes, are processed according to color, size, shape structure, and spacing.
Claim 23
Dependent Claim 24 words
Method according to claim 13 wherein based on the user selection the digital camera performs an automatic eye-defect correction of the captured image scene.
Claim 24
Dependent Claim 40 words
Method according to claim 13 wherein, based on the user selection, the digital camera stores the data of the frame bearing said camera triggered illumination along with the obtained spatial data of the centers of each of said eye-defect regions.
Claim 25
Dependent Claim 169 words
A digital camera capable of locating eye-defect regions comprising: a user input control for initiating the eye-defect correction function of the digital camera; means for capturing two color frames of an image scene with a minimum delay between to minimize spatial registration error, wherein said capturing means is provided with an optical lens defining an optical axis; an illumination source for illuminating one of the two frames of said image scene wherein said illumination source is positioned as close as possible to said optical axis; means for storing the data of said two captured frames of said image scene; and electronic means: for compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; for determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; and for scanning said compensated color difference image and thereby determining pairs of regions of eye-defect, and for calculating spatial coordinates of said eye-defects.
Claim 26
Dependent Claim 169 words
A digital camera capable of locating eye-defect regions comprising: a user input control for initiating the eye-defect correction function of the digital camera; means for capturing two color frames of an image scene with a minimum delay between to minimize spatial registration error, wherein said capturing means is provided with an optical lens defining an optical axis; an illumination source for illuminating one of the two frames of said image scene wherein said illumination source is positioned as close as possible to said optical axis; means for storing the data of said two captured frames of said image scene; and electronic means: for compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; for determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; and for scanning said compensated color difference image and thereby determining pairs of regions of eye-defect, and for calculating spatial coordinates of said eye-defects.
Claim 27
Dependent Claim 25 words
The digital camera according to claim 22 wherein said means for storing is used to store the captured image having the eye-defect correction automatically performed.
Claim 28
Dependent Claim 44 words
The digital camera according to claim 22 wherein said means for storing is used to store the data of the frame bearing said camera triggered illumination along with the calculated data of the spatial coordinates of the centers of each of said eye-defect regions.
Claim 29
Dependent Claim 15 words
Apparatus according to claim 22 wherein the illumination source is a flash light emitting source.
Claim 30
Dependent Claim 27 words
Apparatus according to claim 22 wherein the electronic means is a built-in central processing unit which controls the functions of the digital camera and an attached memory.
Claim 31
Independent Claim 215 words
A method, used in a photographic film camera, for locating eye-defect regions comprising the steps of: selecting a function of the photographic film camera for eye-defect free picture taking; exposing the photographic film with a frame of an image scene bearing the camera triggered illumination and simultaneously storing data of said frame bearing the camera triggered illumination in a memory; capturing a further frame of said image scene with a minimum delay in between to minimize spatial registration error, wherein said frame is captured with an ambient illumination, and storing data representing said frame in said memory; compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; obtaining data of pixel pair coordinates corresponding to the pairs of regions of high intensity pixel value wherein each coordinate corresponds to a center of each of said eye-defect regions; and recording on the photographic film inside the camera coordinates of said pairs of regions of high intensity pixel value preferably in the red channel.
Claim 32
Independent Claim 215 words
A method, used in a photographic film camera, for locating eye-defect regions comprising the steps of: selecting a function of the photographic film camera for eye-defect free picture taking; exposing the photographic film with a frame of an image scene bearing the camera triggered illumination and simultaneously storing data of said frame bearing the camera triggered illumination in a memory; capturing a further frame of said image scene with a minimum delay in between to minimize spatial registration error, wherein said frame is captured with an ambient illumination, and storing data representing said frame in said memory; compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; scanning said compensated color difference image and determining pairs of regions of high intensity pixel value in at least one color channel; obtaining data of pixel pair coordinates corresponding to the pairs of regions of high intensity pixel value wherein each coordinate corresponds to a center of each of said eye-defect regions; and recording on the photographic film inside the camera coordinates of said pairs of regions of high intensity pixel value preferably in the red channel.
Claim 33
Dependent Claim 72 words
The method according to claim 27, wherein said compensating step comprises: converting the value of each pixel in said captured frames into log metric; calculating the mean log intensity value of each of said converted images; adding the difference between said mean log intensity values to the converted image corresponding to the subtrahend of said difference; and converting said compensated log image to the inverse log metric to obtain a compensated frame.
Claim 34
Dependent Claim 25 words
The method according to claim 27 further comprising the step of analyzing the spacing of a potential eye-defect region and the spatial structure for plausibility.
Claim 35
Dependent Claim 33 words
The method according to claim 29, wherein said determined eye-defect regions of high intensity pixel value, are processed in the red channel for human eyes, according to color, size, shape structure, and spacing.
Claim 36
Dependent Claim 33 words
The method according to claim 29, wherein said determined eye-defect regions of high intensity pixel value, are processed in the green channel for animal eyes, according to color, size, shape structure, and spacing.
Claim 37
Dependent Claim 38 words
The method according to claim 29, wherein said determined eye-defect regions of high intensity pixel value, preferably in the red and green channels for human and animal eyes, are processed according to color, size, shape structure, and spacing.
Claim 38
Dependent Claim 24 words
Method according to claim 27 wherein based on the user selection, the digital camera performs an automatic eye-defect correction of the captured image scene.
Claim 39
Dependent Claim 40 words
Method according to claim 27 wherein based on the user selection the digital camera stores the data of the frame bearing said camera triggered illumination along with the obtained spatial data of the centers of each of said eye-defect regions.
Claim 40
Dependent Claim 28 words
The method according to claim 27, wherein said film includes a magnetic layer and, wherein the recording on the photographic film is magnetic recording on said magnetic layer.
Claim 41
Dependent Claim 214 words
A photographic film camera capable of locating eye-defects comprises: a user input control for initiating the eye-defect free function of the film camera; means for capturing two frames of an image scene with a minimum delay between to minimize spatial registration error, wherein said capturing means is provided with an optical lens defining an optical axis; an illumination source for illuminating one of said two frames of said image scene, wherein said illumination source is positioned as close as possible to said optical axis; a photographic film for capturing said illuminated frame of said image scene, said film including a recording layer; means for temporarily storing data of said illuminated frame and data of said frame bearing ambient illumination of said image scene; electronic means for compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; for determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; for scanning said compensated color difference image and thereby determining pairs of regions of eye-defect, and calculating spatial coordinates of said eye-defects; and recording means for writing the spatial coordinates of the eye-defects in said recording layer on said photographic film inside said photographic film camera.
Claim 42
Dependent Claim 214 words
A photographic film camera capable of locating eye-defects comprises: a user input control for initiating the eye-defect free function of the film camera; means for capturing two frames of an image scene with a minimum delay between to minimize spatial registration error, wherein said capturing means is provided with an optical lens defining an optical axis; an illumination source for illuminating one of said two frames of said image scene, wherein said illumination source is positioned as close as possible to said optical axis; a photographic film for capturing said illuminated frame of said image scene, said film including a recording layer; means for temporarily storing data of said illuminated frame and data of said frame bearing ambient illumination of said image scene; electronic means for compensating said frames for the overall illumination change between said two captured frames to obtain a compensated frame; for determining the difference between said compensated frame and an uncompensated frame of said two captured frames to obtain a compensated color difference image; for scanning said compensated color difference image and thereby determining pairs of regions of eye-defect, and calculating spatial coordinates of said eye-defects; and recording means for writing the spatial coordinates of the eye-defects in said recording layer on said photographic film inside said photographic film camera.
Claim 43
Dependent Claim 51 words
The photographic film camera according to claim 36 wherein the capturing means comprise a semi transparent mirror located in said optical axis in order to simultaneously illuminate said photographic film and an image sensor, said illumination of said photographic film is controlled by a shutter, positioned prior to said photographic film.
Claim 44
Dependent Claim 37 words
The photographic film camera according to claim 36 wherein the recording layer on said photographic film is a magnetic recording layer and the recording means are designed for magnetic recording of data on said magnetic recording layer.
Claim 45
Dependent Claim 18 words
The photographic film camera according to claim 36 wherein the illumination source is a flash light emitting source.
Claim 46
Dependent Claim 34 words
The photographic film camera according to claim 36 wherein said electronic means is a camera built-in central processing unit which controls the main camera functions, a memory for temporary storing and said recording means.

References Cited (Backward Citations)

US5231674A Cited
US5570434A Cited
US5325133A Cited
US5680481A Cited
US5432863A Cited
US5850466A Cited
US5790234A Cited
US5990973A Cited
US6016354A Cited
US6064752A Cited
US5978494A Cited

USPTO Annuity & Maintenance Windows

Annuity Stage Window Opens Payment Deadline Surcharge Period Until Estimated Fee Status
3.5 Years (Year 4) 2003-10-17 2004-04-17 2004-10-17 $2000.00 Window Closed / Lapsed
7.5 Years (Year 8) 2007-10-17 2008-04-17 2008-10-17 $3760.00 Window Closed / Lapsed
11.5 Years (Year 12) 2011-10-17 2012-04-17 2012-10-17 $7700.00 Window Closed / Lapsed

Assigned Categories & Tags

No categories assigned to this patent yet.

Assign Category Tag