US6301370B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Face recognition from video images
Assignee: Google LLC
•
Filing: 1998-12-04
•
Granted: 2001-10-09
•
Expires: 2018-12-04
Abstract
Abstract The present invention is embodied in an apparatus, and related method, for detecting and recognizing an object in an image frame. The object may be, for example, a head having particular facial characteristics. The object detection process uses robust and computationally efficient techniques. The object identification and recognition process uses an image processing technique based on model graphs and bunch graphs that efficiently represent image features as jets. The jets are composed of wavelet transforms and are processed at nodes or landmark locations on an image corresponding to readily identifiable features. The system of the invention is particularly advantageous for recognizing a person over a wide variety of pose angles.
Inventors
- 👤 Johannes Bernhard Steffens
- 👤 Egor Valerievich Elagin
- 👤 Luciano Pasquale Agostino Nocera
- 👤 Thomas Maurer
- 👤 Hartmut Neven
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-07 20:01
CPC Classifications
G; G06; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/161; G06V40/165; G06T; G06T7/00; G06T7/20; G06T7/262; G06T7/246; H; H04; H04N; H04N13/00; H04N13/20; H04N13/204; H04N13/239; G06T2207/00; G06T2207/10; G06T2207/10004; G06T2207/10012; H04N13/246; H04N13/30; H04N13/302; H04N13/366; H04N2013/0074; H04N2013/0081
Claims Analysis
Total Claims: 46
Claim 1
Independent Claim
144 words
A process for recognizing objects in an image frame, comprising steps for: detecting an object in the image frame and bounding a portion of the image frame associated with the object resulting in a bound portion of the image frame that is associated with the object and an unbound portion of the image frame that is not associated with the object; transforming only the bound portion and not the unbound portion of the image frame using a wavelet transformation to generate a transformed image; locating, on the transformed image, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed image.
Claim 2
Dependent Claim
144 words
A process for recognizing objects in an image frame, comprising steps for: detecting an object in the image frame and bounding a portion of the image frame associated with the object resulting in a bound portion of the image frame that is associated with the object and an unbound portion of the image frame that is not associated with the object; transforming only the bound portion and not the unbound portion of the image frame using a wavelet transformation to generate a transformed image; locating, on the transformed image, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed image.
Claim 3
Dependent Claim
41 words
A process for recognizing objects as defined in claim 1 , further comprising sizing and centering the detected object within the bound portion of the image such that the detected object has a predetermined size and location within the bound portion.
Claim 4
Dependent Claim
39 words
A process for recognizing objects as defined in claim 1 , further comprising a step for suppressing a background portion, of the bound portion of the image frame, that is not representing the object, prior to identifying the object.
Claim 5
Dependent Claim
32 words
A process for recognizing objects as defined in claim 3 , wherein the suppressed background portion is gradually suppressed near edges of the object in the bound portion of the image frame.
Claim 6
Dependent Claim
24 words
A process for recognizing objects as defined in claim 1 , wherein the object is a head of a person exhibiting a facial region.
Claim 7
Dependent Claim
24 words
A process for recognizing objects as defined in claim 1 , wherein the bunch graph is based on a three-dimensional representation of the object.
Claim 8
Dependent Claim
29 words
A process for recognizing objects as defined in claim 1 , wherein the wavelet transformation is performed using phase calculations that are performed using a hardware adapted phase representation.
Claim 9
Dependent Claim
21 words
A process for recognizing objects as defined in claims 1 , wherein the locating step is performed using a coarse-to-fine approach.
Claim 10
Dependent Claim
20 words
A process for recognizing objects as defined in claim 1 , wherein the bunch graph is based on predetermined poses.
Claim 11
Dependent Claim
23 words
A process for recognizing objects as defined in claim 1 , wherein in the identifying step uses a three-dimensional representation of the object.
Claim 12
Dependent Claim
24 words
A process for recognizing objects as defined in claim 1 , wherein the bound portion covers less than ten percent of the image frame.
Claim 13
Dependent Claim
26 words
A process for recognizing objects as defined in claim 1 , wherein the step for detecting the object includes detecting a color associated with the object.
Claim 14
Dependent Claim
108 words
A process for recognizing objects in a sequence of image frames, comprising: detecting an object in the image frames and bounding a portion of each image frame associated with the object; transforming the bound portion of each image frame using a wavelet transformation to generate a transformed image; locating, on the transformed images, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed images.
Claim 15
Dependent Claim
108 words
A process for recognizing objects in a sequence of image frames, comprising: detecting an object in the image frames and bounding a portion of each image frame associated with the object; transforming the bound portion of each image frame using a wavelet transformation to generate a transformed image; locating, on the transformed images, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed images.
Claim 16
Dependent Claim
34 words
A process for recognizing objects as defined in claim 13 , wherein the step of detecting an object further comprises tracking the object between image frames based on a trajectory associated with the object.
Claim 17
Dependent Claim
36 words
A process for recognizing objects as defined in claim 13 , further comprising a preselecting process that chooses a most suitable view of an object out of sequence of views that belong to a particular trajectory.
Claim 18
Dependent Claim
25 words
A process for recognizing objects as defined in claim 13 , wherein the step of locating the nodes includes tracking the nodes between image frames.
Claim 19
Dependent Claim
30 words
A process for recognizing objects as defined in claim 16 , further comprising reinitializing a tracked node if the node's position deviates beyond a predetermined position constraint between image frames.
Claim 20
Dependent Claim
31 words
A process for recognizing objects as defined in claim 17 , wherein the predetermined position constraint is based on a geometrical position constraint associated with relative positions between the node locations.
Claim 21
Dependent Claim
36 words
A process for recognizing objects as defined in claim 13 , wherein the image frames are stereo images and the step of detecting includes generating a disparity histogram and a silhouette image to detect the object.
Claim 22
Dependent Claim
35 words
A process for recognizing objects as defined in claim 19 , wherein the disparity histogram and silhouette image generate convex regions which are associated with head movement and which are detected by a convex detector.
Claim 23
Dependent Claim
29 words
A process for recognizing objects as defined in claim 13 , wherein the wavelet transformations are performed using phase calculations that are performed using a hardware adapted phase representation.
Claim 24
Dependent Claim
24 words
A process for recognizing objects as defined in claim 13 , wherein the bunch graph is based on a three-dimensional representation of the object.
Claim 25
Dependent Claim
21 words
A process for recognizing objects as defined in claim 13 , wherein the locating step is performed using a coarse-to-fine approach.
Claim 26
Dependent Claim
20 words
A process for recognizing objects as defined in claim 13 , wherein the bunch graph is based on predetermined poses.
Claim 27
Dependent Claim
24 words
A process for recognizing objects as defined in claim 13 , wherein the bound portion covers less than ten percent of the image frame.
Claim 28
Dependent Claim
26 words
A process for recognizing objects as defined in claim 13 , wherein the step for detecting the object includes detecting a color associated with the object.
Claim 29
Dependent Claim
31 words
A process for recognizing objects as defined in claim 13 , wherein the step for detecting the object includes detecting movement associated with the object in the sequence of image frames.
Claim 30
Dependent Claim
148 words
Apparatus for recognizing objects in an image, comprising: means for detecting an object in the image frame and bounding a portion of the image frame associated with the object resulting in a bound portion of the image frame that is associated with the object and an unbound portion of the image frame that is not associated with the object; means for transforming only the bound portion and not the unbound portion of the image frame using a wavelet transformation to generate a transformed image; means for locating, on the transformed image, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; means for identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed image.
Claim 31
Dependent Claim
148 words
Apparatus for recognizing objects in an image, comprising: means for detecting an object in the image frame and bounding a portion of the image frame associated with the object resulting in a bound portion of the image frame that is associated with the object and an unbound portion of the image frame that is not associated with the object; means for transforming only the bound portion and not the unbound portion of the image frame using a wavelet transformation to generate a transformed image; means for locating, on the transformed image, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; means for identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed image.
Claim 32
Dependent Claim
23 words
Apparatus for recognizing objects as defined in claim 28 , wherein the bound portion covers less than ten percent of the image frame.
Claim 33
Dependent Claim
21 words
Apparatus for recognizing objects as defined in claim 28 , wherein the means for detecting the object includes a neural network.
Claim 34
Dependent Claim
27 words
Apparatus for recognizing objects as defined in claim 28 , wherein the means for detecting the object includes means for detecting a color associated with the object.
Claim 35
Dependent Claim
37 words
Apparatus for recognizing objects as defined in claim 28 , further comprising means for suppressing a background portion, of the bound portion of the image frame, that is not representing the object, prior to identifying the object.
Claim 36
Dependent Claim
31 words
Apparatus for recognizing objects as defined in claim 32 , wherein the suppressed background portion is gradually suppressed near edges of the object in the bound portion of the image frame.
Claim 37
Dependent Claim
151 words
Apparatus for recognizing objects in a sequence of image frames, comprising: means for detecting an object in the image frames and bounding a portion of each image frame associated with the object resulting in a bound portion of each image frame that is associated with the object and an unbound portion of each image frame that is not associated with the object; means for transforming only the bound portion and not the unbound portion of each image frame using a wavelet transformation to generate a transformed image; means for locating, on the transformed images, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; means for identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed images.
Claim 38
Dependent Claim
151 words
Apparatus for recognizing objects in a sequence of image frames, comprising: means for detecting an object in the image frames and bounding a portion of each image frame associated with the object resulting in a bound portion of each image frame that is associated with the object and an unbound portion of each image frame that is not associated with the object; means for transforming only the bound portion and not the unbound portion of each image frame using a wavelet transformation to generate a transformed image; means for locating, on the transformed images, nodes associated with distinguishing features of the object defined by wavelet jets of a bunch graph generated from a plurality of representative object images; means for identifying the object based on a similarity between wavelet jets associated with an object image in a gallery of object images and wavelet jets at the nodes on the transformed images.
Claim 39
Dependent Claim
23 words
Apparatus for recognizing objects as defined in claim 34 , wherein the bound portion covers less than ten percent of each image frame.
Claim 40
Dependent Claim
21 words
Apparatus for recognizing objects as defined in claim 34 , wherein the means for detecting the object includes a neural network.
Claim 41
Dependent Claim
27 words
Apparatus for recognizing objects as defined in claim 34 , wherein the means for detecting the object includes means for detecting a color associated with the object.
Claim 42
Dependent Claim
35 words
Apparatus for recognizing objects as defined in claim 34 , wherein the means for detecting an object further comprises means for tracking the object between image frames based on a trajectory associated with the object.
Claim 43
Dependent Claim
26 words
Apparatus for recognizing objects as defined in claim 34 , wherein the means for locating the nodes includes means for tracking the nodes between image frames.
Claim 44
Dependent Claim
31 words
Apparatus for recognizing objects as defined in claim 39 , further comprising means for reinitializing a tracked node if the node's position deviates beyond a predetermined position constraint between image frames.
Claim 45
Dependent Claim
30 words
Apparatus for recognizing objects as defined in claim 40 , wherein the predetermined position constraint is based on a geometrical position constraint associated with relative positions between the node locations.
Claim 46
Dependent Claim
25 words
Apparatus for recognizing objects as defined in claim 34 , wherein the means for transforming uses a hardware adapted phase representation for performing phase calculations.
References Cited (Backward Citations)
US4805224A
Cited
US4725824A
Cited
US4827413A
Cited
US5168529A
Cited
US5432712A
Cited
US5187574A
Cited
US5280530A
Cited
US5220441A
Cited
US5533177A
Cited
US5159647A
Cited
US5680487A
Cited
US5333165A
Cited
US5430809A
Cited
US5383013A
Cited
US5550928A
Cited
US5511153A
Cited
US5581625A
Cited
DE4406020C1
Cited
US5719954A
Cited
US5736982A
Cited
US5699449A
Cited
US5714997A
Cited
US5982853A
Cited
US5588033A
Cited
US5715325A
Cited
US5774591A
Cited
US5802220A
Cited
US5809171A
Cited
US5764803A
Cited
EP0807902A2
Cited
US5828769A
Cited
US6044168A
Cited
US5917937A
Cited
US6052123A
Cited
US5995119A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2004-10-08 | 2005-04-09 | 2005-10-09 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2008-10-08 | 2009-04-09 | 2009-10-09 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2012-10-08 | 2013-04-09 | 2013-10-09 | $7700.00 | Window Closed / Lapsed |