US6879709B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for automatically detecting neutral expressionless faces in digital images
Assignee: Samsung Electronics Co Ltd
•
Filing: 2002-01-17
•
Granted: 2005-04-12
•
Expires: 2022-01-17
Abstract
Abstract A system and method for automatic detecting neutral expressionless faces in digital images and video is described. First a face detector is used to detect the pose and position of a face and find the facial components. Second, the detected face is normalized to a standard size face. Then a set of geometrical facial features and three histograms in zones of mouth are extracted. Finally, by feeding these features to a classifier, the system detects if there is the neutral expressionless face or not.
Inventors
- 👤 Ying-Li Tian
- 👤 Rudolf M. Bolle
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:12
CPC Classifications
G; G06; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/174; G06V40/175
Claims Analysis
Total Claims: 42
Claim 1
Independent Claim
159 words
A system for automatically detecting neutral, expressionless face images in digital images, the system comprising: an image acquisition unit; an face detector receiving input from the image acquisition unit, detecting one or more face subimages of one or more faces in the image; a characteristic point detector that receives input from the face detector and that estimates one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; a facial feature detector that detects one or more contours of one or more facial components; a facial feature analyzer that determines a mouth shape of the mouth from the contour of the mouth and creates a representation of the mouth shape, the mouth being one of the facial components; and a face classification unit that classifies the representation into one of a neutral class and a non-neutral class.
Claim 2
Independent Claim
159 words
A system for automatically detecting neutral, expressionless face images in digital images, the system comprising: an image acquisition unit; an face detector receiving input from the image acquisition unit, detecting one or more face subimages of one or more faces in the image; a characteristic point detector that receives input from the face detector and that estimates one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; a facial feature detector that detects one or more contours of one or more facial components; a facial feature analyzer that determines a mouth shape of the mouth from the contour of the mouth and creates a representation of the mouth shape, the mouth being one of the facial components; and a face classification unit that classifies the representation into one of a neutral class and a non-neutral class.
Claim 3
Dependent Claim
50 words
A system, as in claim 1 , where the mouth shape is determined by a width of the mouth, the first distance from the first mouth corner to a reference in the coordinate systems, and the second distance from the second mouth corner to the reference in the coordinate systems.
Claim 4
Dependent Claim
36 words
A system, as in claim 1 , where one or more of the characteristic facial features is an eye openness measure of an eye of the facial image and the representation includes the eye openness measure.
Claim 5
Dependent Claim
24 words
A system, as in claim 3 , where the eye openness measure of the eyes is measured by a normalized height of the eyes.
Claim 6
Dependent Claim
31 words
A system, as in claim 1 , where one or more characteristic facial features is derived from any one of an eye region and an eyebrow region in the face subimage.
Claim 7
Dependent Claim
16 words
A system, as in claim 1 , where the contours are extracted by an edge detector.
Claim 8
Dependent Claim
18 words
A system, as in claim 1 , where the characteristic facial features are estimated by template matching techniques.
Claim 9
Dependent Claim
32 words
A system, as in claim 1 , where the facial components include any one or more of the following: a mouth, an eye, an eyebrow, a nose, an ear, and a wrinkle.
Claim 10
Dependent Claim
22 words
A system, as in claim 1 , where a mouth position of the mouth is determined with respect to a geometric reference.
Claim 11
Dependent Claim
23 words
A system, as in claim 1 , where a mouth position of the mouth is determined with respect to an image coordinate system.
Claim 12
Dependent Claim
24 words
A system, as in claim 1 , where the characteristic points define one or more characteristic distances to one or more respective geometric references.
Claim 13
Dependent Claim
23 words
A system, as in claim 11 , where one or more of the characteristic distances is normalized to another of the characteristic distances.
Claim 14
Dependent Claim
21 words
A system, as in claim 11 , where the facial feature analyzer divides the face sub-image into two or more zones.
Claim 15
Dependent Claim
31 words
A system, as in claim 13 , where facial feature analyzer determines one or more component shapes from one or more respective component contours in one or more of the zones.
Claim 16
Dependent Claim
29 words
A system, as in claim 14 , where the component is the mouth and the shape of the mouth is extracted from three of the zones containing the mouth.
Claim 17
Dependent Claim
31 words
A system, as in claim 15 , where the shape of the mouth is determined by histograming the contours of the mouth in each of the three zones containing the mouth.
Claim 18
Dependent Claim
38 words
A system as in claim 16 where the mouth shape is defined by three shape histograms each corresponding to one of the three zones containing the mouth obtained by quantizing the contour of the mouth into four directions.
Claim 19
Dependent Claim
32 words
A system, as in claim 15 , where a further component is one or more of an eye and an eyebrow that are extracted from three of the zones containing the eyes.
Claim 20
Dependent Claim
37 words
A system, as in claim 11 , where one or more of the characteristic distances is the normalized width of the mouth, the width of the mouth being the distance between the first and second mouth corners.
Claim 21
Dependent Claim
44 words
A system, as in claim 1 , where the characteristic points further include a location of each eye on the face, being eye characteristic points and one of the characteristic distances is a distance between the eye characteristic points, being an eye separation distance.
Claim 22
Dependent Claim
23 words
A system, as in claim 20 , where one or more of the other characteristic distances is normalized by the eye separation distance.
Claim 23
Dependent Claim
18 words
A system, as in claim 20 , where the eye characteristic points are located on the eye pupil.
Claim 24
Dependent Claim
42 words
A system, as in claim 20 , where a mouth-eye distance between the first mouth corner and a first eye is normalized by the eye separation distance, and the normalized mouth-eye distance is an additional feature used by the face classification unit.
Claim 25
Dependent Claim
41 words
A system, as in claim 20 , where a brow-eye distance between a first eyebrow and a first eye is normalized by the eye separation distance, and the normalized brow-eye distance is an additional feature used by the face classification unit.
Claim 26
Dependent Claim
41 words
A system, as in claim 20 , where a brow distance between the first eyebrow and the second eyebrow is normalized by the eye separation distance, and the normalized brow distance is an additional feature used by the face classification unit.
Claim 27
Dependent Claim
27 words
A system, as in claim 1 , where one or more normalized characteristic distances is used by the face classification unit classify if the face is neutral.
Claim 28
Dependent Claim
38 words
A system, as in claim 26 , where the face classification unit is one or more of the following classifiers: neural networks, nearest neighbor methods, Bayesian inference methods, linear discriminant methods, perceptrons, support vector machines, and related variants.
Claim 29
Dependent Claim
16 words
A system, as in claim 26 , where the face classification unit is neural network (NN).
Claim 30
Dependent Claim
29 words
A system, as in claim 1 , where the face classification unit is used to enroll one or more subjects in a face-based authentication system with a neutral face.
Claim 31
Dependent Claim
29 words
A system, as in claim 1 , where the face classification unit is used to authenticate one or more subjects in a face-based authentication system with a neutral face.
Claim 32
Dependent Claim
21 words
A system, as in claim 1 , used as a user interface that reacts to one or more human emotion states.
Claim 33
Dependent Claim
12 words
A system, as in claim 1 , used as a surveillance device.
Claim 34
Dependent Claim
48 words
A system, as in claim 32 , used in one or more of the following: a financial institution, a bank a retail store, a credit card check, a check in, an air port, an air plane boarding check, a building access system, and a object access authorization system.
Claim 35
Dependent Claim
13 words
A system, as in claim 1 , used as an information retrieval device.
Claim 36
Dependent Claim
47 words
A system, as in claim 34 , where the information retrieval device includes any one or more of the following: a video retrieval of a frame containing a face image with a specified expression, an image of an image containing a face image with a specified expression.
Claim 37
Independent Claim
122 words
A method for automatically detecting neutral, expressionless face images in digital images, the method comprising the steps of: acquiring an image; detecting one or more face subimages of one or more faces in the image; estimating one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; detecting one or more contours of one or more facial components; determining a mouth shape of the mouth from the contour of the mouth; creating a representation of the mouth shape, the mouth being one of the facial components; and classifying the representation into one of a neutral class and a non-neutral class.
Claim 38
Independent Claim
122 words
A method for automatically detecting neutral, expressionless face images in digital images, the method comprising the steps of: acquiring an image; detecting one or more face subimages of one or more faces in the image; estimating one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; detecting one or more contours of one or more facial components; determining a mouth shape of the mouth from the contour of the mouth; creating a representation of the mouth shape, the mouth being one of the facial components; and classifying the representation into one of a neutral class and a non-neutral class.
Claim 39
Independent Claim
133 words
A system for automatically detecting neutral, expressionless face images in digital images, the method comprising: means for acquiring an image; means for detecting one or more face subimages of one or more faces in the image; means for estimating one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; means for detecting one or more contours of one or more facial components; means for determining a mouth shape of the mouth from the contour of the mouth; means for creating a representation of the mouth shape, the mouth being one of the facial components; and means for classifying the representation into one of a neutral class and a non-neutral class.
Claim 40
Independent Claim
133 words
A system for automatically detecting neutral, expressionless face images in digital images, the method comprising: means for acquiring an image; means for detecting one or more face subimages of one or more faces in the image; means for estimating one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; means for detecting one or more contours of one or more facial components; means for determining a mouth shape of the mouth from the contour of the mouth; means for creating a representation of the mouth shape, the mouth being one of the facial components; and means for classifying the representation into one of a neutral class and a non-neutral class.
Claim 41
Dependent Claim
145 words
A program storage device, readable by machine, tangibly embodying a program of instructions executable by the machine to cause the machine to perform a method for automatically detecting neutral, expressionless face images in digital images, the method comprising the steps of: acquiring an image; detecting one or more face subimages of one or more faces in the image; estimating one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; detecting one or more contours of one or more facial components; determining a mouth shape of the mouth from the contour of the mouth; creating a representation of the mouth shape, the mouth being one of the facial components; and classifying the representation into one of a neutral class and a non-neutral class.
Claim 42
Dependent Claim
145 words
A program storage device, readable by machine, tangibly embodying a program of instructions executable by the machine to cause the machine to perform a method for automatically detecting neutral, expressionless face images in digital images, the method comprising the steps of: acquiring an image; detecting one or more face subimages of one or more faces in the image; estimating one or more characteristic facial features as characteristic points in each detected face subimage, one of the characteristic points being a first and a second mouth corner on a mouth of the face; detecting one or more contours of one or more facial components; determining a mouth shape of the mouth from the contour of the mouth; creating a representation of the mouth shape, the mouth being one of the facial components; and classifying the representation into one of a neutral class and a non-neutral class.
References Cited (Backward Citations)
US5164992A
Cited
US5410609A
Cited
US6400835B1
Cited
US6009210A
Cited
US6556196B1
Cited
US6807290B2
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2008-04-11 | 2008-10-11 | 2009-04-12 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2012-04-11 | 2012-10-11 | 2013-04-12 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2016-04-11 | 2016-10-11 | 2017-04-12 | $7700.00 | Window Closed / Lapsed |