Patent US7146028B2 successfully refreshed from Google Patents.
US7146028B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Face detection and tracking in a video sequence
Assignee: Canon Inc
•
Filing: 2003-04-10
•
Granted: 2006-12-05
•
Expires: 2023-04-10
Abstract
Abstract A method ( 100 ) and apparatus ( 700 ) are disclosed for detecting and tracking human faces across a sequence of video frames. Spatiotemporal segmentation is used to segment ( 115 ) the sequence of video frames into 3D segments. 2D segments are then formed from the 3D segments, with each 2D segment being associated with one 3D segment. Features are extracted ( 140 ) from the 2D segments and grouped into groups of features. For each group of features, a probability that the group of features includes human facial features is calculated ( 145 ) based on the similarity of the geometry of the group of features with the geometry of a human face model. Each group of features is also matched with a group of features in a previous 2D segment and an accumulated probability that said group of features includes human facial features is calculated ( 150 ). Each 2D segment is classified ( 155 ) as a face segment or a non-face segment based on the accumulated probability. Human faces are then tracked by finding 2D segments in subsequent frames associated with 3D segments associated with face segments.
Inventors
- 👤 Fabrice Lestideau
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 19:31
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/161
Claims Analysis
Total Claims: 24
Claim 1
Independent Claim
227 words
A method of detecting and tracking human faces across a sequence of video frames, said method comprising the steps of: (a) forming a 3D pixel data block from said sequence of video frames; (b) segmenting said 3D data block into a set of 3D segments using 3D spatiotemporal segmentation; (c) forming 2D segments from an intersection of said 3D segments with a view plane, each 2D segment being associated with one 3D segment; (d) in at least one of said 2D segments, extracting features and grouping said features into one or more groups of features; (e) for each group of features, computing a probability that said group of features represents human facial features based on the similarity of the geometry of said group of features with the geometry of a human face model; (f) matching at least one group of features with a group of features in a previous 2D segment and computing an accumulated probability that said group of features represents human facial features using probabilities of matched groups of features; (g) classifying each 2D segment as a face segment or a non-face segment based on said accumulated probability of at least one group of features in each of said 2D segments; and (h) tracking said human faces by finding an intersection of 3D segments associated with said face segments with at least subsequent view planes.
Claim 2
Independent Claim
227 words
A method of detecting and tracking human faces across a sequence of video frames, said method comprising the steps of: (a) forming a 3D pixel data block from said sequence of video frames; (b) segmenting said 3D data block into a set of 3D segments using 3D spatiotemporal segmentation; (c) forming 2D segments from an intersection of said 3D segments with a view plane, each 2D segment being associated with one 3D segment; (d) in at least one of said 2D segments, extracting features and grouping said features into one or more groups of features; (e) for each group of features, computing a probability that said group of features represents human facial features based on the similarity of the geometry of said group of features with the geometry of a human face model; (f) matching at least one group of features with a group of features in a previous 2D segment and computing an accumulated probability that said group of features represents human facial features using probabilities of matched groups of features; (g) classifying each 2D segment as a face segment or a non-face segment based on said accumulated probability of at least one group of features in each of said 2D segments; and (h) tracking said human faces by finding an intersection of 3D segments associated with said face segments with at least subsequent view planes.
Claim 3
Dependent Claim
26 words
A method according to claim 1 , wherein said features are regions in said 2D segment which are darker than the rest of said 2D segment.
Claim 4
Dependent Claim
18 words
A method according to claim 1 , wherein said features are regions in said 2D segment having edges.
Claim 5
Dependent Claim
15 words
A method according to claim 1 , wherein said group of features forms a triangle.
Claim 6
Dependent Claim
50 words
A method according to claim 1 , wherein said method comprises the further steps of: determining, for each said 2D segment, a first measure of said 2D segment having a colour of human skin; and eliminating 2D segments having said first measure below a first predetermined threshold from further processing.
Claim 7
Dependent Claim
27 words
A method according to claim 1 , wherein said method comprises the further step of: eliminating 2D segments having a form that is non-elliptical from further processing.
Claim 8
Dependent Claim
49 words
A method according to claim 1 , wherein said method comprises the further steps of: determining movements of said 2D segments from positions of previous 2D segments associated with the same 3D segments; and eliminating 2D segments from further processing where said movement is below a second predetermined threshold.
Claim 9
Independent Claim
232 words
An apparatus for detecting and tracking human faces across a sequence of video frames, said apparatus comprising: means for forming a 3D pixel data block from said sequence of video frames; means for segmenting said 3D data block into a set of 3D segments using 3D spatiotemporal segmentation; means for forming 2D segments from an intersection of said 3D segments with a view plane, each 2D segment being associated with one 3D segment; in at least one of said 2D segments, means for extracting features and grouping said feature's into one or more groups of features; for each group of features, means for computing a probability that said group of features represents human facial features based on the similarity of the geometry of said group of features with the geometry of a human face model; means for matching at least one group of features with a group of features in a previous 2D segment and computing an accumulated probability that said group of features represents human facial features using probabilities of matched groups of features; means for classifying each 2D segment as a face segment or a non-face segment based on said accumulated probability of at least one group of features in each of said 2D segments; and means for tracking said human faces by finding an intersection of 3D segments associated with said face segments with at least subsequent view planes.
Claim 10
Independent Claim
232 words
An apparatus for detecting and tracking human faces across a sequence of video frames, said apparatus comprising: means for forming a 3D pixel data block from said sequence of video frames; means for segmenting said 3D data block into a set of 3D segments using 3D spatiotemporal segmentation; means for forming 2D segments from an intersection of said 3D segments with a view plane, each 2D segment being associated with one 3D segment; in at least one of said 2D segments, means for extracting features and grouping said feature's into one or more groups of features; for each group of features, means for computing a probability that said group of features represents human facial features based on the similarity of the geometry of said group of features with the geometry of a human face model; means for matching at least one group of features with a group of features in a previous 2D segment and computing an accumulated probability that said group of features represents human facial features using probabilities of matched groups of features; means for classifying each 2D segment as a face segment or a non-face segment based on said accumulated probability of at least one group of features in each of said 2D segments; and means for tracking said human faces by finding an intersection of 3D segments associated with said face segments with at least subsequent view planes.
Claim 11
Dependent Claim
26 words
An apparatus according to claim 8 , wherein said features are regions in said 2D segment which are darker than the rest of said 2D segment.
Claim 12
Dependent Claim
18 words
An apparatus according to claim 8 , wherein said features are regions in said 2D segment having edges.
Claim 13
Dependent Claim
15 words
An apparatus according to claim 8 , wherein said group of features forms a triangle.
Claim 14
Dependent Claim
51 words
An apparatus according to claim 8 , wherein said apparatus further comprises: means for determining, for each said 2D segment, a first measure of said 2D segment having a colour of human skin; and means for eliminating 2D segments having said first measure below a first predetermined threshold from further processing.
Claim 15
Dependent Claim
26 words
An apparatus according to claim 8 , wherein said apparatus further comprises: means for eliminating 2D segments having a form that is non-elliptical from further processing.
Claim 16
Dependent Claim
50 words
An apparatus according to claim 8 , wherein said apparatus further comprises: means for determining movements of said 2D segments from positions of previous 2D segments associated with the same 3D segments; and means for eliminating 2D segments from further processing where said movement is below a second predetermined threshold.
Claim 17
Dependent Claim
242 words
A computer-executable program stored on a computer readable storage medium, the program for detecting and tracking human faces across a sequence of video frames, said program comprising: code for forming a 3D pixel data block from said sequence of video frames; code for segmenting said 3D data block into a set of 3D segments using 3D spatiotemporal segmentation; code for forming 2D segments from an intersection of said 3D segments with a view plane, each 2D segment being associated with one 3D segment; in at least one of said 2D segments, code for extracting features and grouping said features into one or more groups of features; for each group of features, code for computing a probability that said group of features represents human facial features based on the similarity of the geometry of said group of features with the geometry of a human face model; code for matching at least one group of features with a group of features in a previous 2D segment and computing an accumulated probability that said group of features represents human facial features using probabilities of matched groups of features; code for classifying each 2D segment as a face segment or a non-face segment based on said accumulated probability of at least one group of features in each of said 2D segments; and code for tracking said human faces by finding an intersection of 3D segments associated with said face segments with at least subsequent view planes.
Claim 18
Dependent Claim
242 words
A computer-executable program stored on a computer readable storage medium, the program for detecting and tracking human faces across a sequence of video frames, said program comprising: code for forming a 3D pixel data block from said sequence of video frames; code for segmenting said 3D data block into a set of 3D segments using 3D spatiotemporal segmentation; code for forming 2D segments from an intersection of said 3D segments with a view plane, each 2D segment being associated with one 3D segment; in at least one of said 2D segments, code for extracting features and grouping said features into one or more groups of features; for each group of features, code for computing a probability that said group of features represents human facial features based on the similarity of the geometry of said group of features with the geometry of a human face model; code for matching at least one group of features with a group of features in a previous 2D segment and computing an accumulated probability that said group of features represents human facial features using probabilities of matched groups of features; code for classifying each 2D segment as a face segment or a non-face segment based on said accumulated probability of at least one group of features in each of said 2D segments; and code for tracking said human faces by finding an intersection of 3D segments associated with said face segments with at least subsequent view planes.
Claim 19
Dependent Claim
26 words
A program according to claim 15 , wherein said features are regions in said 2D segment which are darker than the rest of said 2D segment.
Claim 20
Dependent Claim
18 words
A program according to claim 15 , wherein said features are regions in said 2D segment having edges.
Claim 21
Dependent Claim
15 words
A program according to claim 15 , wherein said group of features forms a triangle.
Claim 22
Dependent Claim
51 words
A program according to claim 15 , wherein said program further comprises: code for determining for each said 2D segment, a first measure of said 2D segment having a colour of human skin; and code for eliminating 2D segments having said first measure below a first predetermined threshold from further processing.
Claim 23
Dependent Claim
26 words
A program according to claim 15 , wherein said program further comprises: code for eliminating 2D segments having a form that is non-elliptical from further processing.
Claim 24
Dependent Claim
50 words
A program according to claim 15 , wherein said program further comprises: code for determining movements of said 2D segments from positions of previous 2D segments associated with the same 3D segment; and code for eliminating 2D segments from further processing where said movement is below a second predetermined threshold.
References Cited (Backward Citations)
US5987154A
Cited
US5912980A
Cited
US5802220A
Cited
GB2320839A
Cited
US6445810B2
Cited
US6272231B1
Cited
US6760488B1
Cited
US6707933B1
Cited
JP2001141425A
Cited
US6940545B1
Cited
US6993163B2
Cited
EP1139270A2
Cited
US20010031073A1
Cited
WO2002007095A1
Cited
WO2002009025A1
Cited
US6697502B2
Cited
US6829384B2
Cited
US7099505B2
Cited
US20040017933A1
Cited
US7082211B2
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-12-04 | 2010-06-05 | 2010-12-05 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-12-04 | 2014-06-05 | 2014-12-05 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-12-04 | 2018-06-05 | 2018-12-05 | $7700.00 | Window Closed / Lapsed |