US7274822B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Face annotation for photo management
Assignee: Zhigu Holdings Ltd
•
Filing: 2003-06-30
•
Granted: 2007-09-25
•
Expires: 2023-06-30
Abstract
Abstract Systems and methods for annotating a face in a digital image are described. In one aspect, a probability model is trained by mapping one or more sets of sample facial features to corresponding names of individuals. A face from an input data set of at least one the digital image is then detected. Facial features are then automatically extracted from the detected face. A similarity measure is them modeled as a posterior probability that the facial features match a particular set of features identified in the probability model. The similarity measure is statistically learned. A name is then inferred as a function of the similarity measure. The face is then annotated with the name.
Inventors
- 👤 Lei Zhang
- 👤 Longbin Chen
- 👤 Mingjing Li
- 👤 Hong-Jiang Zhang
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:29
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/172; G06V40/179
Claims Analysis
Total Claims: 40
Claim 1
Independent Claim
106 words
A method implemented by a computing device, the method comprising: deriving a similarity measure by integrating results from content-based image retrieval and face recognition into a Bayesian framework to obtain a maximum posterior estimation that extended face area contextual features, face appearance, and face components associated with a set of facial features match a particular set of features identified in a probability model, the probability model mapping one or more sets of sample facial features independent of any specific individual to corresponding names of individuals; and inferring a name of a particular person that corresponds to the facial features as a function of the similarity measure.
Claim 2
Independent Claim
106 words
A method implemented by a computing device, the method comprising: deriving a similarity measure by integrating results from content-based image retrieval and face recognition into a Bayesian framework to obtain a maximum posterior estimation that extended face area contextual features, face appearance, and face components associated with a set of facial features match a particular set of features identified in a probability model, the probability model mapping one or more sets of sample facial features independent of any specific individual to corresponding names of individuals; and inferring a name of a particular person that corresponds to the facial features as a function of the similarity measure.
Claim 3
Dependent Claim
37 words
The method as of claim 1 , wherein modeling the similarity measure further comprises marginalizing the similarity measure over one or more missing features to arrive at a similarity measure for the one or more missing features.
Claim 4
Dependent Claim
62 words
The method of claim 1 , wherein deriving the similarity measure further comprises comparing extended face area contextual features from a first digital image with contextual features extracted from a second digital images only if the first and second digital images were generated within a configurable number of days from one another, the second digital image being from a training data set.
Claim 5
Dependent Claim
35 words
The method of claim 1 , wherein inferring the name further comprises generating a candidate name list by integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure.
Claim 6
Dependent Claim
45 words
The method of claim 1 : wherein inferring the name further comprises searching historical labeling results to generate a candidate name list; and wherein the method further comprises automatically annotating the face with the name if the similarity measure meets or exceeds a confidence threshold.
Claim 7
Dependent Claim
97 words
The method of claim 1 , wherein the face is an unknown face, and wherein inferring the name further comprises: integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure to generate a candidate name list, the candidate name list comprising the name; based on a nearest neighbor or K-nearest neighbor algorithm, among K nearest labeled faces to the unknown face, sorting names in the candidate name list according to a sum of similarities measures corresponding to the unknown face; and presenting the candidate list to a user for annotation.
Claim 8
Dependent Claim
65 words
The method of claim 1 , wherein inferring the name further comprises: obtaining labeling result from a user; storing the labeling result into a historical record; performing multi-class classifications to identify a candidate name list as a function of the historical record; and wherein annotating the face further comprises presenting the candidate name list to a user, the name being in the candidate name list.
Claim 9
Dependent Claim
18 words
A computer-readable medium comprising computer-program instructions executable by a processor for performing the method of claim 1 .
Claim 10
Dependent Claim
94 words
A computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions comprising instructions for: training a probability model to map one or more sets of sample facial features to corresponding names of individuals; detecting a face in a digital image; extracting facial features from the face; modeling a similarity measure as a posterior probability that the facial features match a particular set of features identified in a probability model, the similarity measure being statistically learned; inferring a name as a function of the similarity measure; and annotating the face with the name.
Claim 11
Dependent Claim
94 words
A computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions comprising instructions for: training a probability model to map one or more sets of sample facial features to corresponding names of individuals; detecting a face in a digital image; extracting facial features from the face; modeling a similarity measure as a posterior probability that the facial features match a particular set of features identified in a probability model, the similarity measure being statistically learned; inferring a name as a function of the similarity measure; and annotating the face with the name.
Claim 12
Dependent Claim
17 words
The computer-readable medium of claim 9 , wherein the digital image is from a family photo album.
Claim 13
Dependent Claim
22 words
The computer-readable medium of claim 9 , wherein the facial features comprise face appearance, face component, and extended face area contextual features.
Claim 14
Dependent Claim
31 words
The computer-readable medium of claim 9 , wherein the facial features comprise face appearance, face component, and extended face area contextual features, the face appearance and components being modeled in eigenspaces.
Claim 15
Dependent Claim
34 words
The computer-readable medium of claim 9 , wherein the instructions for modeling the similarity measure further comprise computer-program instructions for integrating results from content-based image retrieval and face recognition algorithms into a Bayesian framework.
Claim 16
Dependent Claim
43 words
The computer-readable medium of claim 9 , wherein the instructions for modeling the similarity measure further comprise computer-program instructions for marginalizing the similarity measure over one or more missing features to arrive at a similarity measure for the one or more missing features.
Claim 17
Dependent Claim
62 words
The computer-readable medium of claim 9 , wherein the facial features comprise contextual features, and wherein the instructions for modeling the similarity measure further comprise computer-program instructions for comparing contextual features of the face with contextual features from other digital images only if the other digital images were generated within a configurable number of days from when the digital image was created.
Claim 18
Dependent Claim
29 words
The computer-readable medium of claim 9 , wherein the instructions for inferring the name further comprise computer-program instructions for searching historical labeling results to generate a candidate name list.
Claim 19
Dependent Claim
42 words
The computer-readable medium of claim 9 , wherein the instructions for inferring the name further comprise computer-program instructions for generating a candidate name list by integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure.
Claim 20
Dependent Claim
104 words
The computer-readable medium of claim 9 , wherein the face is an unknown face, and wherein the instructions for inferring the name further comprise computer-program instructions for: integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure to generate a candidate name list, the candidate name list comprising the name; based on a nearest neighbor or K-nearest neighbor algorithm, among K nearest labeled faces to the unknown face, sorting names in the candidate name list according to a sum of similarities measures corresponding to the unknown face; and presenting the candidate list to a user for annotation.
Claim 21
Dependent Claim
72 words
The computer-readable medium of claim 9 , wherein the instructions for inferring the name further comprise computer-program instructions for: obtaining labeling result from a user; storing the labeling result into a historical record; performing multi-class classifications to identify a candidate name list as a function of the historical record; and wherein annotating the face further comprises presenting the candidate name list to a user, the name being in the candidate name list.
Claim 22
Dependent Claim
31 words
The computer-readable medium of claim 9 , further comprising computer-program instructions for presenting the name in a user interface (UI), the UI allowing a user to accept or reject the name.
Claim 23
Dependent Claim
119 words
A computing device comprising a processor and a memory, the memory comprising computer-program instructions executable by the processor, the computer-program instructions comprising instructions for: extracting facial features from a detected face in digital image data; deriving a similarity measure by integrating results from content-based image retrieval and face recognition into a Bayesian framework to obtain a maximum posterior estimation that extended face area contextual features, face appearance, and face components associated with the facial features match a particular set of features identified in a probability model, the probability model mapping one or more sets of sample facial features independent of any specific individual to corresponding names of individuals; and inferring a name as a function of the similarity measure.
Claim 24
Dependent Claim
119 words
A computing device comprising a processor and a memory, the memory comprising computer-program instructions executable by the processor, the computer-program instructions comprising instructions for: extracting facial features from a detected face in digital image data; deriving a similarity measure by integrating results from content-based image retrieval and face recognition into a Bayesian framework to obtain a maximum posterior estimation that extended face area contextual features, face appearance, and face components associated with the facial features match a particular set of features identified in a probability model, the probability model mapping one or more sets of sample facial features independent of any specific individual to corresponding names of individuals; and inferring a name as a function of the similarity measure.
Claim 25
Dependent Claim
43 words
The computing device of claim 21 , wherein the instructions for modeling the similarity measure further comprise computer-program instructions for marginalizing the similarity measure over one or more missing features to arrive at a similarity measure for the one or more missing features.
Claim 26
Dependent Claim
75 words
The computing device of claim 21 , wherein the digital image is a first digital image, and wherein the instructions for deriving the similarity measure further comprise computer-program instructions for comparing the extended face area contextual features with contextual features extracted from a second digital images only if the first and second digital images were generated within a configurable number of days from one another, the second digital image being from a training data set.
Claim 27
Dependent Claim
42 words
The computing device of claim 21 , wherein the instructions for inferring the name further comprise computer-program instructions for generating a candidate name list by integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure.
Claim 28
Dependent Claim
54 words
The computing device of claim 21 : wherein the instructions for inferring the name further comprise computer-program instructions for searching historical labeling results to generate a candidate name list; and wherein the computer program instructions further comprise automatically annotating the face with the name if the similarity measure meets or exceeds a confidence threshold.
Claim 29
Dependent Claim
104 words
The computing device of claim 21 , wherein the face is an unknown face, and wherein the instructions for inferring the name further comprise computer-program instructions for: integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure to generate a candidate name list, the candidate name list comprising the name; based on a nearest neighbor or K-nearest neighbor algorithm, among K nearest labeled faces to the unknown face, sorting names in the candidate name list according to a sum of similarities measures corresponding to the unknown face; and presenting the candidate list to a user for annotation.
Claim 30
Dependent Claim
72 words
The computing device of claim 21 , wherein the instructions for inferring the name further comprise computer-program instructions for: obtaining labeling result from a user; storing the labeling result into a historical record; performing multi-class classifications to identify a candidate name list as a function of the historical record; and wherein annotating the face further comprises presenting the candidate name list to a user, the name being in the candidate name list.
Claim 31
Dependent Claim
21 words
The computing device of claim 21 , wherein the computer-program instructions further comprise instructions for annotating the face with the name.
Claim 32
Dependent Claim
104 words
A computing device comprising: means for deriving a similarity measure by integrating results from content-based image retrieval and face recognition into a Bayesian framework to obtain a maximum posterior estimation that extended face area contextual features, face appearance, and face components associated with a set of facial features match a particular set of features identified in a probability model, the probability model mapping one or more sets of sample facial features independent of any specific individual to corresponding names of individuals; and means for inferring a name of a particular person that corresponds to the facial features as a function of the similarity measure.
Claim 33
Dependent Claim
104 words
A computing device comprising: means for deriving a similarity measure by integrating results from content-based image retrieval and face recognition into a Bayesian framework to obtain a maximum posterior estimation that extended face area contextual features, face appearance, and face components associated with a set of facial features match a particular set of features identified in a probability model, the probability model mapping one or more sets of sample facial features independent of any specific individual to corresponding names of individuals; and means for inferring a name of a particular person that corresponds to the facial features as a function of the similarity measure.
Claim 34
Dependent Claim
21 words
The computing device of claim 29 , wherein the facial features are associated with a face extracted from a digital photograph.
Claim 35
Dependent Claim
42 words
The computing device of claim 29 , wherein the means for modeling the similarity measure further comprise means for marginalizing the similarity measure over one or more missing features to arrive at a similarity measure for the one or more missing features.
Claim 36
Dependent Claim
74 words
The computing device of claim 29 : wherein the digital image is a first digital image; and wherein the means for deriving the similarity measure further comprise means for comparing the extended face area contextual features with contextual features extracted from a second digital images only if the first and second digital images were generated within a configurable number of days from one another, the second digital image being from a training data set.
Claim 37
Dependent Claim
41 words
The computing device of claim 29 , wherein the means for inferring the name further comprise means for generating a candidate name list by integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure.
Claim 38
Dependent Claim
54 words
The computing device of claim 29 : wherein the means for inferring the name further comprise means for searching historical labeling results to generate a candidate name list; and wherein the computing device further comprise means for automatically annotating the face with the name if the similarity measure meets or exceeds a confidence threshold.
Claim 39
Dependent Claim
107 words
The computing device of claim 29 , wherein the face is an unknown face, and wherein the means for inferring the name further comprise: means for integrating priors of both intra-personal variations Ω I and inter-personal variations Ω E into the similarity measure to generate a candidate name list, the candidate name list comprising the name; means for based on a nearest neighbor or K-nearest neighbor algorithm, among K nearest labeled faces to the unknown face, sorting names in the candidate name list according to a sum of similarities measures corresponding to the unknown face; and means for presenting the candidate list to a user for annotation.
Claim 40
Dependent Claim
77 words
The computing device of claim 29 , wherein the means for inferring the name further comprises: means for obtaining labeling result from a user; means for storing the labeling result into a historical record; means for performing multi-class classifications to identify a candidate name list as a function of the historical record; and means for wherein annotating the face further comprises presenting the candidate name list to a user, the name being in the candidate name list.
References Cited (Backward Citations)
US6940545B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-09-24 | 2011-03-26 | 2011-09-25 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-09-24 | 2015-03-26 | 2015-09-25 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-09-24 | 2019-03-26 | 2019-09-25 | $7700.00 | Window Closed / Lapsed |