US8014572B2 — Face annotation framework with partial clustering…

Claim 1

Independent Claim 122 words

A method, comprising: identifying clusters of images in a collection of images with a deliberate bias toward finding evident clusters instead of trying to force every image in the collection into a cluster, each cluster including images of a face of a person characterizing the cluster, the evident clusters having a uniform background distribution for noisy data; accepting a user interaction to label each cluster; iteratively formulating proposed clusters from remaining images in the collection to solicit subsequent user interactions for labeling images in each proposed cluster, each iteration of the formulating proposing a cluster such that the subsequent user interactions maximize an information efficiency for subsequently labeling the remaining images in the collection while using a minimum number of user interactions.

Claim 2

Independent Claim 122 words

A method, comprising: identifying clusters of images in a collection of images with a deliberate bias toward finding evident clusters instead of trying to force every image in the collection into a cluster, each cluster including images of a face of a person characterizing the cluster, the evident clusters having a uniform background distribution for noisy data; accepting a user interaction to label each cluster; iteratively formulating proposed clusters from remaining images in the collection to solicit subsequent user interactions for labeling images in each proposed cluster, each iteration of the formulating proposing a cluster such that the subsequent user interactions maximize an information efficiency for subsequently labeling the remaining images in the collection while using a minimum number of user interactions.

Claim 3

Dependent Claim 68 words

The method as recited in claim 1 , wherein identifying clusters comprises including an image in a cluster when a face in the image is identified with a high degree of certainty as the face of the person common to other images in the cluster; and wherein images that contain faces that are not identified with a high degree of certainty remain unclustered in the collection of images.

Claim 4

Dependent Claim 30 words

The method as recited in claim 1 , wherein the iteratively formulating proposed clusters maximizes an information efficiency for formulating subsequent proposed clusters using a minimum number of user operations.

Claim 5

Dependent Claim 38 words

The method as recited in claim 1 , wherein iteratively formulating proposed clusters includes selecting a group of images for each cluster to solicit a user interaction for maximizing the information gain based on a global system entropy.

Claim 6

Dependent Claim 20 words

The method as recited in claim 1 , wherein maximizing the information efficiency is based on a global system entropy.

Claim 7

Dependent Claim 41 words

The method as recited in claim 1 , wherein iteratively formulating proposed clusters includes selecting a group of images for each cluster to solicit a user interaction for minimizing the subsequent user interactions for labeling the remaining images in the collection.

Claim 8

Dependent Claim 19 words

The method as recited in claim 6 , wherein minimizing subsequent user interactions is based on a subset-saliency entropy.

Claim 9

Dependent Claim 79 words

The method as recited in claim 1 , wherein identifying clusters of images in a collection of images includes the automatic steps of: detecting and aligning faces; deriving features associated with each image, including facial features, clothing features, and temporal features; deriving a Bayesian combination of the features into a similarity matrix containing similarity pairs; spectrally embedding the similarity pairs in a compact manifold; and applying an iterative expectation-maximization technique to discern images belonging to one of the clusters.

Claim 10

Dependent Claim 76 words

The method as recited in claim 8 , wherein the deriving a Bayesian combination includes calculating facial feature distances, clothing feature distances, and time feature distances; wherein calculating the feature distances uses a time prior such that a face of a person is associated with the same clothing features during short intervals; and wherein calculating the feature distances uses a cannot-link prior such that a face of a person appears only once in a single image.

Claim 11

Dependent Claim 26 words

The method as recited in claim 8 , further comprising replacing a K-means component of spectral clustering with partial clustering such that only evident clusters emerge.

Claim 12

Dependent Claim 48 words

The method as recited in claim 10 , wherein the similarity pairs are modeled as a mixture of Gaussian distributions in the compact manifold, wherein the mixture includes a uniform background noise distribution and an exponential prior to control variances of each component Gaussian distribution in the mixture.

Claim 13

Dependent Claim 20 words

The method as recited in claim 10 , further comprising using an iterative Expectation-Maximization (EM) technique to derive evident clusters.

Claim 14

Dependent Claim 25 words

The method as recited in claim 12 , wherein the iterative EM technique solves a Maximum a Posteriori (MAP) estimation of a posterior likelihood function.

Claim 15

Independent Claim 125 words

A system, comprising: a storage device for storing a collection of digital images; a face annotation engine to label the digital images using a minimal number of user interactions; a partial clustering engine in the face annotation engine to automatically cluster some of the digital images into evident clusters instead of forcing every image in the collection into a cluster, each evident cluster containing images that include a face of a single person, and the evident clusters having a uniform background distribution for noisy data; and an efficient labeling engine in the face annotation engine to iteratively propose clusters of unlabeled images such that user interactions to label the proposed clusters maximize labeling of the digital images and achieve the minimal number of user interactions.

Claim 16

Independent Claim 125 words

A system, comprising: a storage device for storing a collection of digital images; a face annotation engine to label the digital images using a minimal number of user interactions; a partial clustering engine in the face annotation engine to automatically cluster some of the digital images into evident clusters instead of forcing every image in the collection into a cluster, each evident cluster containing images that include a face of a single person, and the evident clusters having a uniform background distribution for noisy data; and an efficient labeling engine in the face annotation engine to iteratively propose clusters of unlabeled images such that user interactions to label the proposed clusters maximize labeling of the digital images and achieve the minimal number of user interactions.

Claim 17

Dependent Claim 51 words

The system as recited in claim 14 , further comprising: a feature extractor in the partial clustering engine to derive multiple features, including facial features, clothing features, and temporal features of each image; and a Bayesian face similarity comparator to combine the multiple features into a similarity matrix containing similarity pairs.

Claim 18

Dependent Claim 51 words

The system as recited in claim 15 , further comprising a spectral partial clustering module to concentrate similarity pair data based on a mixture of Gaussian distributions, wherein a Gaussian mixture model contains a uniform background noise distribution and an exponential prior to control the variances of each component Gaussian distribution.

Claim 19

Dependent Claim 41 words

The system as recited in claim 16 , further comprising an expectation-maximization engine in the partial clustering engine to iteratively solve a Maximum a Posteriori (MAP) estimation of a posterior likelihood to discern images belonging to each of the evident clusters.

Claim 20

Dependent Claim 47 words

The system as recited in claim 14 , wherein the efficient labeling engine further includes an information efficiency optimizer to maximize an information gain for automatically labeling unlabeled images in the collection of digital images while minimizing the number of user interactions to label the unlabeled images.

Claim 21

Dependent Claim 34 words

The system as recited in claim 18 , wherein the information efficiency optimizer includes: an information gain model based on global system entropy; and a user interactions decrease model based on a subset-saliency entropy.

Claim 22

Independent Claim 161 words

A method performed by a computing device, the method comprising: automatically grouping some images of a collection of images into clusters, the grouping comprising: using spectral embedding to find representation of face of an image in a plurality of feature spaces, the plurality of feature spaces including a facial feature space representing a face of the image, a clothing feature space representing clothing features in association with the face of the image, and a time feature space representing a time when the image was taken; and partially clustering the collection of images into clusters with a deliberate bias toward finding evident clusters, each evident cluster including images that reliably contain a face of a person associated with that cluster, the evident clusters having a uniform background distribution for noisy data; labeling the clusters; and iteratively proposing subsequent clusters that when user-labeled, minimize the number of user interactions for labeling the remaining unlabeled images based on global system entropy and subset-saliency entropy.

Claim 23

Independent Claim 161 words

A method performed by a computing device, the method comprising: automatically grouping some images of a collection of images into clusters, the grouping comprising: using spectral embedding to find representation of face of an image in a plurality of feature spaces, the plurality of feature spaces including a facial feature space representing a face of the image, a clothing feature space representing clothing features in association with the face of the image, and a time feature space representing a time when the image was taken; and partially clustering the collection of images into clusters with a deliberate bias toward finding evident clusters, each evident cluster including images that reliably contain a face of a person associated with that cluster, the evident clusters having a uniform background distribution for noisy data; labeling the clusters; and iteratively proposing subsequent clusters that when user-labeled, minimize the number of user interactions for labeling the remaining unlabeled images based on global system entropy and subset-saliency entropy.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2014-09-05	2015-03-07	2015-09-06	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2018-09-05	2019-03-07	2019-09-06	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2022-09-05	2023-03-07	2023-09-06	$7700.00	Window Closed / Lapsed

Face annotation framework with partial clustering and interactive labeling

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag