US6400828B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Canonical correlation analysis of image/control-point location coupling for the automatic location of control points
Assignee: Elet Systems LLC
•
Filing: 2001-02-13
•
Granted: 2002-06-04
•
Expires: 2021-02-13
Abstract
Abstract The identification of hidden data, such as feature-based control points in an image, from a set of observable data, such as the image, is achieved through a two-stage approach. The first stage involves a learning process, in which a number of sample data sets, e.g. images, are analyzed to identify the correspondence between observable data, such as visual aspects of the image, and the desired hidden data, such as the control points. Two models are created. A feature appearance-only model is created from aligned examples of the feature in the observed data. In addition, each labeled data set is processed to generate a coupled model of the aligned observed data and the associated hidden data. In the second stage of the process, the modeled feature is located in an unmarked, unaligned data set, using the feature appearance-only model. This location is used as an alignment point and the coupled model is then applied to the aligned data, giving an estimate of the hidden data values for that data set.
Inventors
- 👤 Michele Covell
- 👤 Malcolm Slaney
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:27
CPC Classifications
G; G06; G06V; G06V40/00; G06V40/10; G06V40/16; G06V40/168; G06V40/171; G06V10/00; G06V10/40; G06V10/44; G06V10/443; G06V10/46; G06V10/70; G06V10/74; G06V10/75; G06V10/757; G06V10/469; G06V10/476
Claims Analysis
Total Claims: 36
Claim 1
Independent Claim
161 words
A method for determining continuous-valued hidden data from observable data, comprising the steps of: A) conducting a training stage which includes the steps of: labelling a plurality of representative sets of unaligned observed data to identify correct alignment of the observed data and continuous-valued hidden data associated with each set of observed data; analyzing the observed data to generate a first model which represents the aligned observed data; generating a second model on the aligned and labeled data sets which explicitly represents the coupling between aligned observable data and the hidden data; B) for each set of unlabeled data, conducting a labelling stage which includes the steps of: analyzing the unlabeled set of unaligned observed data by means of the first model to determine alignment of the observable data associated therewith; applying the second model to said unlabeled set of aligned observed data; and determining hidden data for the unlabeled set of aligned data from said application of the second model.
Claim 2
Independent Claim
161 words
A method for determining continuous-valued hidden data from observable data, comprising the steps of: A) conducting a training stage which includes the steps of: labelling a plurality of representative sets of unaligned observed data to identify correct alignment of the observed data and continuous-valued hidden data associated with each set of observed data; analyzing the observed data to generate a first model which represents the aligned observed data; generating a second model on the aligned and labeled data sets which explicitly represents the coupling between aligned observable data and the hidden data; B) for each set of unlabeled data, conducting a labelling stage which includes the steps of: analyzing the unlabeled set of unaligned observed data by means of the first model to determine alignment of the observable data associated therewith; applying the second model to said unlabeled set of aligned observed data; and determining hidden data for the unlabeled set of aligned data from said application of the second model.
Claim 3
Dependent Claim
16 words
The method of claim 1 wherein said second model is generated by means of canonical correlation.
Claim 4
Dependent Claim
15 words
The method of claim 1 wherein each set of unaligned observed data defines an image.
Claim 5
Dependent Claim
22 words
The method of claim 3 wherein said hidden data comprises control points which relate to fiduciary points on objects in an image.
Claim 6
Dependent Claim
25 words
The method of claim 4 wherein at least some of said control points relate to fiduciary points on obscured portions of objects in the images.
Claim 7
Dependent Claim
34 words
The method of claim 4 wherein control points are determined for at least two new images, and further including the step of morphing between said new images in accordance with the determined control points.
Claim 8
Dependent Claim
35 words
The method of claim 4 further including the step of creating a composite image by incorporating a new image into another image by means of the determined control points for each of the two images.
Claim 9
Dependent Claim
28 words
The method of claim 4 wherein said images include faces, and further including the step of analyzing the control points to recognize a known face in an image.
Claim 10
Dependent Claim
10 words
The method of claim 4 wherein said images comprise cartoons.
Claim 11
Dependent Claim
30 words
The method of claim 4 wherein said images include faces, and further including the step of analyzing the control points to recognize an expression on a face in an image.
Claim 12
Dependent Claim
33 words
The method of claim 4 , further including the step of controlling a robot to grasp an object in accordance with the fiduciary points that are labeled in the image of the object.
Claim 13
Dependent Claim
18 words
The method of claim 1 wherein said sets of unaligned observed data comprise a sequence of video images.
Claim 14
Dependent Claim
29 words
The method of claim 12 further including the step of analyzing determined control points in said sequence of video images to recognize movement of an object in the images.
Claim 15
Dependent Claim
12 words
The method of claim 13 wherein said movement comprises nearly periodic motion.
Claim 16
Dependent Claim
15 words
The method of claim 1 wherein said sets of unaligned observed data comprise audio signals.
Claim 17
Dependent Claim
72 words
The method of claim 1 further including the steps of selecting a plurality of said representative sets of data, using hidden data in said plurality of data sets to automatically generate interpolated data sets that are based on said plurality of data sets and that include both observable and hidden data, and including said interpolated data sets in the plurality of representative data sets that are analyzed to generate said second model.
Claim 18
Dependent Claim
28 words
The method of claim 16 wherein said second model is a multifaceted model, and said interpolated data sets are at the boundaries of facets in said second model.
Claim 19
Dependent Claim
69 words
The method of claim 1 further including the steps of selecting a plurality of said representative sets of data, using hidden data in said plurality of data sets to automatically generate interpolated data sets that are based on said plurality of data sets and that contain observable data, and including said interpolated data sets in the plurality of representative data sets that are analyzed to generate said first model.
Claim 20
Dependent Claim
28 words
The method of claim 18 wherein said first model is a multifaceted model, and said interpolated data sets are at the boundaries of facets in said first model.
Claim 21
Dependent Claim
18 words
The method of claim 1 wherein said applying and determining steps are carried out in a non-iterative manner.
Claim 22
Dependent Claim
13 words
The method of claim 1 wherein said first model is a manifold model.
Claim 23
Dependent Claim
14 words
The method of claim 21 wherein said first model is an affine manifold model.
Claim 24
Dependent Claim
160 words
The method of claim 21 wherein the step of aligning the observed data in an unlabeled data set comprises the steps of: i) selecting possible locations for the alignment of the data; ii) for each possible location, determining a lower bound for the distance between the unlabeled data set aligned at that location and an expected appearance of aligned data, in accordance with an average appearance defined by the first model; iii) removing the possible locations whose lower bound exceeds a threshold value; iv) for each possible location, determining the coordinate value for a dimension of the first model; v) for each possible location, determining a new lower bound by combining previously determined coordinate values with the distance between the data set aligned at that location and the appearance of the data set under said alignment in accordance with the previously determined coordinate values; and vi) repeating steps iii), iv) and v) for all of the dimensions of the model.
Claim 25
Dependent Claim
25 words
The method of claim 23 wherein said lower bounds are determined in accordance with expected variances along each of the dimensions of the manifold model.
Claim 26
Dependent Claim
19 words
The method of claim 24 wherein said expected variances are progressively smaller on each successive repetition of said steps.
Claim 27
Dependent Claim
82 words
The method of claim 22 wherein the step of applying the second model to the unlabeled set of aligned observed data includes projecting, with the use of an orthonormal transform, the aligned observed unlabeled data onto a subspace of the second model having fewer dimensions than said second model; performing a general matrix multiplication within said subspace; and projecting, with the use of an orthonormal transform, into a second space of the model to determine hidden data for the unlabeled data set.
Claim 28
Dependent Claim
41 words
The method of claim 26 wherein said general matrix multiplication is determined, in part, according to a gradual roll-off in manifold dimensions according to the coherence between the hidden and aligned observed data that is used to generate said second model.
Claim 29
Dependent Claim
36 words
The method of claim 1 further including the step defining the alignment of the observed data in the representative sets of data from an analysis of the hidden data with which the data sets are labeled.
Claim 30
Dependent Claim
19 words
The method of claim 28 wherein an analysis of the observed data is also employed in said alignment process.
Claim 31
Dependent Claim
35 words
The method of claim 28 wherein said defining step comprises dividing the hidden data into separate groups, and assigning a different definition of aligned observed data in each representative data set to the respective groups.
Claim 32
Dependent Claim
25 words
The method of claim 30 wherein the division of the hidden data into separate groups is determined in accordance with analysis of the hidden data.
Claim 33
Dependent Claim
22 words
The method of claim 30 wherein the definition of aligned observed data is determined in accordance with analysis of the hidden data.
Claim 34
Dependent Claim
23 words
The method of claim 32 wherein the definition of aligned observed data is also determined in accordance with analysis of the observed data.
Claim 35
Dependent Claim
20 words
The method of claim 31 wherein the observed data is also used to divide the hidden data into said groups.
Claim 36
Dependent Claim
24 words
The method of claim 32 wherein the division of hidden data into groups is carried out by measuring the coherence of the hidden data.
References Cited (Backward Citations)
GB202994A
Cited
US5164992A
Cited
US6188776B1
Cited
US5963710A
Cited
US5921937A
Cited
US20010000660A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-06-03 | 2005-12-03 | 2006-06-04 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-06-03 | 2009-12-03 | 2010-06-04 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-06-03 | 2013-12-03 | 2014-06-04 | $7700.00 | Window Closed / Lapsed |