Patent US20120076427A1 successfully refreshed from Google Patents.
US6760488B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for generating a three-dimensional model from a two-dimensional image sequence
Assignee: Carnegie Mellon University
•
Filing: 2000-07-12
•
Granted: 2004-07-06
•
Expires: 2020-07-12
Abstract
Abstract A system for generating a three-dimensional model of an object from a two-dimensional image sequence. According to one embodiment, the system includes an image sensor for capturing a sequence of two-dimensional images of a scene, the scene including the object, a two-dimensional motion filter module in communication with the image sensor for determining from the sequence of images a plurality of two-dimensional motion parameters for the object, and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the set of two-dimensional motion parameters using a rank 1 factorization of a matrix.
Inventors
- 👤 Jose' M. F. Moura
- 👤 Pedro M. Q. Aguiar
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:39
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/50; G06T7/55; G06T7/579
Claims Analysis
Total Claims: 63
Claim 1
Independent Claim
105 words
A system for generating a three-dimensional model of an object from a two-dimensional image sequence, comprising: an image sensor for capturing a sequence of two-dimensional images of a scene, the scene including the object; a two-dimensional motion filter module in communication with the image sensor for determining from the sequence of images a plurality of two-dimensional motion parameters for the object; and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the set of two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 2
Independent Claim
105 words
A system for generating a three-dimensional model of an object from a two-dimensional image sequence, comprising: an image sensor for capturing a sequence of two-dimensional images of a scene, the scene including the object; a two-dimensional motion filter module in communication with the image sensor for determining from the sequence of images a plurality of two-dimensional motion parameters for the object; and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the set of two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 3
Dependent Claim
14 words
The system of claim 1 , wherein the image sensor includes a digital camera.
Claim 4
Dependent Claim
39 words
The system of claim 1 , wherein the image sensor includes an analog camera, and further comprising a frame grabber connected between the image sensor and the two-dimensional motion filter module for digitizing images captured by the image sensor.
Claim 5
Dependent Claim
65 words
The system of claim 1 , wherein: the two-dimensional motion filter module is for determining a plurality of two-dimensional motion parameters for each of a plurality of surface patches of the object; and the three-dimensional structure recovery module is for estimating the set of three-dimensional shape parameters from the set of two-dimensional motion parameters for each of the plurality of surface patches of the object.
Claim 6
Dependent Claim
68 words
The system of claim 1 , wherein: the two-dimensional motion filter module is further for assigning confidence weights to certain of the two-dimensional motion parameters; and the three-dimensional structure recovery module is for estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters from the set of two-dimensional motion parameters using a weighted rank 1 factorization of a matrix based on the confidence weights.
Claim 7
Dependent Claim
42 words
The system of claim 1 , further comprising a texture recovery module in communication with the three-dimensional structure recovery module for recovering a set of texture parameters for the object based on the set of three-dimensional shape parameters and three-dimensional motion parameters.
Claim 8
Dependent Claim
30 words
The system of claim 1 , further comprising a three-dimensional shape refinement module in communication with the three-dimensional structure recovery module for generating a refined set of three-dimensional shape parameters.
Claim 9
Dependent Claim
35 words
The system of claim 7 , wherein the three-dimensional shape refinement module is for generating the refined set of three-dimensional shape parameters using an iterative multiresolution continuation method based on a maximum likelihood estimation technique.
Claim 10
Dependent Claim
64 words
The system of claim 7 , further comprising a texture recovery module in communication with the three-dimensional structure recovery module and the three-dimensional shape refinement module for recovering a set of texture parameters for the object based on the refined set of three-dimensional shape parameters determined by the three-dimensional shape refinement module and the three-dimensional motion parameters determined by the three-dimensional structure recovery module.
Claim 11
Independent Claim
72 words
A method for generating a three-dimensional model of an object from a two-dimensional image sequence, comprising: capturing a sequence of images of a scene, the scene including the object; determining a plurality of two-dimensional motion parameters for the object from the sequence of images; and estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 12
Independent Claim
72 words
A method for generating a three-dimensional model of an object from a two-dimensional image sequence, comprising: capturing a sequence of images of a scene, the scene including the object; determining a plurality of two-dimensional motion parameters for the object from the sequence of images; and estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 13
Dependent Claim
18 words
The method of claim 10 , wherein capturing includes capturing a sequence of digitized images of the scene.
Claim 14
Dependent Claim
34 words
The method of claim 10 , wherein capturing includes capturing a sequence of analog images of the scene, and further comprising digitizing the analog images prior to determining the plurality of two-dimensional motion parameters.
Claim 15
Dependent Claim
69 words
The method of claim 10 , wherein: determining includes determining a plurality of two-dimensional motion parameters from the digitized image sequence for each of a plurality of surface patches of the object; and estimating a set of three-dimensional shape parameters includes estimating a set of three-dimensional shape parameters for each of the plurality of surface patches from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 16
Dependent Claim
68 words
The method of claim 10 , further comprising assigning confidence weights to certain of the two-dimensional motion parameters, and wherein estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters includes estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a weighted rank 1 factorization of a matrix based on the confidence weights.
Claim 17
Dependent Claim
32 words
The method of claim 10 , further comprising generating a set of texture parameters for the object based on the set of three-dimensional shape parameters and the set of three-dimensional motion parameters.
Claim 18
Dependent Claim
23 words
The method of claim 10 , further comprising generating a refined set of three-dimensional shape parameters from the set of three-dimensional shape parameters.
Claim 19
Dependent Claim
37 words
The method of claim 10 , wherein generating a refined set of three-dimensional shape parameters includes generating the refined set of three-dimensional shape parameters using an iterative multiresolution continuation method based on a maximum likelihood estimation technique.
Claim 20
Dependent Claim
26 words
The method of claim 16 , further comprising determining a set of texture parameters for the object based on the refined set of three-dimensional shape parameters.
Claim 21
Independent Claim
71 words
A method for generating a three-dimensional model of an object from a sequence of digitized images of a scene, the scene including the object, comprising: determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images of the scene; and estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 22
Independent Claim
71 words
A method for generating a three-dimensional model of an object from a sequence of digitized images of a scene, the scene including the object, comprising: determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images of the scene; and estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 23
Dependent Claim
69 words
The method of claim 19 , wherein: determining includes determining a plurality of two-dimensional motion parameters from the digitized image sequence for each of a plurality of surface patches of the object; and estimating a set of three-dimensional shape parameters includes estimating a set of three-dimensional shape parameters for each of the plurality of surface patches from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 24
Dependent Claim
68 words
The method of claim 19 , further comprising assigning confidence weights to certain of the two-dimensional motion parameters, and where estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters includes estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a weighted rank 1 factorization of a matrix based on the confidence weights.
Claim 25
Dependent Claim
32 words
The method of claim 19 , further comprising generating a set of texture parameters for the object based on the set of three-dimensional shape parameters and the set of three-dimensional motion parameters.
Claim 26
Dependent Claim
23 words
The method of claim 19 , further comprising generating a refined set of three-dimensional shape parameters from the set of three-dimensional shape parameters.
Claim 27
Independent Claim
88 words
A device for generating a three-dimensional model of an object from a sequence of digitized images of a scene, the scene including the object, comprising: a two-dimensional motion filter module for determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images; and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 28
Independent Claim
88 words
A device for generating a three-dimensional model of an object from a sequence of digitized images of a scene, the scene including the object, comprising: a two-dimensional motion filter module for determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images; and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 29
Dependent Claim
73 words
The device of claim 24 , wherein: the two-dimensional motion filter module is for determining a plurality of two-dimensional motion parameters for each of a plurality of surface patches of the object; and the three-dimensional structure recovery module is for estimating the set of three-dimensional shape parameters from the set of two-dimensional motion parameters for each of the plurality of surface patches of the object using a rank 1 factorization of a matrix.
Claim 30
Dependent Claim
66 words
The device of claim 24 , wherein: the two-dimensional motion filter module is further for assigning confidence weights to certain of the two-dimensional motion parameters; and the three-dimensional structure recovery module is for estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters from the two-dimensional motion parameters using a weighted rank 1 factorization of a matrix based on the confidence weights.
Claim 31
Dependent Claim
42 words
The device of claim 24 , further comprising a texture recovery module in communication with the three-dimensional structure recovery module for recovering a set of texture parameters for the object based on the set of three-dimensional shape parameters and three-dimensional motion parameters.
Claim 32
Dependent Claim
30 words
The device of claim 24 , further comprising a three-dimensional shape refinement module in communication with the three-dimensional structure recovery module for generating a refined set of three-dimensional shape parameters.
Claim 33
Independent Claim
75 words
A device for generating a three-dimensional model of an object from a sequence of digitized images of a scene, the scene including the object, comprising: means for determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images of the scene; and means for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 34
Independent Claim
75 words
A device for generating a three-dimensional model of an object from a sequence of digitized images of a scene, the scene including the object, comprising: means for determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images of the scene; and means for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 35
Dependent Claim
84 words
The device of claim 29 , wherein: the means for determining includes means for determining a plurality of two-dimensional motion parameters for each of a plurality of surface patches of the object; and the means for estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters includes means for estimating the set of three-dimensional shape parameters from the two-dimensional motion parameters for each of the plurality of surface patches of the object using a rank 1 factorization of a matrix.
Claim 36
Dependent Claim
75 words
The device of claim 29 , further comprising means for assigning confidence weights to certain of the two-dimensional motion parameters, and wherein the means for estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters includes means for estimating the set of three-dimensional shape parameters and the set of three-dimensional motion parameters from the two-dimensional motion parameters using a weighted rank 1 factorization of a matrix based on the confidence weights.
Claim 37
Dependent Claim
31 words
The device of claim 29 , further comprising means for recovering a set of texture parameters for the object based on the set of three-dimensional shape parameters and three-dimensional motion parameters.
Claim 38
Dependent Claim
18 words
The device of claim 29 , further comprising means for generating a refined set of three-dimensional shape parameters.
Claim 39
Dependent Claim
70 words
A computer-readable medium, having stored thereon instructions, which when executed by a processor, cause the processor to: determine a plurality of two-dimensional motion parameters for an object from a sequence of digitized images of a scene, wherein the scene includes the object; and estimate a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 40
Dependent Claim
70 words
A computer-readable medium, having stored thereon instructions, which when executed by a processor, cause the processor to: determine a plurality of two-dimensional motion parameters for an object from a sequence of digitized images of a scene, wherein the scene includes the object; and estimate a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 41
Dependent Claim
74 words
The computer-readable medium of claim 34 , having further stored thereon instructions, which when executed by the processor, cause the processor to: determine a plurality of two-dimensional motion parameters from the digitized image sequence for each of a plurality of surface patches of the object; and estimate a set of three-dimensional shape parameters for each of the plurality of surface patches from the two-dimensional motion parameters using a rank 1 factorization of a matrix.
Claim 42
Dependent Claim
66 words
The computer-readable medium of claim 34 , having further stored thereon instructions, which when executed by the processor, cause the processor to: assign confidence weights to certain of the two-dimensional motion parameters; and estimate a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a weighted rank 1 factorization of a matrix based on the confidence weights.
Claim 43
Dependent Claim
46 words
The computer-readable medium of claim 34 , having further stored thereon instructions, which when executed by the processor, cause the processor to generate a set of texture parameters for the object based on the set of three-dimensional shape parameters and the set of three-dimensional motion parameters.
Claim 44
Dependent Claim
37 words
The computer-readable medium of claim 34 , having further stored thereon instructions, which when executed by the processor, cause the processor to generate a refined set of three-dimensional shape parameters from the set of three-dimensional shape parameters.
Claim 45
Dependent Claim
69 words
A computer-readable medium, having stored thereon instructions, which when executed by a processor, cause the processor to: generate a matrix of two-dimensional motion parameters, wherein the two-dimensional motion parameters describe motion of an object in an image sequence; and estimate a set of three-dimensional shape parameters and a set of three-dimensional motion parameters for the object from the two-dimensional motion parameters using a rank 1 factorization of the matrix.
Claim 46
Dependent Claim
69 words
A computer-readable medium, having stored thereon instructions, which when executed by a processor, cause the processor to: generate a matrix of two-dimensional motion parameters, wherein the two-dimensional motion parameters describe motion of an object in an image sequence; and estimate a set of three-dimensional shape parameters and a set of three-dimensional motion parameters for the object from the two-dimensional motion parameters using a rank 1 factorization of the matrix.
Claim 47
Dependent Claim
64 words
The computer-readable medium of claim 39 , having further stored thereon instructions, which when executed by the processor, cause the processor to: assign confidence weights to certain of the two-dimensional motion parameters; and estimate the set of three-dimensional shape parameters and the set of three-dimensional motion parameters from the matrix using a weighted rank 1 factorization of the matrix based on the confidence weights.
Claim 48
Dependent Claim
46 words
The computer-readable medium of claim 39 , having further stored thereon instructions, which when executed by the processor, cause the processor to generate a set of texture parameters for the object based on the set of three-dimensional shape parameters and the set of three-dimensional motion parameters.
Claim 49
Dependent Claim
37 words
The computer-readable medium of claim 39 , having further stored thereon instructions, which when executed by the processor, cause the processor to generate a refined set of three-dimensional shape parameters from the set of three-dimensional shape parameters.
Claim 50
Dependent Claim
62 words
A three-dimensional structure recovery module, comprising: means for generating a matrix of two-dimensional motion parameters, wherein the two-dimensional motion parameters describe motion of an object in an image sequence; and means for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters for the object from the two-dimensional motion parameters using a rank 1 factorization of the matrix.
Claim 51
Dependent Claim
62 words
A three-dimensional structure recovery module, comprising: means for generating a matrix of two-dimensional motion parameters, wherein the two-dimensional motion parameters describe motion of an object in an image sequence; and means for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters for the object from the two-dimensional motion parameters using a rank 1 factorization of the matrix.
Claim 52
Dependent Claim
75 words
The three-dimensional recovery module of claim 43 , wherein certain of the two-dimensional motion parameters have confidence weights assigned thereto, and wherein the means for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters includes means for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters from the two-dimensional motion parameters using a weighted rank 1 factorization of the matrix based on the confidence weights.
Claim 53
Independent Claim
122 words
A method for generating a three-dimensional model of an object from a two-dimensional image sequence of a scene including the object, wherein the object includes at least one occlusion, comprising: generating a matrix of two-dimensional motion parameters for the object from the sequence of images; generating an initial estimate of a set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object based a submatrix of the matrix of two-dimensional motion parameters, wherein the submatrix includes no missing elements; and iteratively re-estimating the set three-dimensional shape parameters and the set of three-dimensional motion parameters based the initial estimate to generate a final estimate of the set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object.
Claim 54
Independent Claim
122 words
A method for generating a three-dimensional model of an object from a two-dimensional image sequence of a scene including the object, wherein the object includes at least one occlusion, comprising: generating a matrix of two-dimensional motion parameters for the object from the sequence of images; generating an initial estimate of a set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object based a submatrix of the matrix of two-dimensional motion parameters, wherein the submatrix includes no missing elements; and iteratively re-estimating the set three-dimensional shape parameters and the set of three-dimensional motion parameters based the initial estimate to generate a final estimate of the set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object.
Claim 55
Dependent Claim
40 words
The method of claim 45 , wherein generating an initial estimate includes generating an initial estimate of the set three-dimensional shape parameters and the set of three-dimensional motion parameters of the object using a rank 1 factorization of the submatrix.
Claim 56
Dependent Claim
39 words
The method of claim 45 , wherein iteratively re-estimating includes: computing unknown three-dimensional shape parameters for the object based on known three-dimensional motion parameters; and computing unknown three-dimensional motion parameters for the object based on known three-dimensional shape parameters.
Claim 57
Independent Claim
160 words
A device for generating a three-dimensional model of an object from a sequence of digitized images of a scene including the object, wherein the object includes at least one occlusion, comprising: a two-dimensional motion filter module for determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images; and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for: generating a matrix of two-dimensional motion parameters for the object from the sequence of images; generating an initial estimate of a set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object based on a submatrix of the matrix of two-dimensional motion parameters, wherein the submatrix includes no missing elements; and iteratively re-estimating the set three-dimensional shape parameters and the set of three-dimensional motion parameters based the initial estimate to generate a final estimate of the set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object.
Claim 58
Independent Claim
160 words
A device for generating a three-dimensional model of an object from a sequence of digitized images of a scene including the object, wherein the object includes at least one occlusion, comprising: a two-dimensional motion filter module for determining a plurality two-dimensional motion parameters for the object from the sequence of digitized images; and a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for: generating a matrix of two-dimensional motion parameters for the object from the sequence of images; generating an initial estimate of a set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object based on a submatrix of the matrix of two-dimensional motion parameters, wherein the submatrix includes no missing elements; and iteratively re-estimating the set three-dimensional shape parameters and the set of three-dimensional motion parameters based the initial estimate to generate a final estimate of the set three-dimensional shape parameters and a set of three-dimensional motion parameters for the object.
Claim 59
Dependent Claim
39 words
The device of claim 48 , wherein the three-dimensional structure recovery module is for generating an initial estimate of a set three-dimensional shape parameters and a set of three-dimensional motion parameters using a rank 1 factorization of the submatrix.
Claim 60
Dependent Claim
87 words
A communications network, comprising: a two-dimensional motion filter module for determining a plurality two-dimensional motion parameters for an object from a sequence of digitized images of a scene including the object; a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters for the object from the two-dimensional motion parameters using a rank 1 factorization of a matrix; and an end-user terminal in communication with the three-dimensional structure recovery module.
Claim 61
Dependent Claim
87 words
A communications network, comprising: a two-dimensional motion filter module for determining a plurality two-dimensional motion parameters for an object from a sequence of digitized images of a scene including the object; a three-dimensional structure recovery module in communication with the two-dimensional motion filter module for estimating a set of three-dimensional shape parameters and a set of three-dimensional motion parameters for the object from the two-dimensional motion parameters using a rank 1 factorization of a matrix; and an end-user terminal in communication with the three-dimensional structure recovery module.
Claim 62
Dependent Claim
24 words
The communications network of claim 50 , wherein the end-user terminal is in communication with the three-dimensional structure recovery module via an IP-based network.
Claim 63
Dependent Claim
34 words
The communications network of claim 50 , wherein the end-user terminal is in communication with the three-dimensional structure recovery module via at least one of a wireless communications link and a wireline communications link.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-07-06 | 2008-01-05 | 2008-07-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-07-06 | 2012-01-05 | 2012-07-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-07-06 | 2016-01-05 | 2016-07-06 | $7700.00 | Window Closed / Lapsed |