US5990900A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Two-dimensional to three-dimensional image converting system
Assignee: Be There Now Inc
•
Filing: 1997-12-24
•
Granted: 1999-11-23
•
Expires: 2017-12-24
Abstract
Abstract A method, system and computer-readable medium for converting objects in two-dimensional images into three-dimensional objects. Vanishing points for an object selected from a digitized two-dimensional perspective image are determined. Then, a three-dimensional coordinate space for the selected object is derived based on the determined vanishing points. The system then generates a three-dimensional object in the generated three-dimensional coordinate space based on user-designated vertices at significant features of the selected object in the two-dimensional perspective image. The three-dimensional coordinate space and three-dimensional object are generated using non-linear optimization algorithms. The system extracts surface texture information from the two-dimensional images and maps the extracted texture information onto the generated three-dimensional object. The system also converts multiple three-dimensional objects created from two-dimensional images of a single object into a single three-dimensional object. A set of conjugate vertices, vanishing lines and/or planes sufficient for the determination of the relative orientations of two or more precreated three-dimensional objects are identified between precreated two or more three-dimensional objects. Then, the precreated two or more three-dimensional objects are merged into a single three-dimensional object based on the identified set of conjugate vertices, vanishing lines or planes and a non-linear optimization algorithm.
Inventors
- 👤 Sean W. Seago
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 10:55
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/50; G06T7/536; G06T7/70; G06T7/73; G06T7/74
Claims Analysis
Total Claims: 44
Claim 1
Independent Claim
179 words
An image converting system for converting objects in two-dimensional images of objects into three-dimensional objects, said system comprising: a processor; a memory; a display device for displaying a digitized two-dimensional image of one or more object; an input device; and a computer program stored in the memory and executed by the processor comprises: a vanishing point component for determining perspective vanishing points for a selected object; a three-dimensional coordinate space component for generating a three-dimensional coordinate space based on the determined perspective vanishing points; a graphical interface component for generating a graphical interface displayed on the display device and operable with the input device for allowing designation of vertices or vanishing lines at significant features of the selected object; a planes component for generating plane indexes for each designated vertex or vanishing line based on the determined perspective vanishing points; and a three-dimensional object component for generating a three-dimensional object in the generated three-dimensional coordinate space based on the designated vertices and the generated plane indexes of the selected object, wherein the display device displays the generated three-dimensional object.
Claim 2
Dependent Claim
179 words
An image converting system for converting objects in two-dimensional images of objects into three-dimensional objects, said system comprising: a processor; a memory; a display device for displaying a digitized two-dimensional image of one or more object; an input device; and a computer program stored in the memory and executed by the processor comprises: a vanishing point component for determining perspective vanishing points for a selected object; a three-dimensional coordinate space component for generating a three-dimensional coordinate space based on the determined perspective vanishing points; a graphical interface component for generating a graphical interface displayed on the display device and operable with the input device for allowing designation of vertices or vanishing lines at significant features of the selected object; a planes component for generating plane indexes for each designated vertex or vanishing line based on the determined perspective vanishing points; and a three-dimensional object component for generating a three-dimensional object in the generated three-dimensional coordinate space based on the designated vertices and the generated plane indexes of the selected object, wherein the display device displays the generated three-dimensional object.
Claim 3
Dependent Claim
220 words
The system of claim 1, wherein the three-dimensional coordinate space component of the computer program further comprises: a default setting component for setting a three-dimensional coordinate space at a default orientation relative to the image plane and setting a viewpoint for the image at a default focal length, wherein the origin of the set three-dimensional coordinate space is at the center of the image and wherein the set three-dimensional coordinate space includes left, right and vertical axes that correspond to the left, right and vertical vanishing points, respectively; an axis point component for projecting axis endpoints of the set three-dimensional coordinate space to the image plane; a comparison component for comparing the image plane location of each calculated vanishing point to a corresponding projected axis endpoint; an error function component for determining an error function based on the comparisons and determining if the determined error function is at an acceptable level; an adjusting component for adjusting the orientation of the three-dimensional coordinate space and the focal length of the image's viewpoint based on the determined error function; a repeat component for repeating the functions performed by the above components if the error function is not at an acceptable level; and a saving component for saving the three-dimensional coordinate space in memory, if the error function is at an acceptable level.
Claim 4
Dependent Claim
203 words
The system of claim 1, wherein the three-dimensional object component of the computer program further comprises: a planes setting component for setting planes at a default position, wherein the set planes are referenced to the generated three-dimensional coordinate space; a planes intersection points component for determining the set planes' intersection points in the generated three-dimensional coordinate space according to corresponding designated vertices; a projecting component for projecting the determined intersection points that correspond to vertices to the image plane along a line of sight to a viewpoint, wherein the viewpoint is displaced from the image plane by a focal length and centered above the midpoint of the image; a comparing component for comparing the image plane location of each vertex to the corresponding projected intersection point; an error function component for determining an error function based on all the comparisons; an adjusting component for adjusting the planes' positions based on the determined error function; a repeat component for repeating the functions performed by the above components, if the error function is not at an acceptable level; and a saving component for saving the three-dimensional object that is defined by the intersection of the planes, if the error function is at an acceptable level.
Claim 5
Dependent Claim
85 words
The system of claim 1, wherein the computer program further comprises: a resolution determining component for determining the desired resolution level of the three-dimensional objects' texture; an incrementing component for determining angular increments for vanishing lines based on the determined desired resolution level; a sampling component for sampling image information along vanishing lines based on the determined angular increments; and a mapping component for mapping the sampled image information onto the generated three-dimensional object based on the image information location within the corresponding determined shape.
Claim 6
Dependent Claim
39 words
The system of claim 1, wherein the displayed graphical interface comprises a perspective cross hair cursor, wherein the perspective cross hair cursor comprises lines that intersect at a user controlled cursor and extend to each of the vanishing points.
Claim 7
Dependent Claim
34 words
The system of claim 1, wherein the displayed graphical interface further comprises arrows that appear attached to a user controlled cursor when the cursor is co-located with vanishing lines of any previously designated vertex.
Claim 8
Dependent Claim
106 words
An image converting method for converting objects in two-dimensional images into three-dimensional digital objects, said method comprising: displaying a digitized two-dimensional image of one or more object; determining perspective vanishing points for a selected object; generating a three-dimensional coordinate space based on the determined vanishing points; generating a graphical interface for allowing designation of vertices or vanishing lines at significant features of the selected object; generating plane indexes for each designated vertex or vanishing line based on the determined perspective vanishing points; and generating a three-dimensional object in the generated three-dimensional coordinate space based on the designated vertices and the generated planes of the selected object.
Claim 9
Dependent Claim
106 words
An image converting method for converting objects in two-dimensional images into three-dimensional digital objects, said method comprising: displaying a digitized two-dimensional image of one or more object; determining perspective vanishing points for a selected object; generating a three-dimensional coordinate space based on the determined vanishing points; generating a graphical interface for allowing designation of vertices or vanishing lines at significant features of the selected object; generating plane indexes for each designated vertex or vanishing line based on the determined perspective vanishing points; and generating a three-dimensional object in the generated three-dimensional coordinate space based on the designated vertices and the generated planes of the selected object.
Claim 10
Dependent Claim
192 words
The method of claim 7, wherein the generating of a three-dimensional coordinate space based on the determined vanishing points further comprises: a) setting a three-dimensional coordinate space at a default orientation relative to the image plane and setting a viewpoint at a default focal length for the image, wherein the origin of the set three-dimensional coordinate space is at the center of the image and wherein the set three-dimensional coordinate space includes left, right and vertical axes that correspond to the left, right and vertical vanishing points, respectively; b) projecting axis endpoints of the set three-dimensional coordinate space to the image plane; c) comparing the image plane location of each calculated vanishing point to a corresponding projected axis endpoint; d) determining an error function based on the comparisons; e) determining if the determined error function is at an acceptable level; f) adjusting the orientation of the three-dimensional coordinate space and the focal length of the image's viewpoint based on the determined error function and repeating b)-f), if the error function is not at an acceptable level; and g) saving the three-dimensional coordinate space, if the error function is at an acceptable level.
Claim 11
Dependent Claim
176 words
The method of claim 7, wherein the generating of a three-dimensional object in the generated three-dimensional coordinate space further comprises: a) setting planes at a default position, wherein the set planes are referenced to the generated three-dimensional coordinate space; b) determining the set planes' intersection points in the generated three-dimensional coordinate space according to corresponding designated vertices; c) projecting the determined intersection points that correspond to the designated vertices to the image plane along a line of sight to a viewpoint, wherein the viewpoint is displaced from the image plane by a focal length and centered above the midpoint of the image; d) comparing the image plane location of each placed vertex to the corresponding projected intersection point; e) determining an error function based on all the comparisons; f) adjusting the planes' positions based on the determined error function and repeating b)-f), if the error function is not at an acceptable level; and g) saving the three-dimensional object that is defined by the intersection of the planes, if the error function is at an acceptable level.
Claim 12
Dependent Claim
66 words
The method of claim 7, further comprising: determining the desired resolution level of the texture for the three-dimensional object; determining angular increments for vanishing lines based on the determined desired resolution level; sampling image information along vanishing lines based on the determined angular increments; and mapping the sampled image information onto the generated three-dimensional object based on the image information location within the corresponding determined shape.
Claim 13
Dependent Claim
153 words
An image converting system for converting multiple three-dimensional digital objects of a single object into a single three-dimensional object, wherein each preconverted three-dimensional digital object has an associated viewpoint and two-dimensional image, said system comprising: a processor; a memory; a display device for displaying the preconverted three-dimensional objects; an input device; and a first computer program stored in the memory and executed by the processor comprises: a retrieving component for retrieving precreated two or more three-dimensional digital objects displayed in separate images; an identifying component for identifying a sufficient set of features between the precreated two or more three-dimensional objects displayed in separate images, a orientation component for determining relative orientations of two or more precreated objects within each respective image based on the identified sufficient set of features; and a merging component for merging the precreated two or more three-dimensional objects into a single three-dimensional object based on the determined relative orientations.
Claim 14
Dependent Claim
153 words
An image converting system for converting multiple three-dimensional digital objects of a single object into a single three-dimensional object, wherein each preconverted three-dimensional digital object has an associated viewpoint and two-dimensional image, said system comprising: a processor; a memory; a display device for displaying the preconverted three-dimensional objects; an input device; and a first computer program stored in the memory and executed by the processor comprises: a retrieving component for retrieving precreated two or more three-dimensional digital objects displayed in separate images; an identifying component for identifying a sufficient set of features between the precreated two or more three-dimensional objects displayed in separate images, a orientation component for determining relative orientations of two or more precreated objects within each respective image based on the identified sufficient set of features; and a merging component for merging the precreated two or more three-dimensional objects into a single three-dimensional object based on the determined relative orientations.
Claim 15
Dependent Claim
22 words
The system of claim 11, wherein the set of features includes any combination of conjugate sets of vertices, vanishing lines, or planes.
Claim 16
Dependent Claim
53 words
The system of claim 11, wherein said merging component further comprises: generating a single three-dimensional object in world coordinate space; and manipulating the generated three-dimensional object in world coordinate space based on the determined relative orientations and manipulating the position, orientation, and focal length of each two-dimensional images' viewpoint in world coordinate space.
Claim 17
Dependent Claim
223 words
The system of claim 13, wherein manipulating the generated three-dimensional object in world coordinate space further comprises: a) setting planes as a default position, wherein the set planes are referenced to the world coordinate space; b) setting each image's viewpoint to a default focal length, orientation, and position in the world coordinate space; c) determining the set planes' intersection points in the world coordinate space; d) projecting the determined intersection points that correspond to vertices onto each image's image plane via a direct line to each image's viewpoint using each image's viewpoint position, orientation, and focal length, and determining the projected vertices' two-dimensional values on the respective image planes; e) comparing at each image plane of the multiple two-dimensional images each vertex of the single objects to its corresponding determined projected vertex's determined two-dimensional value; f) determining an error function based on all the comparisons; g) adjusting the position of the planes of the generated three-dimensional object and the position of the viewpoint in world coordinate space and orientation and focal length of the viewpoint of each two-dimensional image based on the determined error function and repeating e)-g), if the error function is not at an acceptable level; and h) saving the merged three-dimensional object that is defined by the intersection of the planes, if the error function is at an acceptable level.
Claim 18
Dependent Claim
26 words
The system of claim 13, further comprising a mapping component for mapping image information from the two or more three-dimensional objects onto the merged three-dimensional object.
Claim 19
Dependent Claim
191 words
The system of claim 11, wherein the precreated two or more three-dimensional objects are precreated by system for converting objects in digitized two-dimensional images into three-dimensional objects, said system comprising: a processor; a memory; a display device for displaying a digitized two-dimensional image of one or more object; an input device; and a second computer program stored in the memory and executed by the processor comprises: a vanishing point component for determining perspective vanishing points for a selected object; a three-dimensional coordinate space component for generating a three-dimensional coordinate space based on the determined perspective vanishing points; a graphical interface component for generating a graphical interface displayed on the display device and operable with the input device for allowing designation of vertices or vanishing lines at significant features of the selected object; a planes component for generating plane indexes for each designated vertex or vanishing line based on the determined perspective vanishing points; and a three-dimensional object component for generating a three-dimensional object in the generated three-dimensional coordinate space based on the designated vertices and the generated planes of the selected object, wherein the display device displays the generated three-dimensional object.
Claim 20
Dependent Claim
231 words
The system for converting objects in digitized two-dimensional images into three-dimensional objects of claim 16, wherein the three-dimensional coordinate space component of the second computer program further comprises: a default setting component for setting a three-dimensional coordinate space at a default orientation relative to the image plane and setting a viewpoint for the image at a default focal length, wherein the origin of the set three-dimensional coordinate space is at the center of the image and wherein the set three-dimensional coordinate space includes left, right and vertical axes that correspond to the left, right and vertical vanishing points, respectively; an axis point component for projecting axis endpoints of the set three-dimensional coordinate space to the image plane; a comparison component for comparing the image plane location of each calculated vanishing point to a corresponding projected axis endpoint; an error function component for determining an error function based on the comparisons and determining if the determined error function is at an acceptable level; an adjusting component for adjusting the orientation of the three-dimensional coordinate space and the focal length of the image's viewpoint based on the determined error function; a repeat component for repeating the functions performed by the above components if the error function is not at an acceptable level; and a saving component for saving the three-dimensional coordinate space in memory, if the error function is at an acceptable level.
Claim 21
Dependent Claim
215 words
The system for converting objects in digitized two-dimensional images into three-dimensional objects of claim 16, wherein the three-dimensional object component of the second computer program further comprises: a planes setting component for setting planes at a default position, wherein the set planes are referenced to the generated three-dimensional coordinate space; a planes intersection points component for determining the set planes' intersection points in the generated three-dimensional coordinate space according to corresponding designated vertices; a projecting component for projecting the determined intersection points that correspond to designated vertices to the image plane along a line of sight to a viewpoint, wherein the viewpoint is displaced from the image plane by a focal length and centered above the midpoint of the image; a comparing component for comparing the image plane location of each vertex to the corresponding projected intersection point; an error function component for determining an error function based on all the comparisons; an adjusting component for adjusting the planes' positions based on the determined error function; a repeat component for repeating the functions performed by the above components, if the error function is not at an acceptable level; and a saving component for saving the three-dimensional object that is defined by the intersection of the planes, if the error function is at an acceptable level.
Claim 22
Dependent Claim
95 words
The system for converting objects in digitized two-dimensional images into three-dimensional objects of claim 16, wherein the computer program further comprises: a resolution determining component for determining the desired resolution level of the three-dimensional objects' texture; an incrementing component for determining angular increments for vanishing lines based on the determined desired resolution level; a sampling component for sampling image information along vanishing lines based on the determined angular increments; and a mapping component for mapping the sampled image information onto the generated three-dimensional object based on the image information location within the corresponding determined shape.
Claim 23
Dependent Claim
123 words
An image converting method for converting multiple three-dimensional digital objects of a single object into a single three-dimensional object, wherein each preconverted three-dimensional digital object has an associated viewpoint and two-dimensional image, said method comprising: retrieving precreated two or more three-dimensional digital objects displayed in separate images; identifying a sufficient set of features between the precreated two or more three-dimensional objects displayed in separate images, wherein the set of features is sufficient for determining relative orientations of two or more precreated objects; determining relative orientations of two or more precreated objects within each respective image based on the identified sufficient set of features; and merging the precreated two or more three-dimensional objects into a single three-dimensional object based on the determined relative orientations.
Claim 24
Dependent Claim
123 words
An image converting method for converting multiple three-dimensional digital objects of a single object into a single three-dimensional object, wherein each preconverted three-dimensional digital object has an associated viewpoint and two-dimensional image, said method comprising: retrieving precreated two or more three-dimensional digital objects displayed in separate images; identifying a sufficient set of features between the precreated two or more three-dimensional objects displayed in separate images, wherein the set of features is sufficient for determining relative orientations of two or more precreated objects; determining relative orientations of two or more precreated objects within each respective image based on the identified sufficient set of features; and merging the precreated two or more three-dimensional objects into a single three-dimensional object based on the determined relative orientations.
Claim 25
Dependent Claim
22 words
The method of claim 20, wherein the set of features include any combination of conjugate sets of vertices, vanishing lines or planes.
Claim 26
Dependent Claim
57 words
The method of claim 20, wherein merging comprises: generating a three-dimensional object in world coordinate space; and manipulating the generated three-dimensional object in world coordinate space based on the identified sufficient set of features and the display location of the objects displayed in the digitized two-dimensional images used to create the precreated two or more three-dimensional objects.
Claim 27
Dependent Claim
224 words
The method of claim 22, wherein manipulating the generated three-dimensional object in world coordinate space further comprises: a) setting planes as a default position, wherein the set planes are referenced to the world coordinate space; b) setting each image's viewpoint to a default focal length, orientation, and position in the world coordinate space; c) determining the set planes' intersection points in the world coordinate space; d) projecting the determined intersection points that correspond to vertices onto each image's image plane via a direct line to each image's viewpoint using each image's viewpoint position, orientation, and focal length, and determining the projected vertices' two-dimensional values on the respective image planes; e) comparing at each image plane of the multiple two-dimensional images each vertex of the single objects to its corresponding determined projected vertex's determined two-dimensional value; f) determining an error function based on all the comparisons; g) adjusting the position of the planes of the generated three-dimensional object and the position of the viewpoint in world coordinate space, and the orientation and focal length of the viewpoint of each two-dimensional image based on the determined error function and repeating e)-g), if the error function is not at an acceptable level; and h) saving the merged three-dimensional object that is defined by the intersection of the planes, if the error function is at an acceptable level.
Claim 28
Dependent Claim
22 words
The method of claim 21, further comprising mapping image information from the two or more three-dimensional objects onto the merged three-dimensional object.
Claim 29
Dependent Claim
130 words
The method of claim 20, wherein the precreated two or more three-dimensional objects are precreated by a method for converting objects in digitized two-dimensional images into three-dimensional objects, said method for converting objects in digitized two-dimensional images into three-dimensional objects comprising: displaying a digitized two-dimensional image of one or more object; determining perspective vanishing points for a selected object; generating a three-dimensional coordinate space based on the determined vanishing points; generating a graphical interface for allowing designation of vertices or vanishing lines at significant features of the selected object; generating plane indexes for each designated vertex or vanishing line based on the determined perspective vanishing points; and generating a three-dimensional object in the generated three-dimensional coordinate space based on the designated vertices and the generated planes of the selected object.
Claim 30
Dependent Claim
201 words
The method for converting objects in digitized two-dimensional images into three-dimensional objects of claim 25, wherein the generating of a three-dimensional coordinate space based on the determined vanishing points further comprises: a) setting a three-dimensional coordinate space at a default orientation relative to the image plane and setting a viewpoint at a default focal length for the image, wherein the origin of the set three-dimensional coordinate space is at the center of the image and wherein the set three-dimensional coordinate space includes left, right and vertical axes that correspond to the left, right and vertical vanishing points, respectively; b) projecting axis endpoints of the set three-dimensional coordinate space to the image plane; c) comparing the image plane location of each calculated vanishing point to a corresponding projected axis endpoint; d) determining an error function based on the comparisons; e) determining if the determined error function is at an acceptable level; f) adjusting the orientation of the three-dimensional coordinate space and focal length of the image's viewpoint based on the determined error function and repeating b)-f), if the error function is not at an acceptable level; and g) saving the three-dimensional coordinate space, if the error function is at an acceptable level.
Claim 31
Dependent Claim
184 words
The method for converting objects in digitized two-dimensional images into three-dimensional objects of claim 25, wherein the generating of a three-dimensional object in the generated three-dimensional coordinate space further comprises: a) setting planes at a default position, wherein the set planes are referenced to the generated three-dimensional coordinate space; b) determining the set planes' intersection points in the generated three-dimensional coordinate space according to corresponding designated vertices; c) projecting the determined intersection points that correspond to vertices to the image plane along a line of sight to a viewpoint, wherein the viewpoint is displaced from the image plane by a focal length and centered above the midpoint of the image; d) comparing the image plane location of each placed vertex to the corresponding projected intersection point; e) determining an error function based on all the comparisons; f) adjusting the planes' positions based on the determined error function and repeating b)-f), if the error function is not at an acceptable level; and g) saving the three-dimensional object that is defined by the intersection of the planes, if the error function is at an acceptable level.
Claim 32
Dependent Claim
76 words
The method for converting objects in digitized two-dimensional images into three-dimensional objects of claim 25, further comprising: determining the desired resolution level of the texture for the three-dimensional object; determining angular increments for vanishing lines based on the determined desired resolution level; sampling image information along vanishing lines based on the determined angular increments; and mapping the sampled image information onto the generated three-dimensional object based on the image information location within the corresponding determined shape.
Claim 33
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 7.
Claim 34
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 8.
Claim 35
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 9.
Claim 36
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 10.
Claim 37
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 20.
Claim 38
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 22.
Claim 39
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 23.
Claim 40
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 24.
Claim 41
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 25.
Claim 42
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 26.
Claim 43
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 27.
Claim 44
Dependent Claim
14 words
A computer-readable medium having computer-executable instructions for performing the steps recited in claim 28.
References Cited (Backward Citations)
US4674046A
Cited
US5124693A
Cited
US4845643A
Cited
US4841292A
Cited
US5079703A
Cited
US5533177A
Cited
US5422987A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-11-22 | 2003-05-24 | 2003-11-23 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-11-22 | 2007-05-24 | 2007-11-23 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-11-22 | 2011-05-24 | 2011-11-23 | $7700.00 | Window Closed / Lapsed |