US6553138B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for generating three-dimensional representations of objects
Assignee: New York University
•
Filing: 1998-12-30
•
Granted: 2003-04-22
•
Expires: 2018-12-30
Abstract
Abstract The preferred embodiment of the invention includes hardware and software for scanning objects and extracting their geometry and color information so as to generate their 3D (three dimensional) representations in a computer. The preferred embodiment uses a computer, a video camera, a light source, and an indicator located within the field of view of the camera in a fixed position. Preferably, the light source projects a crisp line of light on the scanned object. The software of the preferred embodiment processes the frames produced by the camera. It preferably locates the illuminated line and the indicator, and uses this information to determine 3D coordinates of the illuminated points. The 3D coordinates of the points are used to construct 3D representations of objects. The preferred embodiment also obtains color information of the scanned object and maps it to the 3D representation.
Inventors
- 👤 Daniel H. Rozin
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-08 20:49
CPC Classifications
G; G06; G06T; G06T17/00
Claims Analysis
Total Claims: 59
Claim 1
Independent Claim
179 words
An apparatus for generating a 3D computer representation of an object comprising: (a) a camera; (b) a light source for illuminating at least a portion of the object; (c) a support for the video camera and the light source, wherein the support permits moving the camera and the light source so as to scan the object; (d) a stationary position indicator within a field of view of the camera; and (e) a computer with video capture capability for receiving video information captured by the camera, including images each of which comprising pixels representing an illuminated portion of the object and the position indicator, wherein the computer obtains calibration parameters during a calibration procedure, without regard to the indicator, based on a special relationship between the camera, the light source and the object, and wherein, after a completion of the calibration procedure, the computer receives and evaluates the video information associated with the object in real time so as to generate the 3D representation of the object based on a position of the indicator, the images and the calibration parameters.
Claim 2
Independent Claim
179 words
An apparatus for generating a 3D computer representation of an object comprising: (a) a camera; (b) a light source for illuminating at least a portion of the object; (c) a support for the video camera and the light source, wherein the support permits moving the camera and the light source so as to scan the object; (d) a stationary position indicator within a field of view of the camera; and (e) a computer with video capture capability for receiving video information captured by the camera, including images each of which comprising pixels representing an illuminated portion of the object and the position indicator, wherein the computer obtains calibration parameters during a calibration procedure, without regard to the indicator, based on a special relationship between the camera, the light source and the object, and wherein, after a completion of the calibration procedure, the computer receives and evaluates the video information associated with the object in real time so as to generate the 3D representation of the object based on a position of the indicator, the images and the calibration parameters.
Claim 3
Dependent Claim
16 words
The apparatus of claim 1 wherein the light source provides a crisp line illuminating the object.
Claim 4
Dependent Claim
25 words
The apparatus of claim 1 wherein the light source illuminates the object at a substantially small angle with respect to the direction of the camera.
Claim 5
Dependent Claim
56 words
The apparatus of claim 1 wherein the support is such that a user can move the camera and the source and wherein when a user moves the camera and the source more slowly, the 3D representation is more detailed and when the user moves the camera and the source faster, the 3D representation is less detailed.
Claim 6
Dependent Claim
27 words
The apparatus of claim 1 wherein the computer processes video information provided by the camera in real time as a user scans the object using the camera.
Claim 7
Dependent Claim
32 words
The apparatus of claim 1 wherein the video information is recorded on a storage medium during scanning and then provided to the computer at a later time for generating the 3D representation.
Claim 8
Dependent Claim
16 words
The apparatus of claim 1 wherein the indicator is positioned in the background of the object.
Claim 9
Dependent Claim
12 words
The apparatus of claim 1 wherein the indicator is a bright light.
Claim 10
Dependent Claim
13 words
The apparatus of claim 1 wherein the indicator is a visual two-dimensional pattern.
Claim 11
Dependent Claim
25 words
The apparatus of claim 1 wherein the indicator is a color which is used to provide at least one of the orientation and the position.
Claim 12
Dependent Claim
11 words
The apparatus of claim 1 wherein the support is a tripod.
Claim 13
Dependent Claim
17 words
The apparatus of claim 1 wherein the light employed by the light source is a halogen light.
Claim 14
Dependent Claim
27 words
The apparatus of claim 1 wherein the light source generates a vertical line of light and the object is scanned by the camera in a horizontal direction.
Claim 15
Dependent Claim
24 words
The apparatus of claim 1 wherein the light source generates a horizontal line of light and the object is scanned in the vertical direction.
Claim 16
Dependent Claim
24 words
The apparatus of claim 1 wherein the support is configured to enable a user to move the camera and the light source by hand.
Claim 17
Independent Claim
163 words
A method of obtaining a 3D computer representation of an object, comprising: a. storing, in a computer memory, images of a sequence of video frames obtained by scanning, each of which includes a depiction of a portion of the object illuminated by a line of light and a depiction of a statically-mounted orientation indicator; b. determining locations of the orientation indicator within the frames; c. performing a calibration procedure, without regard to the indicator, to obtain calibration parameters based on a spacial relationship between the camera, the light source and the object; d. after a completion of step c, determining 3D locations of at least some points on the portion of the object in real time based on pixels representing the individual lines of light in the sequence of frames obtained by scanning; and e. after step c and during step d, generating the 3D computer representation of the object using a position of the indicator, the calibration parameters and the 3D locations.
Claim 18
Independent Claim
163 words
A method of obtaining a 3D computer representation of an object, comprising: a. storing, in a computer memory, images of a sequence of video frames obtained by scanning, each of which includes a depiction of a portion of the object illuminated by a line of light and a depiction of a statically-mounted orientation indicator; b. determining locations of the orientation indicator within the frames; c. performing a calibration procedure, without regard to the indicator, to obtain calibration parameters based on a spacial relationship between the camera, the light source and the object; d. after a completion of step c, determining 3D locations of at least some points on the portion of the object in real time based on pixels representing the individual lines of light in the sequence of frames obtained by scanning; and e. after step c and during step d, generating the 3D computer representation of the object using a position of the indicator, the calibration parameters and the 3D locations.
Claim 19
Dependent Claim
54 words
The method of claim 16 wherein the step of determining 3D locations of the points comprise determining a distance of a point, represented by a pixel on the image from a camera; determining a horizontal position of the point represented by the pixel; determining a vertical position of the point represented by the pixel.
Claim 20
Dependent Claim
32 words
The method of claim 16 further comprises locating pixels depicting the illuminated portion of the object by identifying the brightest pixel among the pixels of at least a portion of the image.
Claim 21
Dependent Claim
33 words
The method of claim 16 further comprising locating pixels depicting the illuminated portion of the object by identifying the first bright pixel among the pixels of at least a portion of the image.
Claim 22
Dependent Claim
35 words
The method of claim 16 further comprises locating pixels depicted on the illuminated portion of the object by determining the greatest change in brightness between the pixels within at least a portion of the image.
Claim 23
Dependent Claim
23 words
The method of claim 16 wherein the step of determining the location of the orientation indicator comprises identifying the indicator by pixel brightness.
Claim 24
Dependent Claim
23 words
The method of claim 16 wherein the step of determining the location of the orientation indicator comprises identifying the indicator by pixel color.
Claim 25
Dependent Claim
25 words
The method of claim 16 wherein the step of determining the location of the orientation indicator comprises identifying the indicator by a pattern of pixels.
Claim 26
Dependent Claim
14 words
The method of claim 16 further includes the step of creating a color map.
Claim 27
Independent Claim
249 words
An apparatus for scanning an object and generating a three-dimensional representation of at least a portion of the object, comprising: (a) a camera adapted to receive video information, and generate images of at least a portion of the object associated with the video information when the portion of the object is in a field of view of the camera; (b) a light source adapted to illuminate at least a portion of the object within the field of view of the camera within a line of light; (c) a stationary indicator positioned within the field of view of the camera, and (d) a processing arrangement adapted to obtain calibration parameters during a calibration procedure, the processing arrangement generating the three-dimensional representation using the calibration parameters, wherein each of the images comprises a depiction of the stationary indicator and a depiction of particular points of a portion of the object illuminated by the line of light, wherein the calibration parameters are obtained based on a spacial relationship between the camera, the light source and the object and without regard to the indicator, and wherein, after a completion of the calibration procedure, the processing arrangement is adapted receive data associated with the object in real time so as to construct the three-dimensional representation of at least a portion of the object using a position of the indicator, the calibration parameters and a sequence of such images with the particular points of different portions of the object illuminated by the line of light.
Claim 28
Independent Claim
249 words
An apparatus for scanning an object and generating a three-dimensional representation of at least a portion of the object, comprising: (a) a camera adapted to receive video information, and generate images of at least a portion of the object associated with the video information when the portion of the object is in a field of view of the camera; (b) a light source adapted to illuminate at least a portion of the object within the field of view of the camera within a line of light; (c) a stationary indicator positioned within the field of view of the camera, and (d) a processing arrangement adapted to obtain calibration parameters during a calibration procedure, the processing arrangement generating the three-dimensional representation using the calibration parameters, wherein each of the images comprises a depiction of the stationary indicator and a depiction of particular points of a portion of the object illuminated by the line of light, wherein the calibration parameters are obtained based on a spacial relationship between the camera, the light source and the object and without regard to the indicator, and wherein, after a completion of the calibration procedure, the processing arrangement is adapted receive data associated with the object in real time so as to construct the three-dimensional representation of at least a portion of the object using a position of the indicator, the calibration parameters and a sequence of such images with the particular points of different portions of the object illuminated by the line of light.
Claim 29
Dependent Claim
15 words
The apparatus of claim 25 wherein the stationary indicator is a bright source of light.
Claim 30
Dependent Claim
27 words
The apparatus of claim 25 wherein the stationary indicator is a distinctive color which is used to provide at least one of the orientation and the position.
Claim 31
Dependent Claim
14 words
The apparatus of claim 25 wherein the stationary indicator is a predetermined two-dimensional pattern.
Claim 32
Dependent Claim
21 words
The apparatus of claim 25 wherein the camera is a conventional video camera that can be purchased by an ordinary consumer.
Claim 33
Dependent Claim
11 words
The apparatus of claim 25 wherein the camera is a camcorder.
Claim 34
Dependent Claim
13 words
The apparatus of claim 25 wherein the camera is a Hi 8 camera.
Claim 35
Dependent Claim
12 words
The apparatus of claim 25 wherein the camera is a digital camera.
Claim 36
Dependent Claim
12 words
The apparatus of claim 25 wherein the line of light is vertical.
Claim 37
Dependent Claim
12 words
The apparatus of claim 25 wherein the line of light is horizontal.
Claim 38
Dependent Claim
12 words
The apparatus of claim 25 wherein the light source emits halogen light.
Claim 39
Dependent Claim
12 words
The apparatus of claim 25 wherein the light source emits incandescent light.
Claim 40
Dependent Claim
30 words
The apparatus of claim 25 wherein the illumination provided by the light source is in a direction which is at an angle with respect to a direction of the camera.
Claim 41
Dependent Claim
11 words
The apparatus of claim 37 wherein the angle is substantially small.
Claim 42
Dependent Claim
18 words
The apparatus of claim 25 further comprising a support for supporting both the camera and the light source.
Claim 43
Dependent Claim
37 words
The apparatus of claim 25 wherein the scanning includes moving the camera and the light source in a vertical direction while the light source illuminates the object with a line of light oriented in a horizontal direction.
Claim 44
Dependent Claim
36 words
The apparatus of claim 25 wherein the scanning includes moving the camera and the light source in a horizontal direction while the light source illuminates the object with a line of light in a vertical direction.
Claim 45
Dependent Claim
184 words
A computer method for processing a sequence of images, the images including a depiction of a stationary indicator and a depiction of a portion of an object illuminated by a line of light, the computer method comprising: receiving video information associated with at least one portion of the object; identifying a location of the stationary indicator in each of the images in the sequence, wherein the images corresponds to the received video information; performing a calibration procedure to obtain calibration parameters based on a spacial relationship between the camera, the light source and the object and without regard to the indicator; in real time, identifying a portion of the object illuminated by the line of light in each of the images in the sequence; and after a completion of the calibration procedure, determining 3D coordinates of at least some points located on the object so as to generate a 3D representation of the at least one portion of the object using a position of the indicator, the calibration parameters, and the identified locations of a portion of the line of light in the images.
Claim 46
Dependent Claim
184 words
A computer method for processing a sequence of images, the images including a depiction of a stationary indicator and a depiction of a portion of an object illuminated by a line of light, the computer method comprising: receiving video information associated with at least one portion of the object; identifying a location of the stationary indicator in each of the images in the sequence, wherein the images corresponds to the received video information; performing a calibration procedure to obtain calibration parameters based on a spacial relationship between the camera, the light source and the object and without regard to the indicator; in real time, identifying a portion of the object illuminated by the line of light in each of the images in the sequence; and after a completion of the calibration procedure, determining 3D coordinates of at least some points located on the object so as to generate a 3D representation of the at least one portion of the object using a position of the indicator, the calibration parameters, and the identified locations of a portion of the line of light in the images.
Claim 47
Dependent Claim
27 words
The method of claim 42 wherein the step of identifying the location of the indicator includes searching the image for one or more pixels of high brightness.
Claim 48
Dependent Claim
29 words
The method of claim 42 wherein the step of identifying the location of the indicator includes searching the image for one or more pixels having substantially a predetermined color.
Claim 49
Dependent Claim
29 words
The method of claim 42 wherein the step of identifying the location of the indicator includes searching the image for a group of pixels substantially corresponding to a pattern.
Claim 50
Dependent Claim
25 words
The method of claim 42 wherein the illuminated portion of the object depicted on the image comprises an illuminated line on a substantially dark background.
Claim 51
Dependent Claim
13 words
The method of claim 46 further comprising identifying pixels located on the line.
Claim 52
Dependent Claim
35 words
The method of claim 47 wherein the step of identifying the pixels on the line comprises locating in a plurality of pixels the brightest pixel, which is not one of the pixels representing the indicator.
Claim 53
Dependent Claim
34 words
The method of claim 47 wherein the step of identifying the pixels on the line comprises locating in a plurality of pixels a first bright pixel, which is not a pixel representing the indicator.
Claim 54
Dependent Claim
42 words
The method of claim 47 wherein the step of identifying the pixels comprises locating in a plurality of pixels a pixel, not representing the indicator and having the greatest change in brightness with respect to other pixels of the plurality of pixels.
Claim 55
Dependent Claim
37 words
The method of claim 46 wherein the step of determining 3D coordinates comprises determining the distance from the camera of the points on the object corresponding to the pixels of the illuminated line depicted on the image.
Claim 56
Dependent Claim
180 words
An article of manufacture for processing a sequence of images, the images including a depiction of at least one portion of an object illuminate by a line of light and a depiction of a stationary indicator, the article of manufacture comprising: software for identifying locations of such stationary indicator in the images of the sequence, wherein the images are associated with video information received for the at least one portion of the object; software for identifying, in real time, illuminated portions of the object in the images of the sequence; software for performing a calibration procedure to obtain calibration parameters based on a spacial relationship between the camera, the light source and the object and without regard to the indicator; and software for determining 3D coordinates of at least some points located on the object using a position of the indicator, the calibration parameters and the illuminated portion of the object so as to generate a 3D representation of the at least one portion of the object, wherein the 3D coordinates are determined after a completion of the calibration procedure.
Claim 57
Dependent Claim
180 words
An article of manufacture for processing a sequence of images, the images including a depiction of at least one portion of an object illuminate by a line of light and a depiction of a stationary indicator, the article of manufacture comprising: software for identifying locations of such stationary indicator in the images of the sequence, wherein the images are associated with video information received for the at least one portion of the object; software for identifying, in real time, illuminated portions of the object in the images of the sequence; software for performing a calibration procedure to obtain calibration parameters based on a spacial relationship between the camera, the light source and the object and without regard to the indicator; and software for determining 3D coordinates of at least some points located on the object using a position of the indicator, the calibration parameters and the illuminated portion of the object so as to generate a 3D representation of the at least one portion of the object, wherein the 3D coordinates are determined after a completion of the calibration procedure.
Claim 58
Dependent Claim
25 words
The article of claim 52 wherein the illuminated portion of the object depicted on the image comprises an illuminated line on a substantially dark background.
Claim 59
Dependent Claim
15 words
The article of claim 53 further comprising software for identifying pixels located on the line.
References Cited (Backward Citations)
US4534650A
Cited
US4487472A
Cited
US4575304A
Cited
US4628469A
Cited
US4798210A
Cited
US4705401A
Cited
US4737032A
Cited
USRE34438E
Cited
US4919502A
Cited
US4786151A
Cited
US4885594A
Cited
US4982205A
Cited
US5024529A
Cited
US4957369A
Cited
US5146360A
Cited
US5175601A
Cited
GB2264602A
Cited
US5424836A
Cited
WO1994015173A1
Cited
US5363185A
Cited
US5475422A
Cited
US5714762A
Cited
US5668631A
Cited
US5654800A
Cited
US5561526A
Cited
US5671079A
Cited
US5710875A
Cited
WO1996010205A1
Cited
US5969822A
Cited
US5675407A
Cited
US5807042A
Cited
US5675377A
Cited
US5856844A
Cited
US6028955A
Cited
US5642293A
Cited
US5832139A
Cited
US6108497A
Cited
US6363169B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-04-21 | 2006-10-21 | 2007-04-22 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-04-21 | 2010-10-21 | 2011-04-22 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-04-21 | 2014-10-21 | 2015-04-22 | $7700.00 | Window Closed / Lapsed |