Patent US3085549A successfully refreshed from Google Patents.
US4942539A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and system for automatically determining the position and orientation of an object in 3-D space
Assignee: Fanuc Robotics North America Inc
•
Filing: 1988-12-21
•
Granted: 1990-07-17
•
Expires: 2008-12-21
Abstract
Abstract A method and system for automatically determining the position and orientation of an object by utilizing as few as a single digital image generated by as few as a single camera without the use of a structured light. The digital image contains at least three non-colinear geometric features of the object. The three features may be either coplanar or non-coplanar. The features or targets are viewed such that perspective information is present in the digital image. In a single camera system the geometric features are points, and in a multi-camera system, the features are typically combinations of points and lines. The location of the features are determined and processed within a programmed computer together with reference data and camera calibration data to provide at least three non-parallel 3-D lines. The 3-D lines are utilized by an iterative algorithm to obtain data relating to the position and orientation of the object in 3-D space. The resultant data is subsequently utilized to calculate an offset of the object from the camera. The offset is then transformed into the coordinate system or frame of a peripheral device such as a robot, programmable controller, numerically controlled machine, etc. Finally, the programmed computer transfers the transformed offset to the peripheral device which utilizes the transformed offset to modify its preprogrammed path.
Inventors
- 👤 H. Dean McGee
- 👤 Kenneth W. Krause
- 👤 Bruce E. Coldren
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-23 03:47
CPC Classifications
G; G01; G01S; G01S5/00; G01S5/16; G01S5/163; B; B25; B25J; B25J9/00; B25J9/16; B25J9/1694; B25J9/1697; G06; G06T; G06T7/00; G06T7/70; G06T7/73; G06T7/80
Claims Analysis
Total Claims: 45
Claim 1
Independent Claim
211 words
Method for automatically determining the position and orientation of an object in 3-D space from as few as one digital image generated by as few as one 2-D sensor, the method comprising the steps of: generating sensor calibration data relating the position and orientation of the 2-D sensor to the 3-D space; generating reference data relating to at least three non-collinear geometric features of an ideal object; generating the digital image containing the at least three-non-collinear geometric features of the object wherein at least two of the features reside in the image generated by the one 2-D sensor; locating each of the features in the digital image; computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective feature of the object at a reference point; determining a starting point on each of the 3-D lines in the vicinity of its associated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; and utilizing the reference data and the actuaI 3-D location of each of the features to determine the position and orientation of the object.
Claim 2
Dependent Claim
211 words
Method for automatically determining the position and orientation of an object in 3-D space from as few as one digital image generated by as few as one 2-D sensor, the method comprising the steps of: generating sensor calibration data relating the position and orientation of the 2-D sensor to the 3-D space; generating reference data relating to at least three non-collinear geometric features of an ideal object; generating the digital image containing the at least three-non-collinear geometric features of the object wherein at least two of the features reside in the image generated by the one 2-D sensor; locating each of the features in the digital image; computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective feature of the object at a reference point; determining a starting point on each of the 3-D lines in the vicinity of its associated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; and utilizing the reference data and the actuaI 3-D location of each of the features to determine the position and orientation of the object.
Claim 3
Dependent Claim
32 words
The method of claim 1 wherein the step of generating the digital image includes the step of: generating first and second digital sub-images, the first and second sub-images defining the digital image.
Claim 4
Dependent Claim
70 words
The method of claim 2 wherein the method utilizes first and second 2-D sensors to generate the first and second digital sub-images, respectively, wherein the method further comprises the step of generating sensor calibration data relating the position and orientation of the second 2-D sensor to the 3-D space and wherein the at least three 3-D lines are also a function of the calibration data of the second 2-D sensor.
Claim 5
Dependent Claim
30 words
The method of claim 1 wherein each of the at least three features of the ideal object is a point and wherein the 3-D lines intersect at a common point.
Claim 6
Dependent Claim
14 words
The method of claim 1 wherein the image is a gray scale digital image.
Claim 7
Dependent Claim
33 words
The method of claim 4 wherein the 2-D sensor is a camera having a lens system with an optical center and wherein the common point is the optical center of the lens system.
Claim 8
Dependent Claim
12 words
The method of claim 1 wherein the object is a rigid body.
Claim 9
Dependent Claim
12 words
The method of claim 1 wherein the 2-D sensor is a camera.
Claim 10
Dependent Claim
53 words
The method of claim 1 wherein said step of determining a starting point on each of the 3-D lines includes the step of generating at least three initial position estimates of the starting points as a function of the locations of the features of the ideal object and each of said 3-D lines.
Claim 11
Dependent Claim
40 words
The method of claim 9 wherein said step of determining the actual 3-D location of each feature includes the step of computing the distances between pairs of the initial position estimates and between pairs of the features of the object.
Claim 12
Dependent Claim
47 words
The method of claim 10 wherein said step of determining the actual 3-D location of each feature includes the step of minimizing the differences between the distances between the pairs of the initial position estimates and the distances between the pairs of the features of the objects.
Claim 13
Dependent Claim
264 words
Method for automatically generating offset data for use by a programmable peripheral device having a coordinate frame such as a robot to move to an object in 3-D space, the method utilizing as few as one digital image generated by as few as one 2-D sensor, the method comprising the steps of: generating sensor calibration data relating the position and orientation of the 2-D sensor to the 3-D space; generating reference data relating to at least three non-collinear geometric features of an ideal object; generating the digital image containing the at least three non-collinear geometric features of the object wherein at least two of the features reside in the image generated by the one 2-D sensor; locating each of the features in the digital image; computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective feature of the object at a reference point; determining a starting point on each of the 3-D lines in the vicinity of its assoicated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; utilizing the reference data and the actual 3-D location of each of the features to determine the position and orientation of the object; calculating an offset of the object from the 2-D sensor as a function of the position and orientation of the object; and transforming the offset of the object to the coordinate frame of the peripheral device.
Claim 14
Dependent Claim
264 words
Method for automatically generating offset data for use by a programmable peripheral device having a coordinate frame such as a robot to move to an object in 3-D space, the method utilizing as few as one digital image generated by as few as one 2-D sensor, the method comprising the steps of: generating sensor calibration data relating the position and orientation of the 2-D sensor to the 3-D space; generating reference data relating to at least three non-collinear geometric features of an ideal object; generating the digital image containing the at least three non-collinear geometric features of the object wherein at least two of the features reside in the image generated by the one 2-D sensor; locating each of the features in the digital image; computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective feature of the object at a reference point; determining a starting point on each of the 3-D lines in the vicinity of its assoicated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; utilizing the reference data and the actual 3-D location of each of the features to determine the position and orientation of the object; calculating an offset of the object from the 2-D sensor as a function of the position and orientation of the object; and transforming the offset of the object to the coordinate frame of the peripheral device.
Claim 15
Dependent Claim
32 words
The method of claim 12 wherein the step of generating the digital image includes the step of: generating first and second digital sub-images, the first and second sub-images defining the digital image.
Claim 16
Dependent Claim
70 words
The method of claim 13 wherein the method utilizes first and second 2-D sensors to generate the fist and second digital sub-images, respectively, wherein the method further comprises the step of generating sensor calibration data relating the position and orientation of the second 2-D sensor to the 3-D space and wherein the at least three 3-D lines are also a function of the calibration data of the second 2-D sensor.
Claim 17
Dependent Claim
30 words
The method of claim 12 wherein each of the at least three features of the ideal object is a point and wherein the 3-D lines intersect at a common point.
Claim 18
Dependent Claim
25 words
The method of claim 15 wherein the 2-D sensor is a camera and wherein the common point is the optical center of the lens system.
Claim 19
Dependent Claim
12 words
The method of claim 12 wherein the object is a rigid body.
Claim 20
Dependent Claim
12 words
The method of claim 12 wherein the 2-D sensor is a camera.
Claim 21
Dependent Claim
53 words
The method of claim 12 wherein said step of determining a starting point on each of the 3-D lines includes the step of generating at least three initial position estimates of the starting points as a function of the locations of the features of the ideal object and each of said 3-D lines.
Claim 22
Dependent Claim
47 words
The method of claim 19 wherein said step of determining the actual 3-D location of each feature includes the step of generating at least three initial position estimates as a function of the locations of the features of the ideal object and each of said 3-D lines.
Claim 23
Dependent Claim
40 words
The method of claim 20 wherein said step of determining the actual 3-D location of each feature includes the step of computing the distances between pairs of the initial position estimates and between pairs of the features of the object.
Claim 24
Dependent Claim
238 words
A vision system for automatically determining the position and orientation of an object in 3-D D space from as few as one digital image generated by as few as one 2-D sensor, the system comprising: first means for storing reference data relating to at least three non-collinear geometric features of an ideal object; 2-D sensor means for generating the digital image containing the at least three non-collinear geometric features of the object, the 2-D sensor means including the one 2-D sensor wherein at least two of the features reside in the image generated by the one 2-D sensor; second means for storing sensor calibration data relating the position and orientation of the 2-D sensor means to the 3-D space; means for locating each of the features in the digital image; means for computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective feature of the object at a reference point; determining a starting point on each of the 3-D lines in the vicintiy of its associated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; and means for utilzing the reference data and the actual 3-D locations of each of the features to determine the position and orientation of the object.
Claim 25
Dependent Claim
238 words
A vision system for automatically determining the position and orientation of an object in 3-D D space from as few as one digital image generated by as few as one 2-D sensor, the system comprising: first means for storing reference data relating to at least three non-collinear geometric features of an ideal object; 2-D sensor means for generating the digital image containing the at least three non-collinear geometric features of the object, the 2-D sensor means including the one 2-D sensor wherein at least two of the features reside in the image generated by the one 2-D sensor; second means for storing sensor calibration data relating the position and orientation of the 2-D sensor means to the 3-D space; means for locating each of the features in the digital image; means for computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective feature of the object at a reference point; determining a starting point on each of the 3-D lines in the vicintiy of its associated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; and means for utilzing the reference data and the actual 3-D locations of each of the features to determine the position and orientation of the object.
Claim 26
Dependent Claim
32 words
The system of claim 22 wherein said 2-D sensor means includes first and second 2-D sensors for generating first and second digital sub-images, the first and second sub-images defining the digital image.
Claim 27
Dependent Claim
53 words
The system of claim 23 wherein said second means for storing stores sensor calibration data relating the position and orientation of the first and second 2-D sensors to the 3-D space and wherein the at least three 3-D lines are a function of the calibration data of the first and second 2-D sensors.
Claim 28
Dependent Claim
30 words
The system of claim 22 wherein each of the at least three features of the ideal object is a point and wherein the 3-D lines intersect at a common point.
Claim 29
Dependent Claim
33 words
The system of claim 25 wherein the 2-D sensor is a camera having a lens system with an optical center and wherein the common point is the optical center of the lens system.
Claim 30
Dependent Claim
12 words
The system of claim 22 wherein the object is a rigid body.
Claim 31
Dependent Claim
12 words
The system of claim 22 wherein the 2-D sensor is a camera.
Claim 32
Dependent Claim
48 words
The system of claim 22 wherein said means for determining a starting point includes position estimating means for generating at least three initial position estimates of the starting points as a function of the locations of the features of the ideal object and each of said 3-D lines.
Claim 33
Dependent Claim
39 words
The system of claim 29 wherein said means for determining the actual 3-D location of each feature includes means for computing the distances between piars of the initial position estimates and between pairs of the features of the object.
Claim 34
Dependent Claim
48 words
The system as claimed in claim 30 wherein said means for determining the actual 3-D location of each feature includes means for minimizng the differences between the distances between the pairs of the initial position estimates and the distances between the pairs of the features of the object.
Claim 35
Dependent Claim
298 words
A vision system for automatically generating offset data for use by a programmble peripheral device having a coordinate frame such as a robot to enable the peripheral device to move to an object in 3-D space from as few as one digital image generated by as few as one 2-D sensor, the system comprising: first means for storing sensor calibration data relating to at least three non-collinear geometric features of an ideal object; 2-D sensor means for generating the digital image containing the at least three non-collinear geometric features of the object, the 2-D sensor means including the one 2-D sensor wherein at least two of the features reside in the image generated by the one 2-D sensor; second means for storing sensor calibration data relating the position and orientation of the 2-D sensor means to the 3-D space; means for locating each of the features in the digital image; means for computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective features of the object at a reference point; determining a starting point on each of the 3-D lines in the vicinity of its associated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; means for utilizing the reference data and the actual 3-D locations of each of the features to determine the position and orientation of the object; means for calculating an offset of the object from the 2-D sensor means as a function of the position and orientation of the object; and means for transforming the offset of the object to the coordinate frame of the peripheral device.
Claim 36
Dependent Claim
298 words
A vision system for automatically generating offset data for use by a programmble peripheral device having a coordinate frame such as a robot to enable the peripheral device to move to an object in 3-D space from as few as one digital image generated by as few as one 2-D sensor, the system comprising: first means for storing sensor calibration data relating to at least three non-collinear geometric features of an ideal object; 2-D sensor means for generating the digital image containing the at least three non-collinear geometric features of the object, the 2-D sensor means including the one 2-D sensor wherein at least two of the features reside in the image generated by the one 2-D sensor; second means for storing sensor calibration data relating the position and orientation of the 2-D sensor means to the 3-D space; means for locating each of the features in the digital image; means for computing at least three non-parallel 3-D lines as a function of the feature locations and the sensor calibration data, each of the 3-D lines passing through its respective features of the object at a reference point; determining a starting point on each of the 3-D lines in the vicinity of its associated reference point; determining the actual 3-D location of each feature on its associated 3-D line utilizing the reference data and its respective starting point on the 3-D line; means for utilizing the reference data and the actual 3-D locations of each of the features to determine the position and orientation of the object; means for calculating an offset of the object from the 2-D sensor means as a function of the position and orientation of the object; and means for transforming the offset of the object to the coordinate frame of the peripheral device.
Claim 37
Dependent Claim
32 words
The system of claim 32 wherein said 2-D sensor means includes first and second 2-D sensors for generating first and second digital sub-images, the first and second sub-images defining the digital image.
Claim 38
Dependent Claim
53 words
The system of claim 33 wherein said second means for storing stores sensor calibration data relating the position and orientation of the first and second 2-D sensors to the 3-D space and wherein the at least three 3-D lines are a function of the calibration data of the first and second 2-D sensors.
Claim 39
Dependent Claim
30 words
The system of claim 32 wherein each of the at least three features of the ideal object is a point and wherein the 3-D lines intersect at a common point.
Claim 40
Dependent Claim
33 words
The system of claim 35 wherein the 2-D sensor is a camera having a lens system with an optical canter and wherein the common point is the optical canter of the lens system.
Claim 41
Dependent Claim
12 words
The system of claim 32 wherein the object is a rigid body.
Claim 42
Dependent Claim
12 words
The system of claim 32 wherein the 2-D sensor is a camera.
Claim 43
Dependent Claim
47 words
The system of claim 32 wherein said means for determining a starting point includes position estimating means for generating at least three initial position estimates of the starting points as a function of the locations of the features of the ideal object and each said 3-D lines.
Claim 44
Dependent Claim
39 words
The system of claim 39 wherein said means for determining the actual 3-D location of each feature includes means for computing the distances between pairs of the initial position estimates and between pairs of the features of the object.
Claim 45
Dependent Claim
46 words
The system of claim 40 wherein said means for determining the actual 3-D location of each feature includes means for minimizing the differences between the distances between the pairs of the initial position estimates and the distances between the pairs of the features of the object.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1993-07-16 | 1994-01-15 | 1994-07-17 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1997-07-16 | 1998-01-15 | 1998-07-17 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2001-07-16 | 2002-01-15 | 2002-07-17 | $7700.00 | Window Closed / Lapsed |