Hybrid tracking for augmented reality using both camera motion detection and landmark tracking

Claim 3

A method according to claim 1, further comprising the steps of: comparing the actual landmark location within the second image with the predicted location of the landmark to generate a prediction error; and updating a camera location based upon the camera displacement and the prediction error.

Claim 4

A method according to claim 1, wherein said registering step comprises the steps of: determining the head pose of the camera based upon the actual landmark location; and registering the computer generated graphic to objects in the real world based upon the determined head pose of the camera.

Claim 5

A method according to claim 1, wherein said determining step comprises determining the location within the first image of a plurality of landmarks to create an initial landmark location for each of said plurality of landmarks; wherein said predicting step comprises predicting the location of each of said plurality of landmarks within the second image based on the camera displacement and the initial landmark location corresponding to the landmark; and wherein said defining step comprises defining a plurality of search regions within the second image based upon the predicted location of the plurality of landmarks within the second image; wherein said locating step comprises locating each of the plurality of landmarks within the defined search region of the second image to create an actual landmark location for each of said landmarks; and wherein said registering step comprises registering the computer generated graphic to objects in the real world based upon the actual landmark location of said plurality of landmarks.

Claim 6

A method according to claim 4, wherein said registering step comprises the steps of: determining the head pose of the camera based upon the actual location of said plurality of landmarks; and registering the computer generated graphic based upon the head pose of the camera.

Claim 7

A method according to claim 5 further comprising the steps of: evaluating the number of landmarks located within the region by said locating step to determine whether a sufficient number landmarks have been located within the image to register the computer generated graphic based upon actual landmark location; and registering the computer generated graphic based upon the camera displacement if said evaluating step determines that an insufficient number of landmarks have been located within the image to register the computer generated graphic.

Claim 8

A method according to claim 5 further comprising the steps of: evaluating the number of landmarks located within the region by said locating step to determine whether three or more landmarks have been located within the image; and wherein said step of determining the head pose comprises determining the head pose utilizing a global analytical solution.

Claim 9

A method according to claim 7, wherein a final solution to said global analytical solution is determined based upon the camera displacement.

Claim 10

A method according to claim 5 further comprising the steps of: evaluating the number of landmarks located within the region by said locating step to determine whether one or two landmarks have been located within the image; and wherein said step of determining the head pose comprises determining the head pose utilizing a heuristic corrector and the camera displacement.

Claim 11

A method according to claim 4, wherein said plurality of landmarks comprise a plurality of fiducials.

Claim 12

A method according to claim 4, wherein the camera comprises two cameras and wherein the image comprises a pair of stereoscopic images.

Claim 13

A method according to claim 1, wherein said landmark comprises a fiducial.

Claim 14

A method according to claim 1, wherein the camera displacement is determined by a magnetic tracking system.

Claim 17

A system according to claim 14, further comprising: means for comparing the actual landmark location within the second image with the predicted location of the landmark to generate a prediction error; and means for updating a camera location based upon the camera displacement and the prediction error.

Claim 18

A system according to claim 14, wherein said means for registering comprises: means for determining the head pose of the camera based upon the actual landmark location; and means registering the computer generated graphic to objects in the real world based upon the determined head pose of the camera.

Claim 19

A system according to claim 14, wherein said means for determining comprises means for determining the location within the first image of a plurality of landmarks to create an initial landmark location for each of said plurality of landmarks; wherein said means for predicting comprises means for predicting the location of each of said plurality of landmarks within the second image based on the camera displacement and the initial landmark location corresponding to the landmark; wherein said means for defining comprises means for defining a plurality of search regions within the second image based upon the predicted location of the plurality of landmarks within the second image; wherein said means for locating comprises means for locating each of the plurality of landmarks within the defined search region of the second image to create an actual landmark location for each of said landmarks; and wherein said means for registering comprises means for registering the computer generated graphic to objects in the real world based upon the actual landmark location of said plurality of landmarks.

Claim 20

A system according to claim 17, wherein said means for registering comprises: means for determining the head pose of the camera based upon the actual location of said plurality of landmarks; and means for registering the computer generated graphic based upon the head pose of the camera.

Claim 21

A system according to claim 18 further comprising: means for evaluating the number of landmarks located within the region by said means for locating to determine whether a sufficient number landmarks have been located within the image to register the computer generated graphic based upon actual landmark location; and means for registering the computer generated graphic based upon the camera displacement if said means for evaluating determines that an insufficient number of landmarks have been located within the image to register the computer generated graphic.

Claim 22

A system according to claim 19 further comprising: means for evaluating the number of landmarks located within the region by said means for locating to determine whether three or more landmarks have been located within the image; and wherein said means for determining the head pose comprises means for determining the head pose utilizing a global analytical solution.

Claim 23

A system according to claim 18, wherein said plurality of landmarks comprise a plurality of fiducials.

Claim 24

A system according to claim 20, wherein a final solution to said global analytical solution is determined based upon the camera displacement.

Claim 25

A system according to claim 19 further comprising: means for evaluating the number of landmarks located within the region by said means for locating to determine whether one or two landmarks have been located within the image; and wherein said means for determining the head pose comprises means for determining the head pose utilizing a heuristic corrector and the camera displacement.

Claim 26

A system according to claim 18, wherein the camera comprises two cameras and wherein the image comprises a pair of stereoscopic images.

Claim 27

A system according to claim 14, wherein said means for detecting movement comprises a magnetic tracking system.

Claim 28

A computer program product for registering computer generated graphic to objects in the real world to provide a composite augmented reality image, the computer program product comprising: a computer-readable storage medium having computer-readable program code means embodied in said medium, said computer-readable program code means comprising: computer-readable program code means for obtaining a first image corresponding to the field of view of the camera; computer-readable program code means for determining the location within the first image of a landmark to create an initial landmark location; computer-readable program code means for detecting the physical movement of the user operated camera to create a camera displacement; computer-readable program code means for obtaining a second image corresponding to the field of view of said camera; computer-readable program code means for predicting the location of the landmark within the second image based on the camera displacement that reflects the physical movement of the camera independent of the first and second images obtained by the camera and the initial landmark location value determined from the first image; computer-readable program code means for defining a search region within the second image based upon the predicted location of the landmark within the second image; computer-readable program code means for locating the landmark within the defined search region of the second image to create an actual landmark location; and computer-readable program code means for registering the computer generated graphic to objects in the real world based upon the actual landmark location so as to provide a composite augmented reality image.

Claim 29

A computer program product for registering computer generated graphic to objects in the real world to provide a composite augmented reality image, the computer program product comprising: a computer-readable storage medium having computer-readable program code means embodied in said medium, said computer-readable program code means comprising: computer-readable program code means for obtaining a first image corresponding to the field of view of the camera; computer-readable program code means for determining the location within the first image of a landmark to create an initial landmark location; computer-readable program code means for detecting the physical movement of the user operated camera to create a camera displacement; computer-readable program code means for obtaining a second image corresponding to the field of view of said camera; computer-readable program code means for predicting the location of the landmark within the second image based on the camera displacement that reflects the physical movement of the camera independent of the first and second images obtained by the camera and the initial landmark location value determined from the first image; computer-readable program code means for defining a search region within the second image based upon the predicted location of the landmark within the second image; computer-readable program code means for locating the landmark within the defined search region of the second image to create an actual landmark location; and computer-readable program code means for registering the computer generated graphic to objects in the real world based upon the actual landmark location so as to provide a composite augmented reality image.

Claim 30

A computer program product according to claim 26, further comprising: computer-readable program code means for comparing the actual landmark location within the second image with the predicted location of the landmark to generate a prediction error; and computer-readable program code means for updating a camera location based upon the camera displacement and the prediction error.

Claim 31

A computer program product according to claim 26, wherein said computer-readable program code means for registering comprises: computer-readable program code means for determining the head pose of the camera based upon the actual landmark location; and computer-readable program code means registering the computer generated graphic to objects in the real world based upon the determined head pose of the camera.

Claim 32

A computer program product according to claim 26, wherein said computer-readable program code means for determining comprises computer-readable program code means for determining the location within the first image of a plurality of landmarks to create an initial landmark location for each of said plurality of landmarks; wherein said computer-readable program code means for predicting comprises computer-readable program code means for predicting the location of each of said plurality of landmarks within the second image based on the camera displacement and the initial landmark location corresponding to the landmark; wherein said computer-readable program code means for defining comprises computer-readable program code means for defining a plurality of search regions within the second image based upon the predicted location of the plurality of landmarks within the second image; wherein said computer-readable program code means for locating comprises computer-readable program code means for locating each of the plurality of landmarks within the defined search region of the second image to create an actual landmark location for each of said landmarks; and wherein said computer-readable program code means for registering comprises computer-readable program code means for registering the computer generated graphic to objects in the real world based upon the actual landmark location of said plurality of landmarks.

Claim 33

A computer program product according to claim 29, wherein said computer-readable program code means for registering comprises: computer-readable program code means for determining the head pose of the camera based upon the actual location of said plurality of landmarks; and computer-readable program code means for registering the computer generated graphic based upon the head pose of the camera.

Claim 34

A computer program product according to claim 32 further comprising: computer-readable program code means for evaluating the number of landmarks located within the region by said computer-readable program code means for locating to determine whether a sufficient number landmarks have been located within the image to register the computer generated graphic based upon actual landmark location; and computer-readable program code means for registering the computer generated graphic based upon the camera displacement if said computer-readable program code means for evaluating determines that an insufficient number of landmarks have been located within the image to register the computer generated graphic.

Claim 35

A computer program product according to claim 31, wherein the camera comprises two cameras and wherein the image comprises a pair of stereoscopic images.

Claim 36

A computer program product according to claim 30 further comprising: computer-readable program code means for evaluating the number of landmarks located within the region by said computer-readable program code means for locating to determine whether one or two landmarks have been located within the image; and wherein said computer-readable program code means for determining the head pose comprises computer-readable program code means for determining the head pose utilizing a heuristic corrector and the camera displacement.

Claim 37

A computer program product according to claim 29 further comprising: computer-readable program code means for evaluating the number of landmarks located within the region by said computer-readable program code means for locating to determine whether three or more landmarks have been located within the image; and wherein said computer-readable program code means for determining the head pose comprises computer-readable program code means for determining the head pose utilizing a global analytical solution.

Claim 38

A computer program product according to claim 34, wherein a final solution to said global analytical solution is determined based upon the camera displacement.