Method for region tracking in an image sequence using a two-dimensional mesh

Claim 3

The method as in claim 1, wherein step (d3) further includes defining a maximum range for estimating the parameters of the motion model.

Claim 4

The method as in claim 2, further comprising the step of refining the location of boundary nodes on the tracked polygon.

Claim 5

The method as in claim 3 further comprising the step of fitting a mesh to the second predetermined portion that corresponds in nodes and patches to the mesh in the reference polygon.

Claim 6

The method as in claim 4 further comprising the steps of finding parameters of affine transformation between each corresponding pair of patches in the second predetermined portion and the tracked polygon in the previous and subsequent image frames.

Claim 7

The method as in claim 5 further comprising the step of mapping pixels in each patch in the second predetermined portion into the corresponding patch in the tracked polygon using the parameters of the corresponding affine transformation.

Claim 8

An article of manufacture comprising: a computer usable medium having computer readable program means embodied therein for causing tracking of a first predetermined, two-dimensional portion of an image throughout a sequence of images, the computer readable program code means in said article of manufacture comprising: (a) computer readable program means for causing the computer to effect selecting a reference frame; (b) computer readable program means for causing the computer to effect selecting the first predetermined, two-dimensional portion within the reference frame by choosing a reference polygon having at least three comers that defines the boundary of the first predetermined portion; (c) computer readable program means for causing the computer to effect fitting a reference mesh having corner nodes at the comers of the reference polygon and at least one inside the reference polygon; (d) computer readable program means for causing the computer to effect predicting the reference polygon in subsequent or previous image frames by independently tracking the corner of the reference polygon; (e) computer readable program means for dividing the reference polygon and the tracked polygon into a minimum number of triangles so that each triangle in the reference polygon respectively corresponds to a triangle in the tracked polygon; for finding parameters of affine transformation between each corresponding pair of triangles; and for mapping nodes in each triangle of the reference polygon into the respective triangle in the tracked polygon using the parameters of the corresponding affine transformation used for the triangle in which the node is located; (f) computer readable program means for causing the computer to effect refining locations of the inside and corner nodes of the corresponding mesh for tracking local and global motion of the first predetermined portion; (g) means for causing the computer to effect defining the location of boundary nodes on the reference mesh for tracking the local motion around the boundary of the predetermined portion; (h) means for causing the computer to effect tracking illumination changes that occurred between the reference frame and a previous or subsequent frame; (i) means for causing said (c), (d), (e) and (f) computer readable program means to be implemented in a hierarchy of spatial resolutions; (j) means for causing the computer to effect replacing the first predetermined portion with a second predetermined portion throughout a portion of the sequence of images so that the second predetermined portion undergoes the same global and local motion as the first predetermined portion, wherein the corner, inside and boundary nodes divide the reference mesh into either triangular or rectangular patches or a combination of both triangular and rectangular patches; and (k) means for selecting a motion model for the corner nodes; for assigning a cost polygon to each corner node; for estimating parameters of the motion model for each cost polygon; and for mapping the corner nodes with the estimated motion parameters.

Claim 9

An article of manufacture comprising: a computer usable medium having computer readable program means embodied therein for causing tracking of a first predetermined, two-dimensional portion of an image throughout a sequence of images, the computer readable program code means in said article of manufacture comprising: (a) computer readable program means for causing the computer to effect selecting a reference frame; (b) computer readable program means for causing the computer to effect selecting the first predetermined, two-dimensional portion within the reference frame by choosing a reference polygon having at least three comers that defines the boundary of the first predetermined portion; (c) computer readable program means for causing the computer to effect fitting a reference mesh having corner nodes at the comers of the reference polygon and at least one inside the reference polygon; (d) computer readable program means for causing the computer to effect predicting the reference polygon in subsequent or previous image frames by independently tracking the corner of the reference polygon; (e) computer readable program means for dividing the reference polygon and the tracked polygon into a minimum number of triangles so that each triangle in the reference polygon respectively corresponds to a triangle in the tracked polygon; for finding parameters of affine transformation between each corresponding pair of triangles; and for mapping nodes in each triangle of the reference polygon into the respective triangle in the tracked polygon using the parameters of the corresponding affine transformation used for the triangle in which the node is located; (f) computer readable program means for causing the computer to effect refining locations of the inside and corner nodes of the corresponding mesh for tracking local and global motion of the first predetermined portion; (g) means for causing the computer to effect defining the location of boundary nodes on the reference mesh for tracking the local motion around the boundary of the predetermined portion; (h) means for causing the computer to effect tracking illumination changes that occurred between the reference frame and a previous or subsequent frame; (i) means for causing said (c), (d), (e) and (f) computer readable program means to be implemented in a hierarchy of spatial resolutions; (j) means for causing the computer to effect replacing the first predetermined portion with a second predetermined portion throughout a portion of the sequence of images so that the second predetermined portion undergoes the same global and local motion as the first predetermined portion, wherein the corner, inside and boundary nodes divide the reference mesh into either triangular or rectangular patches or a combination of both triangular and rectangular patches; and (k) means for selecting a motion model for the corner nodes; for assigning a cost polygon to each corner node; for estimating parameters of the motion model for each cost polygon; and for mapping the corner nodes with the estimated motion parameters.

Claim 10

The article of manufacture as in claim 7 further comprising computer readable program means for defining a maximum range for estimating the parameters of the motion model.

Claim 11

The article of manufacture as in claim 8 further comprising computer readable program means for refining the location of boundary nodes on the tracked polygon.

Claim 12

The article of manufacture as in claim 9 further comprising computer readable program means for fitting a mesh to the second predetermined portion that corresponds in nodes and patches to the mesh in the reference polygon.

Claim 13

The article of manufacture as in claim 10 further comprising computer readable program means for finding parameters of affine transformation between each corresponding pair of patches in the second predetermined portion and the tracked polygon in the previous and subsequent image frames.

Claim 14

The article of manufacture as in claim 11 further comprising computer readable program means for mapping pixels in each patch in the second predetermined portion into the corresponding patch in the tracked polygon using the parameters of the corresponding affine transformation.