Head motion estimation from four feature points

Claim 2

A linear method for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of only four points P of said first facial image where P ={p 1 , p 2 , p 3 , p 4 }, and p k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of only four points P′ of said second facial image where P′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =(x′ k , y′ k ); and determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′.

Claim 3

The linear method of claim 1 , wherein said only four points P of said first facial image and said only four points P′ of said second facial image include locations of outer corners of each eye and mouth of each respective first and second facial images.

Claim 4

The linear method of claim 1 , wherein said head motion estimation is governed according to: P′ i =RP i +T , where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 and T=[T 1 T 2 T 3 ] T represent camera rotation and translation respectively, said head pose estimation being a specific instance of head motion estimation.

Claim 5

A linear method for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of four points P of said first facial image where P={p 1 , p 2 , p 3 , p 4 }, and p k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of four points P′ of said second facial image where P′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =[x′ k , y′ k ]); and, determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′, wherein said head motion estimation is governed according to: P′ i =RP i +T , where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 and T−[T 1 T 2 T 3 ] T represent camera rotation and translation respectively, said head pose estimation being a specific instance of head motion estimation, and wherein said head motion estimation is governed according to said rotation matrix R, said method further comprising the steps of: determining rotation matrix R that maps points P k to F k for characterizing a head pose, said points F 1 , F 2 , F 3 , F 4 representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P k is the three-dimensional (3-D) coordinates of an arbitrary point where P i =[X i Y i Z i ] T , said mapping governed according to the relation: R ( P 2 −P 1 )∝[1 0 0] T R ( P 6 −P 5 )∝[0 1 0] T wherein P 5 and P 6 are midpoints of respective line segments connecting points P 1 P 2 and P 3 P 4 and, line segment connecting points P 1 P 2 is orthogonal to a line segment connecting points P 5 P 6 , and ∝ indicates a proportionality factor.

Claim 6

A linear method for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of four points P of said first facial image where P={p 1 , p 2 , p 3 , p 4 }, and p k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of four points P′ of said second facial image where P′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =[x′ k , y′ k ]); and, determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′, wherein said head motion estimation is governed according to: P′ i =RP i +T , where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 and T−[T 1 T 2 T 3 ] T represent camera rotation and translation respectively, said head pose estimation being a specific instance of head motion estimation, and wherein said head motion estimation is governed according to said rotation matrix R, said method further comprising the steps of: determining rotation matrix R that maps points P k to F k for characterizing a head pose, said points F 1 , F 2 , F 3 , F 4 representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P k is the three-dimensional (3-D) coordinates of an arbitrary point where P i =[X i Y i Z i ] T , said mapping governed according to the relation: R ( P 2 −P 1 )∝[1 0 0] T R ( P 6 −P 5 )∝[0 1 0] T wherein P 5 and P 6 are midpoints of respective line segments connecting points P 1 P 2 and P 3 P 4 and, line segment connecting points P 1 P 2 is orthogonal to a line segment connecting points P 5 P 6 , and ∝ indicates a proportionality factor.

Claim 7

The linear method of claim 4 , wherein components r 1 , r 2 and r 3 are computed as: r T 2 ( P 2 −P 1 )=0 r T 3 ( P 2 −P 1 )=0 r T 1 ( P 6 −P 5 )=0 r T 3 ( P 6 −P 5 )=0.

Claim 8

The linear method of claim 5 , wherein components r 1 , r 2 and r 3 are computed as: r 3 =( P 6 −P 5 )×( P 2 −P 1 ), r 2 =r 3 ×( P 2 −P 1 ) r 1 =r 3 ×r 3 .

Claim 9

The linear method of claim 4 , wherein [ P i T 0 T 0 T 1 0 0 0 T P i T 0 T 0 1 0 0 T 0 T P i T 0 0 1 ] ⁡ [ r 1 r 2 r 3 T ] = P i ′ , each point pair yielding 3 equations, whereby at least four point pairs are necessary to linearly solve for said rotation and translation.

Claim 10

The linear method of claim 7 , further comprising the step of: decomposing said rotation matrix R using Singular Value Decomposition (SVD) to obtain a form R=USV r .

Claim 11

The linear method of claim 7 , further comprising the step of computing a new rotation matrix according to R=UV r .

Claim 12

A linear method for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining image position of four points P k of a facial image; determining a rotation matrix R that maps points P k to F k for characterizing a head pose, said points F 1 , F 2 , F 3 , F 4 representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P k is the three-dimensional (3-D) coordinates of an arbitrary point where P i =[X i Y i Z i ] T , said mapping governed according to the relation: R ( P 2 −P 1 )∝[1 0 0] T R ( P 6 −P 5 )∝[0 1 0] T wherein P 5 and P 6 are midpoints of respective line segments connecting points P 1 P 2 and P 3 P 4 and, line segment connecting points P 1 P 2 is orthogonal to a line segment connecting points P 5 P 6 , and ∝ indicates a proportionality factor.

Claim 13

A linear method for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining image position of four points P k of a facial image; determining a rotation matrix R that maps points P k to F k for characterizing a head pose, said points F 1 , F 2 , F 3 , F 4 representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P k is the three-dimensional (3-D) coordinates of an arbitrary point where P i =[X i Y i Z i ] T , said mapping governed according to the relation: R ( P 2 −P 1 )∝[1 0 0] T R ( P 6 −P 5 )∝[0 1 0] T wherein P 5 and P 6 are midpoints of respective line segments connecting points P 1 P 2 and P 3 P 4 and, line segment connecting points P 1 P 2 is orthogonal to a line segment connecting points P 5 P 6 , and ∝ indicates a proportionality factor.

Claim 14

The linear method of claim 10 , wherein components r 1 , r 2 and r 3 are computed as: r r 1 ( P 2 −P 1 )=0 r r 3 ( P 2 −P 1 )=0 r r 1 ( P 6 −P 5 )=0 r r 3 ( P 6 −P 5 )=0.

Claim 15

The linear method of claim 11 , wherein components r 1 , r 2 and r 3 are computed as: r 3 =( P 6 −P 5 )=( P 2 −P 1 ) r 2 =r 3 ×( P 2 −P 1 ) r 1 =r 2 ×r 3 .

Claim 16

The linear method of claim 12 , wherein a motion of head points is represented according to P′ i =RP i =T where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 represents image rotation, T=[T 1 T 2 T 3 ] r represents translation, and P′ i denotes a 3-D image position of four points P k of another facial image.

Claim 17

The linear method of claim 13 , wherein [ P i T 0 T 0 T 1 0 0 0 T P i T 0 T 0 1 0 0 T 0 T P i T 0 0 1 ] ⁡ [ r 1 r 2 r 3 T ] = P i ′ , each point pair yielding 3 equations, whereby at least four point pairs are necessary to linearly solve for said rotation and translation.

Claim 18

The linear method of claim 14 , further comprising the step of: decomposing said rotation matrix R using Singular Value Decomposition (SVD) to obtain a form R=USV r .

Claim 19

The linear method of claim 15 , further comprising the step of computing a new rotation matrix according to R=UV r .

Claim 20

A program storage device readable by machine, tangible embodying a program of instructions executable by the machine to perform method steps for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of only four points P of said first facial image where P={P 1 , P 2 , P 3 , P 4 }, and P k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of only four points P′ of said second facial image where ′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =(x′ k ); and, determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′.

Claim 21

A program storage device readable by machine, tangible embodying a program of instructions executable by the machine to perform method steps for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of only four points P of said first facial image where P={P 1 , P 2 , P 3 , P 4 }, and P k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of only four points P′ of said second facial image where ′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =(x′ k ); and, determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′.

Claim 22

The program storage device readable by machine as claimed in claim 17 , wherein said only four points P of said first facial image and only four points P′ of said second facial image include locations of outer corners of each eye and mouth of each respective first and second facial image.

Claim 23

The program storage device readable by machine as claimed in claim 17 , wherein said head motion estimation is governed according to: P′ i =RP i +T , where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 and T=[T 1 T 2 T 3 ] r represent camera rotation and translation respectively, said head pose estimation being a specific instance of head motion estimation.

Claim 24

A program storage device readable by machine, tangible embodying a program of instructions executable by the machine to perform method steps for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of four points P of said first facial image where P={P 1 , P 2 , P 3 , P 4 }, and P k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of four points P′ of said second facial image where P′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =(x′ k , y′); and determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′, wherein said head motion estimation is governed according to: Pv′ i =RP i +T , where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 and T=[T 1 T 2 T 3 ] r represent camera rotation and translation respectively, said head pose estimation being a specific instance of head motion estimation, and wherein said head pose estimation is governed according to said rotation matrix R, said me:hod further comprising the steps of: determining rotation matrix R that maps points P k to F k for characterizing a head pose, said points F 1 , F 2 , F 3 , F 4 representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P k is the three-dimensional (3-D) coordinates of an arbitrary point where P i =[X i Y i Z i ], said mapping governed according to the relation: R ( P 2 −P 1 )∝[1 0 0] r R ( P 6 −P 5 )∝[0 1 0] r wherein P 5 and P 6 are midpoints of respective line segments connecting points P 1 P 2 and P 3 P 4 and, line segment connecting points P 1 P 2 is orthogonal to a line segment connecting points P 5 P 6 , and ∝ indicates a proportionality factor.

Claim 25

A program storage device readable by machine, tangible embodying a program of instructions executable by the machine to perform method steps for performing head motion estimation from facial feature data, the method comprising the steps of: obtaining a first facial image and detecting a head in said first image; detecting position of four points P of said first facial image where P={P 1 , P 2 , P 3 , P 4 }, and P k =(x k , y k ); obtaining a second facial image and detecting a head in said second image; detecting position of four points P′ of said second facial image where P′={p′ 1 , p′ 2 , p′ 3 , p′ 4 }and p′ k =(x′ k , y′); and determining the motion of the head represented by a rotation matrix R and translation vector T using said points P and P′, wherein said head motion estimation is governed according to: Pv′ i =RP i +T , where R = [ r 1 T r 2 T r 3 T ] = [ r ij ] 3 × 3 and T=[T 1 T 2 T 3 ] r represent camera rotation and translation respectively, said head pose estimation being a specific instance of head motion estimation, and wherein said head pose estimation is governed according to said rotation matrix R, said me:hod further comprising the steps of: determining rotation matrix R that maps points P k to F k for characterizing a head pose, said points F 1 , F 2 , F 3 , F 4 representing three-dimensional (3-D) coordinates of the respective four points of a reference, frontal view of said facial image, and P k is the three-dimensional (3-D) coordinates of an arbitrary point where P i =[X i Y i Z i ], said mapping governed according to the relation: R ( P 2 −P 1 )∝[1 0 0] r R ( P 6 −P 5 )∝[0 1 0] r wherein P 5 and P 6 are midpoints of respective line segments connecting points P 1 P 2 and P 3 P 4 and, line segment connecting points P 1 P 2 is orthogonal to a line segment connecting points P 5 P 6 , and ∝ indicates a proportionality factor.

Claim 26

The program storage device readable by machine as claimed in claim 20 , wherein components r 1 , r 2 and r 3 are computed as: r r 2 ( P 2 −P 1 )=0 r r 3 ( P 2 −P 1 )=0 r r 1 ( P 6 −P 5 )=0 r r 3 ( P 6 −P 5 )=0.

Claim 27

The program storage device readable by machine as claimed in claim 20 , wherein components r 1 , r 2 and r 3 are computed as: r 3 =( 6 −p 5 )×( P 2 −P 1 ) r 2 =r 3 ×( P 2 −P 1 ) r 1 =r 2 ×r 3 .

Claim 28

The program storage device readable by machine as claimed in claim 20 , wherein [ P i T 0 T 0 T 1 0 0 0 T P i T 0 T 0 1 0 0 T 0 T P i T 0 0 1 ] ⁡ [ r 1 r 2 r 3 T ] = P i ′ , each point pair yielding 3 equations, whereby at least four point pairs are necessary to linearly solve for said rotation and translation.

Claim 29

The program storage device readable by machine as claimed in claim 23 , further comprising the steps of decomposing said rotation matrix R using Singular Value Decomposition (SVD) to obtain a form R=USV r .

Claim 30

The program storage device readable by machine as claimed in claim 23 , further comprising the steps of computing a new rotation matrix according to R=UV r .