US5821943A — Apparatus and method for recreating and manipulat…

Claim 1

Independent Claim 215 words

An image processing system for generating an image of a scene from three views of the scene, the image processing system comprising: A. a planar view processor for identifying from the three planar views a set P s of at least one point P in the scene which has a respective projection in all three views; B. a trilinear tensor generator for generating from the set P s a trilinear tensor T having components α ijk conforming to α.sub.ijk =v'.sub.i b.sub.jk -v.sub.j".sub.j a.sub.ik (i, j, k=1, 2, 3), where v i ' and v j " are components of vectors v' and v", and a jk and b jk are components of matrices A and B defined such that, for each point P .OR right. P s in the set, projection points p, p' and p" onto the respective first, second and third views have coordinates in the respective view given by p= I,0!P (where "I" represents an identity matrix), p'≅ A,v'!P (where "≅" represents equality up to a scale factor), and p"≅ B,v"!P, A, B, v' and v" being the same for projection points p, p' and p" for point P .OR right. P s ; and C. an image generator for utilizing the trilinear tensor T to generate the image of said scene.

Claim 2

Dependent Claim 215 words

An image processing system for generating an image of a scene from three views of the scene, the image processing system comprising: A. a planar view processor for identifying from the three planar views a set P s of at least one point P in the scene which has a respective projection in all three views; B. a trilinear tensor generator for generating from the set P s a trilinear tensor T having components α ijk conforming to α.sub.ijk =v'.sub.i b.sub.jk -v.sub.j".sub.j a.sub.ik (i, j, k=1, 2, 3), where v i ' and v j " are components of vectors v' and v", and a jk and b jk are components of matrices A and B defined such that, for each point P .OR right. P s in the set, projection points p, p' and p" onto the respective first, second and third views have coordinates in the respective view given by p= I,0!P (where "I" represents an identity matrix), p'≅ A,v'!P (where "≅" represents equality up to a scale factor), and p"≅ B,v"!P, A, B, v' and v" being the same for projection points p, p' and p" for point P .OR right. P s ; and C. an image generator for utilizing the trilinear tensor T to generate the image of said scene.

Claim 3

Dependent Claim 31 words

An image processing system as defined in claim 1 in which said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 4

Dependent Claim 21 words

An image processing system as defined in claim 1 further comprising a planar view generator for generating said three planar views.

Claim 5

Dependent Claim 101 words

The image processing system as defined in claim 3 in which said trilinear tensor generator generates the components α ijk of the trilinear tensor T in accordance with x"α.sub.13..sup.T p-x"x'α.sub.33..sup.T p+x'α.sub.31..sup.T p-α.sub.11..sup.T p=0, y"α.sub.13..sup.T p-y"x'α.sub.33..sup.T p+x'α.sub.32..sup.T p-α.sub.12..sup.T p=0, x"α.sub.23..sup.T p-x"y'α.sub.33..sup.T p+y'α.sub.31..sup.T p-α.sub.21..sup.T p=0, and y"α.sub.23..sup.T p-y"y'α.sub.33..sup.T p+y'α.sub.32..sup.T p-α.sub.22..sup.T p=0 for said points P s , where each point p' has coordinates (x', y', 1) T and each point p" has coordinates (x", y", 1) T , a ij . for each "i" and "j" (i, j=1, 2, 3) represents a vector of said trilinear tensor, and "T" represents a transpose operation.

Claim 6

Dependent Claim 29 words

An image processing system as defined in claim 2 in which said set P s comprises seven points P, each point P having a projection in all three views.

Claim 7

Dependent Claim 21 words

An image processing system as defined in claim 1 in which said image generator generates said image as a two-dimensional image.

Claim 8

Dependent Claim 19 words

An image processing system as defined in claim 6 in which said image generator displays the image generated thereby.

Claim 9

Dependent Claim 61 words

An image processing system as defined in claim 6 in which said image generator utilizes the trilinear tensor T to generate, from coordinates for projection points in two of said views for a point P x in the scene, coordinates for a corresponding projection point in the third of said views, the corresponding projection point comprising a point in said image.

Claim 10

Dependent Claim 41 words

An image processing system as defined in claim 8 in which said image generator generates the coordinates for the corresponding projection point by plugging into the trilinear tensor T the coordinates for the projection points in said two of said views.

Claim 11

Dependent Claim 63 words

An image processing system as defined in claim 6 in which said image is of said scene at a particular point in time, the image processing system further including a comparator for comparing said image to another image of said scene at a different point in time thereby to facilitate the identification of differences in the scene at the respective points in time.

Claim 12

Dependent Claim 21 words

An image processing system as defined in claim 1 in which said image generator generates said image as a three-dimensional image.

Claim 13

Dependent Claim 35 words

An image processing system as defined in claim 11 further comprising a novel view generator for generating, from said three-dimensional image, a two-dimensional image comprising a novel view of said scene from a predetermined direction.

Claim 14

Dependent Claim 20 words

An image processing system as defined in claim 12 in which said image generator further displays the image generated thereby.

Claim 15

Dependent Claim 236 words

An image processing system as defined in claim 13, the system generating said image from more than three views, A. said planar view processor identifying, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s of at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generator for generating, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said image processing system further comprising a correspondence matrix generating element comprising: i. a correspondence matrix generator for generating, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; and ii. a concatenation element for concatenating the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; and D. said epipolar line generator generates each said epipolar line as the cross product of corresponding columns of four largest principal components of the concatenated correspondence matrix CG m .

Claim 16

Dependent Claim 37 words

An image processing system as defined in claim 14 in which said each set P s comprises a plurality of points P, each point P having a projection in all three views of the respective set "m."

Claim 17

Dependent Claim 35 words

An image processing system as defined in claim 15 in which each said set P s comprises seven points P, each point P having a projection in all three views of the respective set "m."

Claim 18

Dependent Claim 51 words

An image processing system as defined in claim 11 in which said scene includes at least one object, the image processing system further comprising an image comparator for performing a comparison operation in connection with a portion of the image corresponding to the object and a predetermined model for said object.

Claim 19

Dependent Claim 43 words

An image processing system as defined in claim 17 in which the image comparator, if it determines that a difference exists between the portion of the image corresponding to the object and the predetermined model, further compares said difference with a predetermined tolerance.

Claim 20

Dependent Claim 121 words

An image processing system as defined in claim 1 in which said image generator comprises: A. a fundamental matrix generator for generating a fundamental matrix F from the trilinear tensor T; B. an epipole generator for generating coordinates for an epipole v' in said second view; C. an affine depth generator for using the fundamental matrix F, said coordinates for said epipole v', and coordinates (x r , y r ) and (x' r , y' r ) of corresponding points p r and p' r in said respective first and second views to generate an affine depth value κ r , thereby to provide coordinates for (x, y, 1, κ r ) for a point P r in the scene.

Claim 21

Dependent Claim 58 words

An image processing system as defined in claim 19 in which said fundamental matrix generator generates the fundamental matrix F as E j T F+F T E j =0 (j=1, 2, 3), where E j are three matrices E j =α 1j1 , α 1j2 , . . . , α 3j3 and "T" represents a transpose operation.

Claim 22

Dependent Claim 251 words

An image processing system as defined in claim 20, the system generating said image from more than three views, A. said planar view processor identifying, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s having at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generator generates, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said image processing system further comprising a correspondence matrix generating element comprising: i. a correspondence matrix generator for generating, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; ii. a concatenation element for concatenating the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; and D. said fundamental matrix generator generates the fundamental matrix F as C k T F+F T C k =0 (k=1, 2, 3, 4) where each C k represents one of four largest principal components of the concatenated correspondence matrix CG m and "T" represents a transpose operation.

Claim 23

Dependent Claim 33 words

An image processing system as defined in claim 21 in which said each said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 24

Dependent Claim 30 words

An image processing system as defined in claim 22 in which each said set P s comprises seven points P, each point P having a projection in all three views.

Claim 25

Dependent Claim 31 words

An image processing system as defined in claim 19 in which said affine depth generator generates said affine depth κ in conformance with p' r ≅ v'!F p r +κ v'.

Claim 26

Dependent Claim 123 words

An image processing system as defined in claim 19 in which said image generator comprises: A. a homography generator for generating a homography matrix E i from the trilinear tensor T; B. an epipole generator for generating coordinates for an epipole v' in said second view; C. an affine depth generator for using the homography matrix E i , said coordinates for said epipole v', and coordinates (x r , y r ) and (x' r , y' r ) of corresponding points p r and p' r in said respective first and second views to generate an affine depth value κ r , thereby to provide coordinates for (x, y, 1, κ r ) for a point P r in the scene.

Claim 27

Dependent Claim 33 words

An image processing system as defined in claim 25 in which said affine depth generator generates the affine depth value κ r in accordance with p' r ≅E i p r +κ v'.

Claim 28

Dependent Claim 22 words

An image processing system as defined in claim 21 further comprising a planar view generator for generating said more than three views.

Claim 29

Dependent Claim 27 words

An image processing system as defined in claim 27 in which said epipole coordinate determination element uses a least-squares methodology to determine the coordinate of said epipole.

Claim 30

Dependent Claim 37 words

An image processing system as defined in claim 21 in which said epipole generator generates said coordinates for said epipole v' as F T v'=0, where "F" is said fundamental matrix and "T" represents a transpose operation.

Claim 31

Dependent Claim 54 words

An image processing system as defined in claim 21 in which said epipole generator comprises: A. an epipolar line generator for generating, from the tensor T, a plurality of epipolar lines; and B. an epipole coordinate determination element for determining the coordinate of said epipole as the point of intersection of said epipolar lines.

Claim 32

Dependent Claim 50 words

An image processing system as defined in claim 22 in which said epipolar line generator generates each said epipolar line as the cross product of corresponding columns of two of the three matrices E j =α 1j1 , α 1j2 , . . . , α 3j3 (j=1, 2, 3).

Claim 33

Dependent Claim 22 words

An image processing system as defined in claim 14 further comprising a planar view generator for generating said more than three views.

Claim 34

Dependent Claim 169 words

An image processing system as defined in claim 1, the system generating said image from more than three views, A. said planar view processor identifying, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s of at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generator generates, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said image processing system further comprising a correspondence element for processing said trilinear tensors T m in accordance with a rank-4 constraint, thereby to generate a set of processed trilinear tensors T m ' which are linear related to each other, the image generator utilizing at least one processed trilinear tensor for at least one subset to generate said image instead of the associated trilinear tensor for the same subset.

Claim 35

Dependent Claim 37 words

An image processing system as defined in claim 33 in which said each set P s comprises a plurality of points P, each point P having a projection in all three views of the respective set "m."

Claim 36

Dependent Claim 35 words

An image processing system as defined in claim 33 in which each said set P s comprises seven points P, each point P having a projection in all three views of the respective set "m."

Claim 37

Dependent Claim 163 words

An image processing system as defined in claim 33 in which said correspondence element comprises: A. a correspondence matrix generator for generating, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; B. a concatenation element for concatenating the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; C. a rank-4 processing element for processing said concatenated correspondence matrix CG m in accordance with the rank-4 constraint, thereby to generate a processed concatenated correspondence matrix CG m ; and D. a processed trilinear tensor generator for generating from the processed concatenated correspondence matrix CG m the set of processed trilinear tensors T m '.

Claim 38

Dependent Claim 91 words

An image processing system as defined in claim 36 in which: A. said concatenation element concatenates the correspondence matrices G m such that columns of successive ones of the correspondence matrices G m form successive sets of columns of the concatenated correspondence matrix CG m ; and B. said processed trilinear tensor generator generates each processed trilinear tensor T m ' associated with a subset "m" from the ones of the set of columns of the concatenated correspondence matrix associated with the correspondence matrix G m for the same subset "m."

Claim 39

Dependent Claim 202 words

A computer-implemented image processing method for generating an image of a scene from three views of the scene, the image processing method comprising the steps of: A. identifying from said three planar views a set P s of at least one point P in the scene which has a projection in all three views; B. generating from the set P s a trilinear tensor T having components α ijk conforming to: α.sub.ijk =v.sub.i 'b.sub.jk -v.sub.j "a.sub.ik (i, j, k=1, 2, 3), where v i ' and v j ' are components of vectors v' and v", and a jk and b jk are components of matrices A and B defined such that, for said point P .OR right. P s in the set, projection points p, p' and p" onto the respective first, second and third views have coordinates in the respective view given by p= I,0!P (where "I" represents an identity matrix), p'≅ A,v'!P (where "≅" represents equality up to a scale factor), and p"≅ B,v"!P, A, B, v' and v" being the same for projection points p, p' and p" for all points P .OR right. P s ; and C. utilizing the trilinear tensor T to generate said image.

Claim 40

Dependent Claim 202 words

A computer-implemented image processing method for generating an image of a scene from three views of the scene, the image processing method comprising the steps of: A. identifying from said three planar views a set P s of at least one point P in the scene which has a projection in all three views; B. generating from the set P s a trilinear tensor T having components α ijk conforming to: α.sub.ijk =v.sub.i 'b.sub.jk -v.sub.j "a.sub.ik (i, j, k=1, 2, 3), where v i ' and v j ' are components of vectors v' and v", and a jk and b jk are components of matrices A and B defined such that, for said point P .OR right. P s in the set, projection points p, p' and p" onto the respective first, second and third views have coordinates in the respective view given by p= I,0!P (where "I" represents an identity matrix), p'≅ A,v'!P (where "≅" represents equality up to a scale factor), and p"≅ B,v"!P, A, B, v' and v" being the same for projection points p, p' and p" for all points P .OR right. P s ; and C. utilizing the trilinear tensor T to generate said image.

Claim 41

Dependent Claim 32 words

A computer-implemented image processing method as defined in claim 38 in which said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 42

Dependent Claim 30 words

A computer-implemented image processing method as defined in claim 39 in which said set P s comprises seven points P, each point P having a projection in all three views.

Claim 43

Dependent Claim 107 words

A computer-implemented image processing method as defined in claim 40 in which said trilinear tensor generating step includes the step of generating the components α ijk of the trilinear tensor T in accordance with: x"α.sub.13..sup.T p-x"x'α.sub.33..sup.T p+x'α.sub.31..sup.T p-α.sub.11..sup.T p=0, y"α.sub.13..sup.T p-y"x'α.sub.33..sup.T p+x'α.sub.32..sup.T p-α.sub.12..sup.T p=0, x"α.sub.23..sup.T p-x"y'α.sub.33..sup.T p+y'α.sub.31..sup.T p-α.sub.21..sup.T p=0, and y"α.sub.23..sup.T p-y"y'α.sub.33..sup.T p+y'α.sub.32..sup.T p-α.sub.22..sup.T p=0, for said points P s , where each point p' has coordinates (x', y', 1) T and each point p" has coordinates (x", y", 1) T , α ij . for each "i" and "j" (i, j=1, 2, 3) represents a vector of said trilinear tensor, and "T" represents a transpose operation.

Claim 44

Dependent Claim 20 words

A computer-implemented image processing method as defined in claim 38 further comprising the step of generating said three planar views.

Claim 45

Dependent Claim 27 words

A computer-implemented image processing method as defined in claim 38 in which said image generation step includes the step of generating said image as a two-dimensional image.

Claim 46

Dependent Claim 25 words

A computer-implemented image processing method as defined in claim 43 in which said image generation step further includes the step of displaying the generated image.

Claim 47

Dependent Claim 63 words

A computer-implemented image processing method as defined in claim 43 in which said image is of said scene at a particular point in time, the image processing method further comprising the step of comparing said image to another image of said scene at a different point in time thereby to facilitate identification of differences in the scene at the respective points in time.

Claim 48

Dependent Claim 67 words

A computer-implemented image processing method as defined in claim 38 in which said image generation step includes the step of utilizing the trilinear tensor T to generate, from coordinates for projection points in two of said views for a point P x in the scene, coordinates for a corresponding projection point in the third of said views, the corresponding projection point comprising a point in said image.

Claim 49

Dependent Claim 47 words

A computer-implemented image processing method as defined in claim 46 in which said image generation step includes the step of generating the coordinates for the corresponding projection point by plugging into the trilinear tensor T the coordinates for the projection points in said two of said views.

Claim 50

Dependent Claim 33 words

A computer-implemented image processing method as defined in claim 38 in which said image generation step includes the step of utilizing the trilinear tensor T to generate a three-dimensional image of the scene.

Claim 51

Dependent Claim 34 words

A computer-implemented image processing method as defined in claim 48 further comprising the step of generating, from the three-dimensional image, a two-dimensional image comprising a novel view of said scene from a predetermined direction.

Claim 52

Dependent Claim 22 words

A computer-implemented image processing method as defined in claim 49 further comprising the step of displaying the image comprising the novel view.

Claim 53

Dependent Claim 47 words

A computer-implemented image processing method as defined in claim 48 in which said scene includes at least one object, the image processing method further comprising the step of comparing at least a portion of the image corresponding to the object and a predetermined model for said object.

Claim 54

Dependent Claim 56 words

A computer-implemented image processing method as defined in claim 51 further comprising the step of, if a difference is determined between the at least a portion of the image corresponding to the object and the predetermined model for said object, comparing said difference to a predetermined tolerance and generating an indication in response to the comparison.

Claim 55

Dependent Claim 113 words

A computer-implemented image processing method as defined in claim 51 in which said utilization step comprises the steps of: A. generating a homography matrix E i from the trilinear tensor T; B. generating coordinates for an epipole v' in said second view; C. using the homography matrix E i , said coordinates for said epipole v', and coordinates (x r , y r ) and (x' r ,y' r ) of corresponding points p r and p' r in said respective first and second views to generate an affine depth value κ r , thereby to provide coordinates for (x, y, 1, κ r ) for a point P r in the scene.

Claim 56

Dependent Claim 39 words

A computer-implemented image processing method as defined in claim 53 in which said affine depth generation step includes the step of generating the affine depth value κ r in accordance with p' r ≅E i p r +κ v'.

Claim 57

Dependent Claim 111 words

A computer-implemented image processing method as defined in claim 48 in which said utilization step includes the steps of: A. generating a fundamental matrix F from the trilinear tensor T; B. generating coordinates for an epipole v' in said second view; C. using the fundamental matrix F, said coordinates for said epipole v', and coordinates (x r , y r ) and (x' r , y' r ) of corresponding points p r and p' r in said respective first and second views to generate an affine depth value κ r , thereby to provide coordinates for (x, y, 1, κ r ) for a point P r in the scene.

Claim 58

Dependent Claim 64 words

A computer-implemented image processing method as defined in claim 55 in which said fundamental matrix generating step includes the step of generating the fundamental matrix F as E j T F+F T E j =0 (j=1, 2, 3), where E j are three matrices E j =α 1j1 , α 1j2 , . . . , α 3j3 and "T" represents a transpose operation.

Claim 59

Dependent Claim 265 words

A computer-implemented image processing method as defined in claim 55, the method generating said image from more than three planar views of a scene, in which A. said point set identifying step includes the step of identifying, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generating step includes the step of generating, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said method further includes a correspondence matrix generating step comprising the steps of: i. generating, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; and ii. concatenating the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; and D. said fundamental matrix generating step includes the step of generating the fundamental matrix F as C k T F+F T C k =0 (k=1, 2, 3, 4) where each C k represents one of four largest principal components of the concatenated correspondence matrix CG m and "T" represents a transpose operation.

Claim 60

Dependent Claim 34 words

A computer-implemented image processing method as defined in claim 57 in which said each said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 61

Dependent Claim 31 words

A computer-implemented image processing method as defined in claim 58 in which each said set P s comprises seven points P, each point P having a projection in all three views.

Claim 62

Dependent Claim 20 words

A computer-implemented image processing method as defined in claim 57 further comprising the step of generating said three planar views.

Claim 63

Dependent Claim 43 words

A computer-implemented image processing method as defined in claim 57 in which said epipole generating step includes the step of generating said coordinates for said epipole v' as F T v'=0, where "F" is said fundamental matrix and "T" represents a transpose operation.

Claim 64

Dependent Claim 48 words

A computer-implemented image processing method as defined in claim 55 in which said epipole generating step includes the steps of A. generating, from the tensor T, a plurality of epipolar lines; and B. determining the coordinate of said epipole as the point of intersection of said epipolar lines.

Claim 65

Dependent Claim 56 words

A computer-implemented image processing method as defined in claim 62 in which said epipolar line generating step includes the step of generating each said epipolar line as the cross product of corresponding columns of two of the three matrices E j =α 1j1 , α 1j2 , . . . , α 3j3 (j=1, 2, 3).

Claim 66

Dependent Claim 247 words

A computer-implemented image processing method as defined in claim 63, the method generating said image from more than three planar views of a scene, in which A. said point set identifying step includes the step of identifying, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generating step includes the step of generating, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said method further comprises a correspondence matrix generating step comprising the steps of: i. generating, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . α 3j3 ,m of the respective trilinear tensor T m ; and ii. concatenating the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; and D. said epipolar line generating step includes the step of generating each said epipolar line as the cross product of corresponding columns of four largest principal components of the concatenated correspondence matrix CG m .

Claim 67

Dependent Claim 34 words

A computer-implemented image processing method as defined in claim 64 in which said each said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 68

Dependent Claim 31 words

A computer-implemented image processing method as defined in claim 65 in which each said set P s comprises seven points P, each point P having a projection in all three views.

Claim 69

Dependent Claim 32 words

A computer-implemented image processing method as defined in claim 63 in which said epipole coordinate determination step includes the step of using a least-squares methodology to determine the coordinate of said epipole.

Claim 70

Dependent Claim 20 words

A computer-implemented image processing method as defined in claim 64 further comprising the step of generating said three planar views.

Claim 71

Dependent Claim 37 words

A computer-implemented image processing method as defined in claim 55 in which said affine depth generating step includes the step of generating said affine depth κ in conformance with p' r ≅ v'!F p r +κ v'.

Claim 72

Dependent Claim 169 words

A computer-implemented image processing method as defined in claim 38, the method generating said image from more than three planar views of a scene, in which A. said point set identifying step includes the step of identifying, for each of a plurality of subsets "m" of three of said planar views, a set P s at least one point in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generating step includes the step of generating, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said method further comprising a correspondence step for processing said trilinear tensors T m in accordance with a rank-4 constraint, thereby to generate a set of processed trilinear tensors T m ' which are linear related to each other, the utilization element utilizing at least one processed trilinear tensor for at least one subset to generate said image instead of the associated trilinear tensor for the same subset.

Claim 73

Dependent Claim 34 words

A computer-implemented image processing method as defined in claim 70 in which said each said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 74

Dependent Claim 31 words

A computer-implemented image processing method as defined in claim 71 in which each said set P s comprises seven points P, each point P having a projection in all three views.

Claim 75

Dependent Claim 20 words

A computer-implemented image processing method as defined in claim 72 further comprising the step of generating said three planar views.

Claim 76

Dependent Claim 160 words

A computer-implemented image processing method as defined in claim 70 in which said correspondence step comprises the steps of: A. generating, for each trilinear tensor T m associated with a subset of views "m," two of said views being the same for all of said subsets "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; B. concatenating the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; C. processing said concatenated correspondence matrix CG m in accordance with the rank-4 constraint, thereby to generate a processed concatenated correspondence matrix CG m ; and D. generating from the processed concatenated correspondence matrix CG m the set of processed trilinear tensors T m '.

Claim 77

Dependent Claim 101 words

A computer-implemented image processing method as defined in claim 74 in which: A. said concatenation step includes the step of concatenating the correspondence matrices G m such that columns of successive ones of the correspondence matrices G m form successive sets of columns of the concatenated correspondence matrix CG m ; and B. said processed trilinear tensor generating step includes the step of generating each processed trilinear tensor T m ' associated with a subset "m" from the ones of the set of columns of the concatenated correspondence matrix associated with the correspondence matrix G m for the same subset "m."

Claim 78

Dependent Claim 227 words

A computer program product for enabling a computer to generate an image of a scene from three views of the scene, the three views having a set P s of at least one point P in the scene which has a respective projection in all three views, the computer program product comprising a computer-readable medium having encoded thereon: A. a trilinear tensor generation module for enabling said computer to generate from the set P s a trilinear tensor T having components α ijk conforming to α.sub.ijk =v.sub.i 'b.sub.jk -v.sub.j "a.sub.ik (i, j, k=1, 2, 3), where v i ' and v j " are components of vectors v' and v", and a jk and b jk are components of matrices A and B defined such that, for each point P .OR right. P s in the set, projection points p, p' and p" onto the respective first, second and third views have coordinates in the respective view given by p= I,0!P (where "I" represents an identity matrix), p'≅ A,v'!P (where "≅" represents equality up to a scale factor), and p"≅ B,v"!P, A, B, v' and v" being the same for projection points p, p' and p" for point P .OR right. P s ; and B. an image generation module for enabling said computer to utilize the trilinear tensor T to generate the image of said scene.

Claim 79

Dependent Claim 227 words

A computer program product for enabling a computer to generate an image of a scene from three views of the scene, the three views having a set P s of at least one point P in the scene which has a respective projection in all three views, the computer program product comprising a computer-readable medium having encoded thereon: A. a trilinear tensor generation module for enabling said computer to generate from the set P s a trilinear tensor T having components α ijk conforming to α.sub.ijk =v.sub.i 'b.sub.jk -v.sub.j "a.sub.ik (i, j, k=1, 2, 3), where v i ' and v j " are components of vectors v' and v", and a jk and b jk are components of matrices A and B defined such that, for each point P .OR right. P s in the set, projection points p, p' and p" onto the respective first, second and third views have coordinates in the respective view given by p= I,0!P (where "I" represents an identity matrix), p'≅ A,v'!P (where "≅" represents equality up to a scale factor), and p"≅ B,v"!P, A, B, v' and v" being the same for projection points p, p' and p" for point P .OR right. P s ; and B. an image generation module for enabling said computer to utilize the trilinear tensor T to generate the image of said scene.

Claim 80

Dependent Claim 49 words

A computer program product as defined in claim 76 further comprising a planar view processing module for enabling said computer to identify from the three planar views the set P s of at least one point P in the scene which has a respective projection in all three views.

Claim 81

Dependent Claim 26 words

A computer program product as defined in claim 77 in which said image generation module enables said computer to generate said image as a two-dimensional image.

Claim 82

Dependent Claim 25 words

A computer program product as defined in claim 78 in which said image generator module further enables said computer to display the image generated thereby.

Claim 83

Dependent Claim 66 words

A computer program product as defined in claim 78 in which said image generation module enables said computer to utilize the trilinear tensor T to generate, from coordinates for projection points in two of said views for a point P x in the scene, coordinates for a corresponding projection point in the third of said views, the corresponding projection point comprising a point in said image.

Claim 84

Dependent Claim 46 words

A computer program product as defined in claim 80 in which said image generation module enables said computer to generate the coordinates for the corresponding projection point by plugging into the trilinear tensor T the coordinates for the projection points in said two of said views.

Claim 85

Dependent Claim 68 words

A computer program product as defined in claim 78 in which said image is of said scene at a particular point in time, the computer program product further including a comparator module for enabling said computer to compare said image to another image of said scene at a different point in time thereby to facilitate the identification of differences in the scene at the respective points in time.

Claim 86

Dependent Claim 31 words

A computer program product as defined in claim 76 in which said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 87

Dependent Claim 28 words

A computer program as defined in claim 83 in which said set P s comprises seven points P, each point P having a projection in all three views.

Claim 88

Dependent Claim 26 words

A computer program product as defined in claim 76 further comprising a planar view generator module for enabling said computer to generate said three planar views.

Claim 89

Dependent Claim 106 words

A computer program product as defined in claim 85 in which said trilinear tensor generation module enables said computer to generate the components α ijk of the trilinear tensor T in accordance with x"α.sub.13..sup.T p-x"x'α.sub.33..sup.T p+x'α.sub.31..sup.T p-α.sub.11..sup.T p=0, y"α.sub.13..sup.T p-y"x'α.sub.33..sup.T p+x'α.sub.32..sup.T p-α.sub.12..sup.T p=0, x"α.sub.23..sup.T p-x"y'α.sub.33..sup.T p+y'α.sub.31..sup.T p-α.sub.21..sup.T p=0, and y"α.sub.23..sup.T p-y"y'α.sub.33..sup.T p+y'α.sub.32..sup.T p-α.sub.22..sup.T p=0, for said points P s , where each point p' has coordinates (x', y', 1) T and each point p" has coordinates (x", y", 1) T , α ij . for each "i" and "j" (i, j=1, 2, 3) represents a vector of said trilinear tensor, and "T" represents a transpose operation.

Claim 90

Dependent Claim 26 words

A computer program product as defined in claim 76 in which said image generator module enables said computer to generate said image as a three-dimensional image.

Claim 91

Dependent Claim 40 words

A computer program product as defined in claim 87 further comprising a novel view generator module for enabling said computer to generate, from said three-dimensional image, a two-dimensional image comprising a novel view of said scene from a predetermined direction.

Claim 92

Dependent Claim 25 words

A computer program product as defined in claim 88 in which said image generator module further enables said computer to display the image generated thereby.

Claim 93

Dependent Claim 271 words

A computer program product as defined in claim 89, the computer program product enabling said computer to generate said image from more than three views, A. said computer program product comprising a planar view processor module for enabling said computer to identify, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s of at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generator module enabling said computer to generate, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said computer program product further comprising a correspondence matrix generating module comprising: i. a correspondence matrix generator module for enabling said computer to generate, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; and ii. a concatenation module for enabling said computer to concatenate the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; and D. said epipolar line generator module enabling said computer to generate each said epipolar line as the cross product of corresponding columns of four largest principal components of the concatenated correspondence matrix CG m .

Claim 94

Dependent Claim 37 words

A computer program product as defined in claim 90 in which said each set P s comprises a plurality of points P, each point P having a projection in all three views of the respective set "m."

Claim 95

Dependent Claim 35 words

A computer program product as defined in claim 91 in which each said set P s comprises seven points P, each point P having a projection in all three views of the respective set "m."

Claim 96

Dependent Claim 27 words

A computer program product as defined in claim 90 further comprising a planar view generator module for enabling said computer to generate said more than three views.

Claim 97

Dependent Claim 56 words

A computer program product as defined in claim 87 in which said scene includes at least one object, the computer program product further comprising an image comparator module for enabling said computer to perform a comparison operation in connection with a portion of the image corresponding to the object and a predetermined model for said object.

Claim 98

Dependent Claim 49 words

A computer program product as defined in claim 94 in which the image comparator module further enables said computer to, if it determines that a difference exists between the portion of the image corresponding to the object and the predetermined model, further compare said difference with a predetermined tolerance.

Claim 99

Dependent Claim 137 words

A computer program product as defined in claim 87 in which said image generator module comprises: A. a fundamental matrix generator module for enabling said computer to generate a fundamental matrix F from the trilinear tensor T; B. an epipole generator module for enabling said computer to generate coordinates for an epipole v' in said second view; C. an affine depth generator module for enabling said computer to use the fundamental matrix F, said coordinates for said epipole v', and coordinates (x r , y r ) and (x' r , y' r ) of corresponding points p r and p' r in said respective first and second views to generate an affine depth value κ r , thereby to provide coordinates for (x, y, 1, κ r ) for a point P r in the scene.

Claim 100

Dependent Claim 64 words

A computer program product as defined in claim 96 in which said fundamental matrix generator module for enabling said computer to generate the fundamental matrix F as E j T F+F T E j =0 (j=1, 2, 3), where E j are three matrices E j =α 1j1 , α 1j2 , . . . , α 3j3 and "T" represents a transpose operation.

Claim 101

Dependent Claim 288 words

A computer program product as defined in claim 97, the computer program product enabling said computer to generate said image from more than three views, A. the computer program product including a planar view processor module for enabling said computer to identify, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s having at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generator module for enabling said computer to generate, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said computer program product further comprising a correspondence matrix generating module comprising: i. a correspondence matrix generator module for enabling said computer to generate, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; ii. a concatenation module for enabling said computer to concatenate the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; and D. said fundamental matrix generator module enabling said computer to generate the fundamental matrix F as C k T F+F T C k =0 (k=1, 2, 3, 4) where each C k represents one of four largest principal components of the concatenated correspondence matrix CG m and "T" represents a transpose operation.

Claim 102

Dependent Claim 33 words

A computer program product as defined in claim 98 in which said each said set P s comprises a plurality of points P, each point P having a projection in all three views.

Claim 103

Dependent Claim 30 words

A computer program product as defined in claim 91 in which each said set P s comprises seven points P, each point P having a projection in all three views.

Claim 104

Dependent Claim 37 words

A computer program product as defined in claim 96 in which said affine depth generator module for enabling said computer to generate said affine depth κ in conformance with p' r ≅ v'!F p r +κ v'.

Claim 105

Dependent Claim 139 words

A computer program product as defined in claim 96 in which said image generator module comprises: A. a homography generator module for enabling said computer to generate a homography matrix E i from the trilinear tensor T; B. an epipole generator module for enabling said computer to generate coordinates for an epipole v' in said second view; C. an affine depth generator module for enabling said computer to use the homography matrix E i , said coordinates for said epipole v', and coordinates (x r , y r ) and (x' r , y' r ) of corresponding points p r and p' r in said respective first and second views to generate an affine depth value ε r , thereby to provide coordinates for (x, y, 1, κ r ) for a point P r in the scene.

Claim 106

Dependent Claim 39 words

A computer program product as defined in claim 102 in which said affine depth generator module for enabling said computer to generate the affine depth value κ r in accordance with p' r ≅E i p r +κ v'.

Claim 107

Dependent Claim 27 words

A computer program product as defined in claim 98 further comprising a planar view generator module for enabling said computer to generate said more than three views.

Claim 108

Dependent Claim 32 words

A computer program product as defined in claim 104 in which said epipole coordinate determination module for enabling said computer to use a least-squares methodology to determine the coordinate of said epipole.

Claim 109

Dependent Claim 42 words

A computer program product as defined in claim 98 in which said epipole generator module enables said computer to generate said coordinates for said epipole v' as F T v'=0, where "F" is said fundamental matrix and "T" represents a transpose operation.

Claim 110

Dependent Claim 64 words

A computer program product as defined in claim 98 in which said epipole generator module comprises: A. an epipolar line generator module for enabling said computer to generate, from the tensor T, a plurality of epipolar lines; and B. an epipole coordinate determination module for enabling said computer to determine the coordinate of said epipole as the point of intersection of said epipolar lines.

Claim 111

Dependent Claim 55 words

A computer program product as defined in claim 91 in which said epipolar line generator module enables said computer to generate each said epipolar line as the cross product of corresponding columns of two of the three matrices E j =α 1j1 , α 1j2 , . . . , α 3j3 (j=1, 2, 3).

Claim 112

Dependent Claim 195 words

A computer program product as defined in claim 96, the computer program product enabling said computer to generate said image from more than three views, A. said computer program product including a planar view processor module for enabling said computer to identify, for each of a plurality of subsets "m" of three of said planar views, two of said views being the same for all of said subsets "m," a set P s of at least one point P in the scene which has a projection in all three views of the respective subset; B. said trilinear tensor generator module enabling said computer to generate, for each subset "m," a trilinear tensor T m having components α ijk ,m ; C. said computer program product further comprising a correspondence module for enabling said computer to process said trilinear tensors T m in accordance with a rank-4 constraint, thereby to generate a set of processed trilinear tensors T m ' which are linear related to each other, the image generator utilizing at least one processed trilinear tensor for at least one subset to generate said image instead of the associated trilinear tensor for the same subset.

Claim 113

Dependent Claim 37 words

A computer program product as defined in claim 109 in which said each set P s comprises a plurality of points P, each point P having a projection in all three views of the respective set "m."

Claim 114

Dependent Claim 35 words

A computer program product as defined in claim 109 in which each said set P s comprises seven points P, each point P having a projection in all three views of the respective set "m."

Claim 115

Dependent Claim 181 words

A computer program product as defined in claim 109 in which said correspondence module comprises: A. a correspondence matrix generator module for enabling said computer to generate, for each trilinear tensor T m associated with a subset of views "m," a two-dimensional correspondence matrix G m having a plurality of rows and columns, each column "j" of the correspondence matrix G m containing the elements α 1j1 ,m, α 1j2 ,m, . . . , α 3j3 ,m of the respective trilinear tensor T m ; B. a concatenation module for enabling said computer to concatenate the correspondence matrices G m for the respective subsets together, thereby to generate a concatenated correspondence matrix CG m ; C. a rank-4 processing module for enabling said computer to process said concatenated correspondence matrix CG m in accordance with the rank-4 constraint, thereby to generate a processed concatenated correspondence matrix CG m ; and D. a processed trilinear tensor generator module for enabling said computer to generate from the processed concatenated correspondence matrix CG m the set of processed trilinear tensors T m '.

Claim 116

Dependent Claim 100 words

A computer program product as defined in claim 112 in which: A. said concatenation module enables said computer to concatenate the correspondence matrices G m such that columns of successive ones of the correspondence matrices G m form successive sets of columns of the concatenated correspondence matrix CG m ; and B. said processed trilinear tensor generator module enables said computer to generate each processed trilinear tensor T m ' associated with a subset "m" from the ones of the set of columns of the concatenated correspondence matrix associated with the correspondence matrix G m for the same subset "m."

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2001-10-12	2002-04-13	2002-10-13	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2005-10-12	2006-04-13	2006-10-13	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2009-10-12	2010-04-13	2010-10-13	$7700.00	Window Closed / Lapsed

Apparatus and method for recreating and manipulating a 3D object based on a 2D projection thereof

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag