Compression and editing of movies by multi-image morphing

Claim 3

A computer program stored on a computer-readable medium for compressing a movie having an original frame sequence, the program comprising instructions to: generate a pixel level frame to frame correspondence; represent the original frame sequence as a multi-image morph, wherein instructions to represent the original frame sequence as a multi-image morph comprise instructions to represent each frame in the sequence as a point (a 1 , a 2 , . . . , a n ) in an n-dimensional affine morph space defined by a set of warping mappings W i and associated color definitions f i (1≦i≦n), where each a i is a number and each W i and f i is an operator on a planar image space, the coefficients a i satisfying the constraint a 1 +a 2 + . . . +a n =1; derive from the multi-image morph a morph basis of feature vectors describing principal changes among frames; and represent the movie in terms of the morph basis.

Claim 4

A computer program stored on a computer-readable medium for compressing a movie having an original frame sequence, the program comprising instructions to: generate a pixel level frame to frame correspondence; represent the original frame sequence as a multi-image morph, wherein instructions to represent the original frame sequence as a multi-image morph comprise instructions to represent each frame in the sequence as a point (a 1 , a 2 , . . . , a n ) in an n-dimensional affine morph space defined by a set of warping mappings W i and associated color definitions f i (1≦i≦n), where each a i is a number and each W i and f i is an operator on a planar image space, the coefficients a i satisfying the constraint a 1 +a 2 + . . . +a n =1; derive from the multi-image morph a morph basis of feature vectors describing principal changes among frames; and represent the movie in terms of the morph basis.

Claim 7

A computer program stored on a computer-readable medium for compressing a movie having an original frame sequence and for generating a motion blur in the movie, the program comprising instructions to: generate a pixel level frame to frame correspondence; represent the original frame sequence as a multi-image morph; derive from the multi-image morph a morph basis of feature vectors describing principal changes among frames; represent the movie in terms of the morph basis; and calculate an average color to substitute for an original color for each point in a blurred frame in the movie, the average color being calculated by averaging colors taken from each basis frame along a velocity field generated by a tangent vector and the warping transformations at the blurred frame, the tangent vector being a vector tangent to a curve representing the movie in a morph space defined by the morph basis.

Claim 8

A computer program stored on a computer-readable medium for compressing a movie having an original frame sequence and for generating a motion blur in the movie, the program comprising instructions to: generate a pixel level frame to frame correspondence; represent the original frame sequence as a multi-image morph; derive from the multi-image morph a morph basis of feature vectors describing principal changes among frames; represent the movie in terms of the morph basis; and calculate an average color to substitute for an original color for each point in a blurred frame in the movie, the average color being calculated by averaging colors taken from each basis frame along a velocity field generated by a tangent vector and the warping transformations at the blurred frame, the tangent vector being a vector tangent to a curve representing the movie in a morph space defined by the morph basis.

Claim 9

The method of claim 1 , wherein the correspondence generating step processes the frames using an optical flow method.

Claim 10

The method of claim 1 , wherein the deriving step uses a principal components method.

Claim 11

The method of claim 6 , further comprising compensating for vibrations of movie frames represented in terms of the morph basis by discarding one or more high weight principal components.

Claim 12

The method of claim 1 , wherein the movie has a first frame rate, further comprising resampling the movie represented in terms of the morph basis at a second frame rate.

Claim 13

The method of claim 8 , wherein the movie is represented as a curve defined in an affine morph space and resampling the movie comprises calculating points on the curve at intervals defined by the second frame rate.

Claim 14

The method of claim 1 , wherein the movie has a character with a gesture, further comprising editing the gesture.

Claim 17

The method of claim 1 , further comprising: selecting a compression level; and using the compression level to limit the number of feature vectors in the morph basis.

Claim 18

The method of claim 2 , further comprising: constructing a set of new basis frames from the basis of feature vectors, each new basis frame corresponding to one of the feature vectors in the basis of feature vectors, wherein the new basis frames are not in the original frame sequence.

Claim 19

The method of claim 1 , further comprising representing the original frame sequence as a parametric curve fit to the points (a 1 , a 2 , . . . , a n ) in the morph space.

Claim 20

The program of claims 4 or 2 , wherein the correspondence generating instruction processes the frames using an optical flow method.

Claim 21

The program of claims 4 or 2 , wherein the deriving instruction uses a principal components method.

Claim 22

The program of claim 17 , further comprising instructions for compensating for vibrations of movie frames represented in terms of the morph basis by discarding one or more high weight principal components.

Claim 23

The program of claim 2 , wherein the movie has a first frame rate, further comprising instructions for resampling the movie represented in terms of the morph basis at a second frame rate.

Claim 24

The program of claim 19 , wherein the movie is represented as a curve defined in an affine morph space and resampling the movie comprises calculating points on the curve at intervals defined by the second frame rate.

Claim 25

The program of claims 4 or 2 , wherein the movie has a character with a gesture, further comprising instructions for editing the gesture.

Claim 26

The program of claims 4 or 2 , further comprising instructions to compress the movie by using a subset of the basis.

Claim 27

The program of claims 4 or 2 , further comprising instructions to display the movie by performing a multi-image morphing.

Claim 28

The program of claim 2 , further comprising instructions to: select a compression level; and use the compression level to limit the number of feature vectors in the morph basis.

Claim 29

The program of claim 2 , further comprising instructions to: construct a set of new basis frames from the basis of feature vectors, each new basis frame corresponding to one of the feature vectors in the basis of feature vectors, wherein the new basis frames are not in the original frame sequence.

Claim 30

The program of claim 2 , further comprising instructions to represent the original frame sequence as a parametric curve fit to the points (a 1 , a 2 , . . . , a n ) in the morph space.

Claim 31

The program of claim 2 for generating a motion blur in the movie, the program further comprising instructions to: calculate an average color to substitute for an original color for each point in a blurred frame in the movie, the average color being calculated by averaging colors taken from each basis frame along a velocity field generated by a tangent vector and the warping transformations at the blurred frame, the tangent vector being a vector tangent to a curve representing the movie in a morph space defined by the morph basis.