System and method for performing video coding/decoding using motion compensation

Claim 2

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading the original image and the reference image from the first frame memory and the second frame memory, performing a motion estimation process and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including an interpolation circuit for determining by interpolation the luminance value of at least a point lacking a pixel and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively on said reference image, and synthesizing a high-resolution reference image with a pixel density m1 times and m2 times larger in horizontal and vertical directions respectively, a third memory for storing the high-resolution reference image synthesized at said interpolation circuit, and a matching circuit for searching for a selected one of a motion vector and a motion parameter minimizing the prediction error between said original image and said reference image and outputting motion information, wherein said matching circuit reads from said third memory the luminance value of at least a pixel located on said high-resolution image nearest to a corresponding point lacking a pixel on said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point lacking a pixel on the reference image becomes required.

Claim 3

A video coder according to claim 1, wherein the values of said m1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 4

A video coder according to claim 2, wherein the values of said m1 and m2 are both 2.

Claim 5

A video coder according to claim 2, wherein the values of said m1 and m2 are both 4.

Claim 6

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories, performing a motion estimation process and outputting motion information; a synthesis circuit for synthesizing a predicted image using said reference image read from said second frame memory and said motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including an interpolation circuit for reading a part of said reference image from the second frame memory, determining by interpolation the luminance value of at least a point lacking a pixel and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, in said part of the reference image, and synthesizing a part of a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a third memory for storing the part of the high-resolution reference image synthesized at said interpolation circuit, and a matching circuit for searching for a selected one of a motion vector of a grid point and a motion parameter of a transformation function minimizing the prediction error between the original image and the reference image and outputting motion information; wherein said matching circuit reads from said third memory the luminance value of at least a pixel located on a part of said high-resolution image nearest to a corresponding point lacking a pixel on a part of said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point becomes required.

Claim 7

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories, performing a motion estimation process and outputting motion information; a synthesis circuit for synthesizing a predicted image using said reference image read from said second frame memory and said motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including an interpolation circuit for reading a part of said reference image from the second frame memory, determining by interpolation the luminance value of at least a point lacking a pixel and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, in said part of the reference image, and synthesizing a part of a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a third memory for storing the part of the high-resolution reference image synthesized at said interpolation circuit, and a matching circuit for searching for a selected one of a motion vector of a grid point and a motion parameter of a transformation function minimizing the prediction error between the original image and the reference image and outputting motion information; wherein said matching circuit reads from said third memory the luminance value of at least a pixel located on a part of said high-resolution image nearest to a corresponding point lacking a pixel on a part of said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point becomes required.

Claim 8

A video coder according to claim 5, wherein the values of said m1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 9

A video coder according to claim 6, wherein the values of said m1 and m2 are both 2.

Claim 10

A video coder according to claim 6, wherein the values of said m1 and m2 are both 4.

Claim 11

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation method in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories respectively, performing the motion estimation and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including first circuit means for reading said original image and said reference image from said first and second frame memories respectively and carrying out coarse motion estimation of grid points on said reference image; a grid point coordinate memory for storing the coordinate information of grid points on said reference image from said first circuit means; an interpolation circuit for reading a part of said reference image from said second frame memory, determining by interpolation the luminance value of at least a point having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively lacking a pixel on said part of said reference image, and synthesizing a part of a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a matching circuit for reading the coordinate information of a grid point to be corrected and at least a nearby grid point from said grid point coordinate memory, performing the correction process by hexagonal matching using the part of the original image read from said first frame memory and the part of the high-resolution image synthesized at said interpolation circuit, and rewriting the coordinate information of the grid point on said grid point coordinate memory; and a motion vector computation circuit for reading the coordinate information of a grid point corrected by said matching circuit from said grid point coordinate memory, computing a motion vector of said grid point, and outputting motion information; wherein said matching circuit reads the luminance value of at least a pixel located on a part of said high-resolution image nearest to a corresponding point lacking a pixel on a part of said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point becomes required.

Claim 12

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation method in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories respectively, performing the motion estimation and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including first circuit means for reading said original image and said reference image from said first and second frame memories respectively and carrying out coarse motion estimation of grid points on said reference image; a grid point coordinate memory for storing the coordinate information of grid points on said reference image from said first circuit means; an interpolation circuit for reading a part of said reference image from said second frame memory, determining by interpolation the luminance value of at least a point having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively lacking a pixel on said part of said reference image, and synthesizing a part of a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a matching circuit for reading the coordinate information of a grid point to be corrected and at least a nearby grid point from said grid point coordinate memory, performing the correction process by hexagonal matching using the part of the original image read from said first frame memory and the part of the high-resolution image synthesized at said interpolation circuit, and rewriting the coordinate information of the grid point on said grid point coordinate memory; and a motion vector computation circuit for reading the coordinate information of a grid point corrected by said matching circuit from said grid point coordinate memory, computing a motion vector of said grid point, and outputting motion information; wherein said matching circuit reads the luminance value of at least a pixel located on a part of said high-resolution image nearest to a corresponding point lacking a pixel on a part of said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point becomes required.

Claim 13

A video coder according to claim 9, wherein the values of said m1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 14

A video coder according to claim 10, wherein the values of said m1 and m2 are both 2.

Claim 15

A video coder according to claim 10, wherein the values of said m1 and m2 are both 4.

Claim 16

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integral multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories respectively, performing the motion estimation and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said synthesis circuit including means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels.

Claim 17

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integral multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories respectively, performing the motion estimation and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said synthesis circuit including means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels.

Claim 18

A video coder according to claim 13, wherein the values of said d1 and d2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 19

A video coder according to claim 14, wherein the values of said d1 and d2 are both 2.

Claim 20

A video coder according to claim 14, wherein the values of said d1 and d2 are both 4.

Claim 21

A video coder according to claim 14, wherein the values of said d1 and d2 are both 8.

Claim 22

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector, and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories respectively, performing a motion estimation process and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit, and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including an interpolation circuit for determining by interpolation the luminance value of a point lacking a pixel on said reference image and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively on said reference image and synthesizing a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a third memory for storing the high-resolution reference image synthesized at said interpolation circuit; and a matching circuit for searching for a selected one of a motion vector and a motion parameter minimizing the prediction error between the original image and said reference image and outputting motion information; said synthesis circuit including means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 (d1 and d2 being an integer not less than 2) respectively of the distance between adjacent pixels; wherein said matching circuit reads from said third memory the luminance value of at least a pixel located on said high-resolution image nearest to a corresponding point lacking a pixel on said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point lacking a pixel on said reference image becomes required.

Claim 23

A video coder for coding image sequences with a compressed amount of information by carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector, and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a first frame memory for storing an original image of a current frame; a second frame memory for storing a reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said reference image from said first and second frame memories respectively, performing a motion estimation process and outputting motion information; a synthesis circuit for synthesizing a predicted image using the reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the predicted image from said synthesis circuit, and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said motion estimation section including an interpolation circuit for determining by interpolation the luminance value of a point lacking a pixel on said reference image and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively on said reference image and synthesizing a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a third memory for storing the high-resolution reference image synthesized at said interpolation circuit; and a matching circuit for searching for a selected one of a motion vector and a motion parameter minimizing the prediction error between the original image and said reference image and outputting motion information; said synthesis circuit including means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 (d1 and d2 being an integer not less than 2) respectively of the distance between adjacent pixels; wherein said matching circuit reads from said third memory the luminance value of at least a pixel located on said high-resolution image nearest to a corresponding point lacking a pixel on said reference image and uses said luminance value for evaluating the prediction error in the case where the luminance value of said corresponding point lacking a pixel on said reference image becomes required.

Claim 24

A video coder according to claim 18, wherein the relation between the values of 1/m1 and 1/d1 is given as 1/m1>1/d1, and the relation between the values of 1/m2 and 1/d2 is given as 1/m2>1/d2.

Claim 25

A video coder according to claim 19, wherein the values of said m1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively, and the values of said d1 and d2 are 2 W1' and 2 W2' (W1' and W2' being positive integers) respectively.

Claim 26

A video decoder for receiving prediction error information and motion information transmitted from a video coder which codes image sequences with a compressed amount of information, said video decoder decoding the original image sequences by carrying out a motion compensation scheme in which all a pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a frame memory for storing a reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a synthesis circuit for synthesizing a predicted image using said reference image read from said frame memory and said motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the predicted image from said synthesis circuit; said synthesis circuit including means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels.

Claim 27

A video decoder for receiving prediction error information and motion information transmitted from a video coder which codes image sequences with a compressed amount of information, said video decoder decoding the original image sequences by carrying out a motion compensation scheme in which all a pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a frame memory for storing a reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a synthesis circuit for synthesizing a predicted image using said reference image read from said frame memory and said motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the predicted image from said synthesis circuit; said synthesis circuit including means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels.

Claim 28

A video decoder according to claim 21, wherein the values of said d1 and d2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 29

A video decoder according to claim 22, wherein the values of said d1 and d2 are both 2.

Claim 30

A video decoder according to claim 22, wherein the values of said d1 and d2 are both 4.

Claim 31

A video decoder according to claim 22, wherein the values of said d1 and d2 are both 8.

Claim 32

A video coding/decoding system for carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted thereto and decoding the original image sequences; said video coder further comprising a first frame memory for storing an original image of a current frame; a second frame memory for storing a first reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said first reference image from said first and second frame memories respectively, performing a motion estimation process and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising a third frame memory for storing a second reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and said motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said first synthesis circuit including first means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 of the distance between adjacent pixels.

Claim 33

A video coding/decoding system for carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted thereto and decoding the original image sequences; said video coder further comprising a first frame memory for storing an original image of a current frame; a second frame memory for storing a first reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said first reference image from said first and second frame memories respectively, performing a motion estimation process and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising a third frame memory for storing a second reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and said motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said first synthesis circuit including first means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 of the distance between adjacent pixels.

Claim 34

A video coding/decoding system according to claim 26, wherein the values of said d1 and d2 are 2 W1 and 2 W2 respectively, W1 and W2 being positive integers.

Claim 35

A video coding/decoding system according to claim 27, wherein the values of said d1 and d2 are both 2.

Claim 36

A video coding/decoding system according to claim 27, wherein the values of said d1 and d2 are both 4.

Claim 37

A video coding/decoding system according to claim 27, wherein the values of said d1 and d2 are both 8.

Claim 38

A video coding/decoding system for carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted thereto and decoding the original image sequences; said video coder further comprising a first frame memory for storing an original image of a current frame; a second frame memory for storing a first reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said first reference image from said first and second frame memories respectively, performing a motion estimation process and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising: a third frame memory for storing a second reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and the motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said first synthesis circuit including first means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1' and 1/d2', d1' and d2' being an integer not less than 2, respectively of the distance between adjacent pixels; wherein said video coding/decoding system is such that at least said video decoder transmits to said video coder information relating to values of d1' and d2' that can be employed by said second synthesis circuit before transmission of said prediction error information and said motion information, and said video coder selects the values of d1 and d2 that can be employed by said first synthesis circuit from among said values of d1' and d2' respectively, and transmits the result of selection to said video decoder.

Claim 39

A video coding/decoding system for carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted thereto and decoding the original image sequences; said video coder further comprising a first frame memory for storing an original image of a current frame; a second frame memory for storing a first reference image providing a decoded image of a frame already coded; a motion estimation section for reading said original image and said first reference image from said first and second frame memories respectively, performing a motion estimation process and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising: a third frame memory for storing a second reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and the motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said first synthesis circuit including first means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1' and 1/d2', d1' and d2' being an integer not less than 2, respectively of the distance between adjacent pixels; wherein said video coding/decoding system is such that at least said video decoder transmits to said video coder information relating to values of d1' and d2' that can be employed by said second synthesis circuit before transmission of said prediction error information and said motion information, and said video coder selects the values of d1 and d2 that can be employed by said first synthesis circuit from among said values of d1' and d2' respectively, and transmits the result of selection to said video decoder.

Claim 40

A video coding/decoding system according to claim 31, wherein the values of said d1 and d2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, and the values of said d1' and d2' are 2 W1' and 2 W2' , W1' and W2' being positive integers.

Claim 41

A video coding/decoding system for carrying out a motion compensation scheme in which all the pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of the motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted and decoding the original image sequences; said video coder further comprising: a first frame memory for storing an original image of the current frame; a second frame memory for storing a first reference image providing a decoded image of the frame already coded; a motion estimation section for reading the original image and the first reference image from said first and second frame memories respectively, performing the motion estimation process and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising: a third frame memory for storing a second reference image providing a decoded image of the frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and the motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said first synthesis circuit including first means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 (d1 and d2 being an integer not less than 2) respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to 1/d1' and 1/d2', d1' and d2' being an integer not less than 2, respectively of the distance between adjacent pixels; wherein said video coding/decoding system is such that at least said video decoder transmits to said video coder the upper limit values of d1' and d2' that can be employed by said second synthesis circuit before starting to transmit the prediction error information and the motion information; and said video coder selects the value of said d1 and d2 from the upper limit value that can be employed as d1' and d2' and the upper limit value that can be employed by said first synthesis circuit, whichever is lower, and transmits the result of said selection to said video decoder.

Claim 42

A video coding/decoding system for carrying out a motion compensation scheme in which all the pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of the motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted and decoding the original image sequences; said video coder further comprising: a first frame memory for storing an original image of the current frame; a second frame memory for storing a first reference image providing a decoded image of the frame already coded; a motion estimation section for reading the original image and the first reference image from said first and second frame memories respectively, performing the motion estimation process and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising: a third frame memory for storing a second reference image providing a decoded image of the frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and the motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said first synthesis circuit including first means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 (d1 and d2 being an integer not less than 2) respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to 1/d1' and 1/d2', d1' and d2' being an integer not less than 2, respectively of the distance between adjacent pixels; wherein said video coding/decoding system is such that at least said video decoder transmits to said video coder the upper limit values of d1' and d2' that can be employed by said second synthesis circuit before starting to transmit the prediction error information and the motion information; and said video coder selects the value of said d1 and d2 from the upper limit value that can be employed as d1' and d2' and the upper limit value that can be employed by said first synthesis circuit, whichever is lower, and transmits the result of said selection to said video decoder.

Claim 43

A video coding/decoding system according to claim 33, wherein the values of said d1 and d2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively, and the values of said d1' and d2' are 2 W1' and 2 W2' , W1' and W2' being positive integers, respectively.

Claim 44

A video coding/decoding system for carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted thereto and decoding the original image sequences; said video coder further comprising a first frame memory for storing an original image of a current frame; a second frame memory for storing a first reference image providing a decoded image of a frame already coded; a motion estimation section for reading the original image and the first reference image from said first and second frame memories respectively, performing a motion estimation and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit, and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising: a third frame memory for storing a second reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and said motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said motion estimation section including an interpolation circuit for determining by interpolation the luminance value of a point lacking a pixel and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively on said reference image, and synthesizing a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a fourth memory for storing the high-resolution reference image synthesized at said interpolation circuit; and a matching circuit for searching for a selected one of a motion vector and a motion parameter minimizing the prediction error between said original image and said reference image and outputting motion information; said first synthesis circuit including first means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 of the distance between adjacent pixels; wherein said matching circuit reads from said fourth memory the luminance value of a pixel located on said high-resolution reference image nearest to a corresponding point lacking a pixel on said reference image and uses the luminance value for evaluating a prediction error in the case where the luminance value of said corresponding point on said reference image becomes required.

Claim 45

A video coding/decoding system for carrying out a motion compensation scheme in which all pixels associated with a same patch are not restricted to have a common motion vector and the horizontal and vertical components of a motion vector of a pixel can assume an arbitrary value other than an integer multiple of a distance between adjacent pixels, comprising: a video coder for coding image sequences with a compressed amount of information and transmitting prediction error information and motion information; and a video decoder for receiving the prediction error information and the motion information transmitted thereto and decoding the original image sequences; said video coder further comprising a first frame memory for storing an original image of a current frame; a second frame memory for storing a first reference image providing a decoded image of a frame already coded; a motion estimation section for reading the original image and the first reference image from said first and second frame memories respectively, performing a motion estimation and outputting motion information; a first synthesis circuit for synthesizing a first predicted image using the first reference image read from said second frame memory and the motion information from said motion estimation section; a subtractor for calculating the difference between the original image read from said first frame memory and the first predicted image from said first synthesis circuit, and outputting a prediction error; and a prediction error coder for coding the prediction error from said subtractor and outputting prediction error information; said video decoder further comprising: a third frame memory for storing a second reference image providing a decoded image of a frame already decoded; a prediction error decoding circuit for decoding the prediction error information received and outputting a prediction error; a second synthesis circuit for synthesizing a second predicted image using the second reference image read from said third frame memory and said motion information received; and an adder for adding and outputting the sum of the prediction error from said prediction error decoding circuit and the second predicted image from said second synthesis circuit; said motion estimation section including an interpolation circuit for determining by interpolation the luminance value of a point lacking a pixel and having horizontal and vertical coordinates of an integer multiple of 1/m1 and 1/m2, m1 and m2 being an integer not less than 2, respectively on said reference image, and synthesizing a high-resolution reference image with a pixel density m1 and m2 times larger in horizontal and vertical directions respectively; a fourth memory for storing the high-resolution reference image synthesized at said interpolation circuit; and a matching circuit for searching for a selected one of a motion vector and a motion parameter minimizing the prediction error between said original image and said reference image and outputting motion information; said first synthesis circuit including first means for limiting the horizontal and vertical components of a motion vector of each pixel to an integer multiple of 1/d1 and 1/d2, d1 and d2 being an integer not less than 2, respectively of the distance between adjacent pixels; said second synthesis circuit including second means for limiting the horizontal and vertical components of the motion vector of each pixel to an integer multiple of 1/d1 and 1/d2 of the distance between adjacent pixels; wherein said matching circuit reads from said fourth memory the luminance value of a pixel located on said high-resolution reference image nearest to a corresponding point lacking a pixel on said reference image and uses the luminance value for evaluating a prediction error in the case where the luminance value of said corresponding point on said reference image becomes required.

Claim 46

A video coding/decoding system according to claim 35, wherein the relation between the values of said 1/m1 and said 1/d1 is given as 1/m1>1/d1, and the relation between the values of said 1/m2 and said 1/d2 is given as 1/m2>1/d2.

Claim 47

A video coding/decoding system according to claim 36, wherein the values of said m1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively, and the values of said d1 and d2 are 2 W1' and 2 W2' (W1' and W2' being positive integers, respectively.

Claim 50

A method according to claim 38, wherein the values of said m1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 53

A method according to claim 40, wherein the values of said M1 and m2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 56

A method according to claim 42, wherein the values of said d1 and d2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.

Claim 59

A method for decoding video according to claim 44, wherein the values of said d1 and d2 are 2 W1 and 2 W2 , W1 and W2 being positive integers, respectively.