Image processing method and image processing apparatus for obtaining overlaid image

Claim 2

An image processing method used for an image processing apparatus to overlay an image of an object to another image, said method comprising: a step of receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; a transparency calculation step of obtaining, from said n images, a transparency image having a transparency value for each pixel; a color calculation step of obtaining a color image of said object from said n images; and an overlaying step of overlaying said object on said another image by using said transparency image and said color image, wherein a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.

Claim 3

The image processing method as claimed in claim 1 , wherein transparency AP (x, y) of each pixel of said transparency image is obtained by an equation 1−|(P 1 (x,y)−P 2 (x,y))/PPMAX|, wherein PPMAX is PPB 1 −PPB 2 , P 1 (x, y) is a pixel value of a first image in said n images, P 2 (x, y) is a pixel value of a second image in said n images, PPB 1 is a pixel value of a pixel in a part other than the object in said first image, PPB 2 is a pixel value of a pixel in said second image corresponding to said pixel in said first image.

Claim 4

The image processing method as claimed in claim 2 , wherein PPMAX is the greatest value in absolute values of P 1 (x, y)−P 2 (x, y) for every pixel.

Claim 5

The image processing method as claimed in claim 2 , wherein PPMAX is calculated for each line in said first image and said second image.

Claim 6

The image processing method as claimed in claim 4 , wherein PPMAX is calculated as PE 1 −PE 2 , wherein PE 1 is a pixel value of a pixel of an end of said line in said first image and PE 2 is a pixel value of a pixel in said second image corresponding to said pixel of said end of said line.

Claim 7

The image processing method as claimed in claim 2 , wherein said first image and said second image are a combination selected from said n images, and said transparency AP(x, y) is calculated as a mean value of transparencies for every combination in said n images.

Claim 8

The image processing method as claimed in claim 1 , wherein transparency AP (x, y) of each pixel of said transparency image is obtained by an equation 1−|(P 1 (x,y)−P 2 (x,y))/(PB 1 (x,y)−PB 2 (x,y))|, wherein P 1 (x, y) is a pixel value of a first image in said n images, P 2 (x, y) is a pixel value of a second image in said n images, PB 1 (x, y) is a pixel value of a pixel (x, y) in a first background image that does not include said object and the background of said first background image is the same as the background of said first image, PB 2 (x, y) is a pixel value of a pixel (x, y) in a second background image that does not include said object and the background of said second background image is the same as the background of said second image.

Claim 9

The image processing method as claimed in claim 1 , wherein said color image is one of said n images.

Claim 10

The image processing method as claimed in claim 1 , wherein each pixel value of said color image is a mean value of pixel values of pixels in the same position in said n images.

Claim 11

The image processing method as claimed in claim 1 , wherein influence of color of the background is removed from said color image by using said transparency image.

Claim 12

The image processing method as claimed in claim 1 , wherein a first rectangular image enclosing said object is cut out from said transparency image, and a second rectangular image enclosing said object is cut out from said color image.

Claim 13

The image processing method as claimed in claim 11 , wherein said first rectangular image includes pixels each of which has a transparency greater than a threshold, and the size and the position of said second rectangular image are the same as the size and the position of said first rectangular image.

Claim 14

The image processing method as claimed in claim 1 , said image processing method further comprising a noise removing step of applying a noise removing filter to said transparency image.

Claim 15

The image processing method as claimed in claim 1 , wherein said overlaid image is obtained after said another image is filtered by a filter by which blurring amount is changed according to a pixel value of said transparency image.

Claim 16

The image processing method as claimed in claim 14 , wherein said filter is a moving average filter or a median filter in which the size of a matrix is changed according to a pixel value of said transparency image.

Claim 17

The image processing method as claimed in claim 1 , wherein said object is overlaid on said another image by using an object color image, instead of said color image, that is taken separately from said n images.

Claim 18

The image processing method as claimed in claim 16 , wherein said object color image is an image selected from object color images each of which is taken under a different lighting condition, and the lighting condition for said object color image that is selected is the closest to the lighting condition under which said another image is taken.

Claim 19

An image processing apparatus for overlaying an image of an object on another image, said image processing apparatus comprising: a part for receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; a part for obtaining, from said n images, a transparency image having a transparency value for each pixel; a part for obtaining a color image of said object from said n images; and a part for overlaying said object on said another image by using said transparency image and said color image, wherein a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.

Claim 20

An image processing apparatus for overlaying an image of an object on another image, said image processing apparatus comprising: a part for receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; a part for obtaining, from said n images, a transparency image having a transparency value for each pixel; a part for obtaining a color image of said object from said n images; and a part for overlaying said object on said another image by using said transparency image and said color image, wherein a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.

Claim 21

The image processing apparatus as claimed in claim 18 , wherein said n images are taken by using an image capturing system, said image capturing system comprising: a background display apparatus for displaying a background that changes either or both of color and brightness by using an electronic signal; a table on which the object is placed, wherein either or both of color and brightness of said table changes in the same way as said background display apparatus; an image pick-up apparatus for taking an image of said background display apparatus, said object and said table.

Claim 22

The image processing apparatus as claimed in claim 18 , said image processing apparatus further comprising a noise removing filter for removing noise from said transparency image.

Claim 23

A computer readable medium storing program code for causing a computer to overlay an image of an object on another image, said computer readable medium comprising: program code means for receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; transparency calculation program code means for obtaining, from said n images, a transparency image having a transparency value for each pixel; color calculation program code means for obtaining a color image of said object from said n images; and overlaying program code means for overlaying said object on said another image by using said transparency image and said color image, wherein, in said overlaying program code means, a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.

Claim 24

A computer readable medium storing program code for causing a computer to overlay an image of an object on another image, said computer readable medium comprising: program code means for receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; transparency calculation program code means for obtaining, from said n images, a transparency image having a transparency value for each pixel; color calculation program code means for obtaining a color image of said object from said n images; and overlaying program code means for overlaying said object on said another image by using said transparency image and said color image, wherein, in said overlaying program code means, a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.

Claim 25

The computer readable medium as claimed in claim 21 , wherein, in said transparency calculation program code means, transparency AP (x, y) of each pixel of said transparency image is obtained by an equation 1−|(P 1 (x,y)−P 2 (x,y))/PPMAX|, wherein PPMAX is PPB 1 −PPB 2 , P 1 (x, y) is a pixel value of a first image in said n images, P 2 (x, y) is a pixel value of a second image in said n images, PPB 1 is a pixel value of a pixel in a part other than the object in said first image, and PPB 2 is a pixel value of a pixel in said second image corresponding to said pixel in said first image.

Claim 26

The computer readable medium as claimed in claim 22 , wherein PPMAX is the greatest value in absolute values of P 1 (x, y)−P 2 (x, y) for every pixel.

Claim 27

The computer readable medium as claimed in claim 22 , wherein PPMAX is calculated for each line in said first image and said second image.

Claim 28

The computer readable medium as claimed in claim 24 , wherein PPMAX is calculated as PE 1 −PE 2 , wherein PE 1 is a pixel value of a pixel of an end of said line in said first image and PE 2 is a pixel value of a pixel in said second image corresponding to said pixel of said end of said line.

Claim 29

The computer readable medium as claimed in claim 22 , wherein said first image and said second image are a combination selected from said n images, and said transparency AP(x, y) is calculated as a mean value of transparencies for every combination in said n images.

Claim 30

The computer readable medium as claimed in claim 21 , wherein, in said transparency calculation program code means, transparency AP (x, y) of each pixel of said transparency image is obtained by an equation 1−|(P 1 (x,y)−P 2 (x,y))/(PB 1 (x,y)−PB 2 (x,y))|, wherein P 1 (x, y) is a pixel value of a first image in said n images, P 2 (x, y) is a pixel value of a second image in said n images, PB 1 (x, y) is a pixel value of a pixel (x, y) in a first background image that does not include said object and the background of said first background image is the same as the background of said first image, PB 2 (x, y) is a pixel value of a pixel (x, y) in a second background image that does not include said object and the background of said second background image is the same as the background of said second image.

Claim 31

The computer readable medium as claimed in claim 21 , wherein, in said color calculation program code means, said color image is one of said n images.

Claim 32

The computer readable medium as claimed in claim 21 , wherein, in said color calculation program code means, each pixel value of said color image is a mean value of pixel values of pixels in the same position in said n images.

Claim 33

The computer readable medium as claimed in claim 21 , wherein, in said color calculation program code means, influence of color of the background is removed from said color image by using said transparency image.

Claim 34

The computer readable medium as claimed in claim 21 , said computer readable medium further comprising: program code means for cutting out a first rectangular image enclosing said object from said transparency image, and cutting out a second rectangular image enclosing said object from said color image.

Claim 35

The computer readable medium as claimed in claim 31 , wherein said first rectangular image includes pixels each of which has a transparency greater than a threshold, and the size and the position of said second rectangular image are the same as the size and the position of said first rectangular image.

Claim 36

The computer readable medium as claimed in claim 21 , said computer readable medium further comprising noise removing program code means for applying a noise removing filter to said transparency image.

Claim 37

The computer readable medium as claimed in claim 21 , wherein, in said overlaying program code means, said overlaid image is obtained after said another image is filtered by a filter by which blurring amount is changed according to a pixel value of said transparency image.

Claim 38

The computer readable medium as claimed in claim 34 , wherein said filter is a moving average filter or a median filter in which a size of the matrix is changed according to a pixel value of said transparency image.

Claim 39

The computer readable medium as claimed in claim 21 , wherein, in said overlaying program code means, said object is overlaid on said another image by using an object color image, instead of said color image, that is taken separately from said n images.

Claim 40

The computer readable medium as claimed in claim 36 , wherein, in said overlaying program code means, said object color image is an image selected from object color images each of which is taken under a different lighting condition, and the lighting condition for said object color image that is selected is the closest to the lighting condition under which said another image is taken.

Claim 41

A computer program for causing a computer to overlay an image of an object on another image, said computer program comprising: program code means for receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; transparency calculation program code means for obtaining, from said n images, a transparency image having a transparency value for each pixel; color calculation program code means for obtaining a color image of said object from said n images; and overlaying program code means for overlaying said object on said another image by using said transparency image and said color image, wherein, in said overlaying program code means, a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.

Claim 42

A computer program for causing a computer to overlay an image of an object on another image, said computer program comprising: program code means for receiving n images each of which includes said object, wherein n is an integer no less than two, and either or both of color and brightness are different for each of the n images; transparency calculation program code means for obtaining, from said n images, a transparency image having a transparency value for each pixel; color calculation program code means for obtaining a color image of said object from said n images; and overlaying program code means for overlaying said object on said another image by using said transparency image and said color image, wherein, in said overlaying program code means, a pixel value of an overlaid image is obtained by P ⁢ ⁢ S ⁢ ( x , y ) = P ⁢ ⁢ C ⁢ ( x , y ) × P ⁢ ⁢ T ⁢ ( x , y ) + ( 1 - P ⁢ ⁢ T ⁢ ( x , y ) ) × 1 ( k + 1 ) 2 ⁢ ∑ m = - k 2 m = k 2 ⁢ ∑ n = - k 2 n = k 2 ⁢ P ⁢ ⁢ Z ⁢ ( x + m , y + n ) and k=K×PT(x, y), wherein PT(x, y) is a pixel value of said transparency image, PC(x, y) is a pixel value of said color image, and PZ(x, y) is a pixel value of said another image.