Image pickup method and apparatus, and image processing method and apparatus for adapting to synthesize a plurality of images

Claim 2

An image pickup method of synthesizing a plurality of images acquired by sensing an object under different exposure conditions to produce a single image excellent in gradation reproducibility, the method comprising the steps of: sensing an object under different exposure conditions to acquire a plurality of images; calculating a pixel value for each of the plurality of images; multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate a positive value compensation amount; compensating the levels of the plurality of images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting the positive value compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image of a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 3

The method as set forth in claim 1 , further comprising the steps of: calculating a mean pixel value of each of the plurality of images; and multiplying the mean pixel value obtained at the mean calculating step by a factor set based on the exposure condition to calculate the positive value compensation amount.

Claim 4

The method as set forth in claim 2 , further comprising the steps of: time-smoothing the positive value obtained at the multiplying step; and subtracting the time-smoothed positive value from the pixel level of each of the plurality of images.

Claim 5

The method as set forth in claim 2 , wherein the factor is set larger for the image having been sensed with a larger exposure at the image sensing step.

Claim 6

The method as set forth in claim 1 , further comprising the steps of: filtering the signal of each of the plurality of images by a predetermined low-pass filter; and multiplying an output obtained at the filtering step by a factor set based on the exposure condition under which the image has been sensed to calculate the positive value compensation amount.

Claim 7

The method as set forth in claim 5 , further comprising the steps of: time-smoothing the positive value obtained at the multiplying step; and subtracting, at the subtracting step, the positive value obtained at the time-smoothing step from the pixel level of each of the plurality of images.

Claim 8

The method as set forth in claim 5 , wherein the factor is set larger for the image having been sensed with a larger exposure at the image sensing step.

Claim 9

The method as set forth in claim 1 , wherein each of the plurality of images acquired at the image sensing step is an image signal in which a frequency-modulated color signal is superposed on a brightness signal, the method further comprising the step of separating the image signal into the brightness and color signals; and the brightness signal and color signal separated at the signal separating step being compensated at the level compensating step to produce a compensated brightness signal and a compensated color signal, respectively; the compensated brightness and color signals being synthesized at the image synthesizing step to produce a synthetic brightness signal and a synthetic color signal, respectively; wherein a compensation amount is calculated based on the brightness signal separated at the signal separating step the level of brightness signal is compensated with the compensation amount to produce a compensated brightness signal; and the compensated color signal is produced based on the compensated brightness signal; and the synthetic brightness and color signals being compressed at the image compressing step to produce a compressed brightness signal and a compressed color signal, respectively.

Claim 10

The method as set forth in claim 8 , further comprising the step of mixing the compressed brightness and color signals.

Claim 11

An image pickup apparatus adapted to produce a single image excellent in gradation reproducibility from a plurality of images acquired by sensing an object under different exposure conditions, the apparatus comprising: means for sensing an object under different exposure conditions to acquire a plurality of images; means for calculating a pixel value for each of the plurality of images; means for multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate a positive value compensation amount; means for compensating the levels of the plurality of images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting the positive value compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image of a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; means for synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and means for compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 12

An image pickup apparatus adapted to produce a single image excellent in gradation reproducibility from a plurality of images acquired by sensing an object under different exposure conditions, the apparatus comprising: means for sensing an object under different exposure conditions to acquire a plurality of images; means for calculating a pixel value for each of the plurality of images; means for multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate a positive value compensation amount; means for compensating the levels of the plurality of images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting the positive value compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image of a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; means for synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and means for compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 13

The apparatus as set forth in claim 10 , wherein the level compensating means comprises: means for calculating a mean pixel value of each of the plurality of images; and means for multiplying the mean pixel value provided by the mean calculating means by a factor set based on the exposure condition to calculate the positive value compensation amount.

Claim 14

The apparatus as set forth in claim 11 , wherein the level compensating means comprises: means for time-smoothing the positive value provided by multiplying means; and means for subtracting the time-smoothed positive value from the pixel level of each of the plurality of images.

Claim 15

The method as set forth in claim 11 , wherein the factor is set larger for the image having been sensed with a larger exposure by the image sensing means.

Claim 16

The apparatus as set forth in claim 10 , wherein the level compensating means comprises: means for allowing to pass the signal of each of the plurality of images through a predetermined low-pass filter; and means for multiplying an output of the signal filtering means by a factor set based on the exposure condition under which the image has been sensed to calculate the positive value compensation amount.

Claim 17

The apparatus as set forth in claim 14 , wherein the level compensating means comprises means for time-smoothing time positive value provided by the multiplying means; the subtracting means subtracting the time-smoothed positive value from the pixel level of each of the plurality of images.

Claim 18

The apparatus as set forth in claim 14 , wherein the factor is set larger for the image having been sensed with a larger exposure by the image sensing means.

Claim 19

The apparatus as set forth in claim 10 , wherein the image sensing means is adapted to output an image signal in which a frequency-modulated color signal is superposed on a brightness signal, the apparatus further comprising means for separating the image signal into the brightness and color signals; and the level compensating means compensating the brightness signal and color signal separated by the signal separating means to produce a compensated brightness signal and a compensated color signal, respectively; wherein: a compensation amount is calculated based on the brightness signal separated at the signal separating step: the level of brightness signal is compensated with the compensation amount to produce a compensated brightness signal; and the compensated color signal is produced based on the compensated brightness signal; the signal synthesizing means synthesizing the compensated brightness and color signals to produce a synthetic brightness signal and a synthetic color signal, respectively; and the image compressing means the synthetic brightness and color signals to produce a compressed brightness signal and a compressed color signal, respectively.

Claim 20

The method as set forth in claim 17 , further comprising means for mixing the compressed brightness and color signals.

Claim 21

An image processing method of synthesizing a plurality of input images acquired by sensing an object under different exposure conditions to produce a single image excellent in gradation reproducibility, the method comprising the steps of: receiving a plurality of images acquired by sensing an object under different exposure conditions and compensating the levels of the plurality of input images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting a positive value compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image by a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; calculating a pixel value for each of the plurality of images; multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate the positive value compensation amount; synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 22

An image processing method of synthesizing a plurality of input images acquired by sensing an object under different exposure conditions to produce a single image excellent in gradation reproducibility, the method comprising the steps of: receiving a plurality of images acquired by sensing an object under different exposure conditions and compensating the levels of the plurality of input images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting a positive value compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image by a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; calculating a pixel value for each of the plurality of images; multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate the positive value compensation amount; synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 23

The method as set forth in claim 19 , further comprising the steps of: calculating a mean pixel value of each of the plurality of input images; and multiplying the mean pixel value obtained at the mean calculating step by a factor set based on the exposure condition to calculate the positive value compensation amount.

Claim 24

The method as set forth in claim 20 , further comprising the steps of: time-smoothing the positive value obtained at the multiplying step; and subtracting the time-smoothed positive value from the pixel level of each of the plurality of input images.

Claim 25

The method as set forth in claim 20 , wherein the factor is set larger for the input image having been sensed with a larger exposure at the image sensing step.

Claim 26

The method as set forth in claim 19 , further comprising the steps of: filtering the signal of each of the plurality of input images by a predetermined low-pass filter; and multiplying an output obtained at the filtering step by a factor set based on the exposure condition under which the image has been sensed to calculate the positive value compensation amount.

Claim 27

The method as set forth in claim 23 , further comprising the steps of: time-smoothing the positive value obtained at the multiplying step; and subtracting, at the subtracting step, the positive value obtained at the time-smoothing step from the pixel level of each of the plurality of input images.

Claim 28

The method as set forth in claim 23 , wherein the factor is set larger for the input image having been sensed with a larger exposure at the image sensing step.

Claim 29

The method as set forth in claim 19 , wherein each of the plurality of input images acquired at the image sensing step is an image signal in which a frequency-modulated color signal is superposed on a brightness signal, the method further comprising the step of separating the image signal into the brightness and color signals; and the brightness signal and color signal separated at the signal separating step being compensated at the level compensating step to produce a compensated brightness signal and a compensated color signal, respectively; the compensated brightness and color signals being synthesized at the image synthesizing step to produce a synthetic brightness signal and a synthetic color signal, respectively; wherein a compensation amount is calculated based on the brightness signal separated at the signal separating step the level of brightness signal is compensated with the compensation amount to produce a compensated brightness signal; and the compensated color signal is produced based on the compensated brightness signal; and the synthetic brightness and color signals being compressed at the image compressing step to produce a compressed brightness signal and a compressed color signal, respectively.

Claim 30

The method as set forth in claim 26 , further comprising the step of mixing the compressed brightness and color signals.

Claim 31

An image processing apparatus adapted to produce a single image excellent in gradation reproducibility from a plurality of input images acquired by sensing an object under different exposure conditions, the apparatus comprising: means for receiving a plurality of input images acquired by sensing an object under different exposure conditions and compensating the levels of the plurality of input images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting a positive valued compensation amount for each image; said positive valued compensation amount being calculated by multiplying a level average of each image by a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; means for calculating a pixel value for each of the plurality of images; means for multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate the positive value compensation amount; means for synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and means for compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 32

An image processing apparatus adapted to produce a single image excellent in gradation reproducibility from a plurality of input images acquired by sensing an object under different exposure conditions, the apparatus comprising: means for receiving a plurality of input images acquired by sensing an object under different exposure conditions and compensating the levels of the plurality of input images on the basis of the exposure conditions under which they have been sensed respectively, to provide a plurality of compensated images by subtracting a positive valued compensation amount for each image; said positive valued compensation amount being calculated by multiplying a level average of each image by a coefficient based on the corresponding exposure condition; said coefficient being selected for each image based on the exposure condition of that image; means for calculating a pixel value for each of the plurality of images; means for multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate the positive value compensation amount; means for synthesizing the plurality of compensated images to produce a single synthetic image having a wide dynamic range; and means for compressing the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image.

Claim 33

The apparatus as set forth in claim 28 , wherein the level compensating means comprises: means for calculating a mean pixel value of each of the plurality of input images; and means for multiplying the mean pixel value provided by the mean calculating means by a factor set based on the exposure condition to calculate the positive value compensation amount.

Claim 34

The apparatus as set forth in claim 29 , wherein the level compensating means comprises: means for time-smoothing the positive value provided by the multiplying means; and means for subtracting the time-smoothed positive value from the pixel level of each of the plurality of input images.

Claim 35

The apparatus as set forth in claim 29 , wherein the factor is set larger for the input image having been sensed with a larger exposure by the image sensing means.

Claim 36

The apparatus as set forth in claim 28 , wherein the level compensating means comprises: means for filtering the signal of each of the plurality of input images by a predetermined low-pass filter; and means for multiplying an output of the signal filtering means by a factor set based on the exposure condition under which the image has been sensed to calculate the positive value compensation amount.

Claim 37

The apparatus as set forth in claim 32 , wherein the level compensating means comprises means for time-smoothing the positive value provided by the multiplying means; the subtracting means subtracting the time-smoothed positive value from the pixel level of each of the plurality of input images.

Claim 38

The apparatus as set forth in claim 32 , wherein the factor is set larger for the input image having been sensed with a larger exposure by the image sensing means.

Claim 39

The apparatus as set forth in claim 28 , wherein the image sensing means is adapted to output an image signal in which a frequency-modulated color signal is superposed on a brightness signal, the apparatus further comprising means for separating the image signal into the brightness and color signals; and the level compensating means compensating the brightness signal and color signal separated by the signal separating means to produce a compensated brightness signal and a compensated color signal, respectively; wherein: a compensation amount is calculated based on the brightness signal separated at the signal separating step the level of brightness signal is compensated with the compensation amount to produce a compensated brightness signal; and the compensated color signal is produced based on the compensated brightness signal; and the signal synthesizing means synthesizing the compensated brightness and color signals to produce a synthetic brightness signal and a synthetic color signal, respectively; and the image compressing means the synthetic brightness and color signals to produce a compressed brightness signal and a compressed color signal, respectively.

Claim 40

The apparatus as set forth in claim 35 , further comprising means for mixing the compressed brightness and color signals.

Claim 41

An image pickup apparatus adapted to synthesize a plurality of images acquired by sensing an object under different exposure conditions to produce a single image excellent in gradation and reproducibility, the apparatus comprising: means for sensing an object under different exposure conditions to acquire a plurality of images; means for calculating a pixel value for each of the plurality of images; means for multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate a positive value compensation amount; means for synthesizing plural images to produce a single synthetic image having a wide dynamic range; means for compressing either the synthetic image or the dynamic range of the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image; and means for compensating the level of images supposed thereto to provide compensated images, whereby said compensating means is supplied with either (a) the images acquired by said sensing means to compensate the levels of said images on the basis of the exposure conditions under which said images have been sensed, respectively, by subtracting the positive valued compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image by a coefficient based on the corresponding exposure condition, and said synthesizing means is supplied with the provided compensated images, or (b) the compressed image and said synthesizing means is supplied with the acquired uncompensated images; said coefficient being selected for each image based on the exposure condition of that image.

Claim 42

An image pickup apparatus adapted to synthesize a plurality of images acquired by sensing an object under different exposure conditions to produce a single image excellent in gradation and reproducibility, the apparatus comprising: means for sensing an object under different exposure conditions to acquire a plurality of images; means for calculating a pixel value for each of the plurality of images; means for multiplying the pixel value obtained at the calculating step by a factor set based on the exposure condition to calculate a positive value compensation amount; means for synthesizing plural images to produce a single synthetic image having a wide dynamic range; means for compressing either the synthetic image or the dynamic range of the synthetic image to an extent depending upon the performance of its output destination to produce a compressed image; and means for compensating the level of images supposed thereto to provide compensated images, whereby said compensating means is supplied with either (a) the images acquired by said sensing means to compensate the levels of said images on the basis of the exposure conditions under which said images have been sensed, respectively, by subtracting the positive valued compensation amount from each image; said positive valued compensation amount being calculated by multiplying a level average of each image by a coefficient based on the corresponding exposure condition, and said synthesizing means is supplied with the provided compensated images, or (b) the compressed image and said synthesizing means is supplied with the acquired uncompensated images; said coefficient being selected for each image based on the exposure condition of that image.