Method and apparatus for expanding the dynamic range of sensed color images

Claim 2

An image processing method of forming an image based on color signals output from an image sensing device, comprising: the image picking-up step of picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; the threshold value setting step of setting a different threshold value for each color; and the combining step of generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the standard exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of the threshold value set for the color.

Claim 3

The method according to claim 1, wherein the combining step further comprises the steps of: determining whether or not a part of a color signal of the first plurality of color signals is saturated using a threshold value for the color signal set in the threshold value setting step; and replacing the part of the color signal which is determined as saturated with the corresponding part of the corresponding color signal of the second plurality of color signals.

Claim 4

The method according to claim 2, wherein an exposure amount of the non-standard exposure is smaller than an exposure amount of the standard exposure.

Claim 5

The method according to claim 1, wherein the combining step further comprises the steps of: determining whether or not a part of a color signal of the first plurality of color signals which have been picked up in the standard exposure exhibits a dark noise with respect to a threshold value for the color signal set in the threshold value setting step; and replacing the part of the color signal which is determined as of low-sensitivity with the corresponding part of the corresponding color signal of the second plurality of color signals.

Claim 6

The method according to claim 4, wherein an exposure amount of the non-standard exposure is larger than an exposure amount of the standard exposure.

Claim 7

The method according to claim 1, wherein the threshold value setting step includes the step of determining a threshold value corresponding to each color on the basis of the first and second plurality of color signals.

Claim 8

The method according to claim 1, wherein the threshold value setting step includes the step of calculating the threshold value corresponding to each color by using an average value of a difference between the first and second plurality of color signals.

Claim 9

The method according to claim 1, wherein the threshold value setting step includes the step of determining the threshold value corresponding to each color depending on whether a degree of change in the color signal out of the second plurality of color signals with respect to the color signal out of the first plurality of color signals obtained in the picking-up step is large.

Claim 10

An image processing method of forming an image based on color signals output from an image sensing device, comprising: the image picking-up step of picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; the threshold value setting step of setting a threshold value for each color on the basis of the first plurality of color signals obtained in the standard exposure and the second plurality of color signals obtained in the non-standard exposure, in the image picking-up step; and the combining step of generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the standard exposure with the corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a lowest threshold value out of threshold values for the color set in the threshold value setting step.

Claim 11

An image processing method of forming an image based on color signals output from an image sensing device, comprising: the image picking-up step of picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; the threshold value setting step of setting a threshold value for each color on the basis of the first plurality of color signals obtained in the standard exposure and the second plurality of color signals obtained in the non-standard exposure, in the image picking-up step; and the combining step of generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the standard exposure with the corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a lowest threshold value out of threshold values for the color set in the threshold value setting step.

Claim 12

The method according to claim 9, wherein the combining step further comprises the steps of: determining whether or not a part of a color signal of the first plurality of color signals is saturated using a threshold value for the color signal set in the threshold value setting step; and replacing the part of the color signal which is determined as saturated with the corresponding part of the corresponding color signal of the second plurality of color signals.

Claim 13

The method according to claim 10, further comprising the step of performing smoothing processing of the replaced image.

Claim 14

The method according to claim 9, wherein the threshold value setting step includes the step of calculating a threshold value corresponding to each color by using an average value of a difference between the first and second plurality of color signals.

Claim 15

The method according to claim 9, wherein the threshold value setting step includes the step of determining a threshold value corresponding to each color depending on whether a degree of change in the color signal out of the second plurality of color signals with respect to the color signal out of the first plurality of color signals is large.

Claim 16

The method according to claim 9, wherein an exposure amount of the non-standard exposure is smaller than an exposure amount of the standard exposure.

Claim 17

An image processing method of forming an image based on color signals output from an image sensing device, comprising: the image picking-up step of picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; the threshold value setting step of setting a threshold value for each color on the basis of the first plurality of color signals obtained in the standard exposure and the second plurality of color signals obtained in the non-standard exposure, in the image picking-up step; and the combining step of generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the standard exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a highest threshold value out of threshold values for the color set in the threshold value setting step.

Claim 18

An image processing method of forming an image based on color signals output from an image sensing device, comprising: the image picking-up step of picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; the threshold value setting step of setting a threshold value for each color on the basis of the first plurality of color signals obtained in the standard exposure and the second plurality of color signals obtained in the non-standard exposure, in the image picking-up step; and the combining step of generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the standard exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a highest threshold value out of threshold values for the color set in the threshold value setting step.

Claim 19

The method according to claim 15, wherein the combining step further comprises the steps of: determining whether or not a part of a color signal of the first plurality of color signals exhibits a dark noise with respect to a threshold value for the color signal set in the threshold value setting step; and replacing the part of the color signal which is determined as of low-sensitivity with the corresponding part of the corresponding color signal of the second plurality of color signals.

Claim 20

The method according to claim 10, further comprising the step of performing smoothing processing of the replaced image.

Claim 21

The method according to claim 15, wherein the threshold value setting step includes the step of calculating a threshold value corresponding to each color by using an average value of a difference between the first and second plurality of color signals.

Claim 22

The method according to claim 15, wherein the threshold value setting step includes the step of determining a threshold value corresponding to each color depending on whether a degree of change in the color signal out of the second plurality of color signals with respect to the color signal out of the first plurality of color signals is large.

Claim 23

The method according to claim 15, wherein an exposure amount of the non-standard exposure is larger than an exposure amount of the standard exposure.

Claim 24

An image processing apparatus which forms an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; threshold value setting means for setting a threshold value for each color signal; and combining means for generating a combined color signal by replacing, for each color, with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of the threshold value set for the color.

Claim 25

An image processing apparatus which forms an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a first plurality of color signals in a standard exposure and a second plurality of color signals in a non-standard exposure; threshold value setting means for setting a threshold value for each color signal; and combining means for generating a combined color signal by replacing, for each color, with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of the threshold value set for the color.

Claim 26

An image processing apparatus for forming an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a first plurality of color signals in a first exposure and a second plurality of color signals in a second exposure, an amount of the second exposure being larger than an amount of the first exposure; threshold value setting means for setting a threshold value for each color on the basis of the first plurality of color signals obtained in the first exposure and the second plurality of color signals obtained in the second exposure, by the image picking-up means; and combining means for generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the first exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a lowest threshold value out of threshold values for the color set by the threshold value setting means.

Claim 27

An image processing apparatus for forming an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a first plurality of color signals in a first exposure and a second plurality of color signals in a second exposure, an amount of the second exposure being larger than an amount of the first exposure; threshold value setting means for setting a threshold value for each color on the basis of the first plurality of color signals obtained in the first exposure and the second plurality of color signals obtained in the second exposure, by the image picking-up means; and combining means for generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the first exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a lowest threshold value out of threshold values for the color set by the threshold value setting means.

Claim 28

An image processing apparatus for forming an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a first plurality of color signals in a first exposure and a second plurality of color signals in a second exposure, an amount of the second exposure being larger than an amount of the first exposure; threshold value setting means for setting a threshold value predetermined color filter of said image sensing device for each color on the basis of the first plurality of color signals obtained in the first exposure and the second plurality of color signals obtained in the second exposure, by the image picking-up means; and combining means for generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the first exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a highest threshold value out of threshold values for the color set by the threshold value setting means.

Claim 29

An image processing apparatus for forming an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a first plurality of color signals in a first exposure and a second plurality of color signals in a second exposure, an amount of the second exposure being larger than an amount of the first exposure; threshold value setting means for setting a threshold value predetermined color filter of said image sensing device for each color on the basis of the first plurality of color signals obtained in the first exposure and the second plurality of color signals obtained in the second exposure, by the image picking-up means; and combining means for generating a combined color signal by replacing, for each color, a part of a color signal out of the first plurality of color signals obtained in the first exposure with a corresponding part of a corresponding color signal of the second plurality of color signals, a replacement of a color signal part being made on the basis of a highest threshold value out of threshold values for the color set by the threshold value setting means.

Claim 30

An image processing method of forming an image based on color signals output from an image sensing device, comprising: an image picking-up step of picking up, in advance by said image sensing device, a plurality of color signals in respective exposures, the respective exposures being different from each other; a threshold value setting step of setting a threshold value for each color on the basis of a predetermined kind of color signal from among the plurality of color signals obtained in the image picking-up step; and the combining step of generating a combined color signal by selecting, for each color, a color signal from among the plurality of color signals obtained in the image picking-up step, the color signal being selected using the one of the threshold values set in the threshold value setting step which corresponds to the selected color signal.

Claim 31

An image processing method of forming an image based on color signals output from an image sensing device, comprising: an image picking-up step of picking up, in advance by said image sensing device, a plurality of color signals in respective exposures, the respective exposures being different from each other; a threshold value setting step of setting a threshold value for each color on the basis of a predetermined kind of color signal from among the plurality of color signals obtained in the image picking-up step; and the combining step of generating a combined color signal by selecting, for each color, a color signal from among the plurality of color signals obtained in the image picking-up step, the color signal being selected using the one of the threshold values set in the threshold value setting step which corresponds to the selected color signal.

Claim 32

The method according to claim 24, wherein the threshold value is a value that does not saturate all color signals obtained in at least two different exposures from among the respective exposures.

Claim 33

The method according to claim 24, wherein the threshold value is a value that does not generate a noise in all color signals obtained in at least two different exposures from among the respective exposures.

Claim 34

The method according to claim 24, wherein the predetermined kind of color signal used in the threshold value setting step is yellow.

Claim 35

An image processing apparatus for forming an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a plurality of color signals in respective exposures, the respective exposures being different from each other; threshold value setting means for setting a threshold value for each color on the basis of a predetermined kind of color signal from among the plurality of color signals obtained by the image picking-up means; and combining means for generating a combined color signal by selecting, for each color, a color signal from among the plurality of color signals obtained by the image picking-up means, the color signal being selected using the one of the threshold values set in the threshold value setting means which corresponds to the selected color signal.

Claim 36

An image processing apparatus for forming an image based on color signals output from an image sensing device, comprising: image picking-up means for picking up, in advance by said image sensing device, a plurality of color signals in respective exposures, the respective exposures being different from each other; threshold value setting means for setting a threshold value for each color on the basis of a predetermined kind of color signal from among the plurality of color signals obtained by the image picking-up means; and combining means for generating a combined color signal by selecting, for each color, a color signal from among the plurality of color signals obtained by the image picking-up means, the color signal being selected using the one of the threshold values set in the threshold value setting means which corresponds to the selected color signal.

Claim 37

The apparatus according to claim 28, wherein the threshold value is a value that does not saturate all color signals obtained in at least two different exposures from among the respective exposures.

Claim 38

The apparatus according to claim 28, wherein the threshold value is a value that does not generate a noise in all color signals obtained in at least two different exposures from among the respective exposures.

Claim 39

The apparatus according to claim 28, wherein the predetermined kind of color signal used by the threshold value setting means is yellow.