Edge-dependent interpolation method for color reconstruction in image processing devices

Claim 2

An image processor for reconstructing the missing color components for image pixels of an image from an associated image sensor array with three color components, each image pixel having a single color component pixel intensity value, said image processor comprising: means for determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, wherein each edge detection kernel comprises a plurality of edge detection kernel pixels neighboring said image pixel; and means for reconstructing the missing color components of each image pixel having: edge direction detection means for detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernels, and color reconstruction means for determining, for each image pixel, an intensity value of each said missing color component and for saving all color components for each image pixel in a reconstructed array, if the edge is found, said edge direction detection means determining the side of the edge the image pixel belongs to by comparing an intensity value of the image pixel with an average intensity value of the neighboring same-color pixels at each side of the determined edge, and said color reconstruction means determining the intensity value of each said missing color component as a hue-weighted intensity value of the neighboring image pixels of the same missing color, on the same side of the detected image edge, or, if the image pixel is on the edge, of the neighboring image pixels of the same missing color from the edge or a line parallel with and closest to the edge, and in the absence of an edge, said color reconstruction means determining the intensity value of each said missing color component as a hue-weighted average intensity value of the surrounding image pixels of the same missing color, said hue-weighing comprising multiplication with the image pixel intensity value and division with an average intensity value of the image pixel and a plurality of neighboring image pixels of the same color.

Claim 3

The processor according to claim 1 , wherein said edge direction detection means comparing the resulting intensity values obtained for each said edge detection kernel, wherein each said resulting intensity value is a total of the intensity values of all said edge detection kernel pixels of said kernel, and finding a maximum resulting intensity value higher than a predetermined threshold.

Claim 4

The processor according to claim 1 , wherein said edge direction detection means and color reconstruction means are included in a computer program embeddable within the image processor hardware.

Claim 5

The processor according to claim 1 , wherein: said image sensor array being covered with a color filter array; said image sensor array and said color filter array having a half-pixel offset pixel pattern, wherein each image pixel in every second row is offset by a half-pixel in horizontal direction from image pixels in neighboring rows, in order to get a high resolution image; and a 45 degree kernel and a −45 degree kernel are sensitive to +30 degrees and −30 degrees edges, respectively.

Claim 6

The processor according to claim 4 , wherein said reconstructed array also having a half-pixel offset, and wherein said processor further comprises a horizontal interpolation means and a vertical interpolation means.

Claim 7

The processor according to claim 5 , wherein said processor comparing the intensity values of the same-color component of the horizontally neighboring pixels of the image pixel and, if the difference is higher than a threshold, using the vertical interpolation means to obtain intensity values of all color components for said image pixel by averaging intensity values of the corresponding color component of the vertically neighboring pixels.

Claim 8

The processor according to claim 5 , wherein said processor comparing the intensity values of the same-color component of two horizontally neighboring pixels of the image pixel and, if the difference is lower than a threshold, using the horizontal interpolation means to obtain intensity values of all color components for said image pixel by averaging intensity values of the corresponding color component of the two horizontally neighboring pixels.

Claim 9

An image processor for reconstructing the missing color components for image pixels of an image from an associated image sensor array with three color components, each image pixel having a single color component pixel intensity value, said image processor comprising: means for determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, wherein each edge detection kernel comprises a plurality of edge detection kernel pixels neighboring said image pixel; and means for reconstructing the missing color components of each image pixel having: edge direction detection means for detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernels, and color reconstruction means for determining, for each image pixel, an intensity value of each said missing color component and for saving all color components for each image pixel in a reconstructed array, said plurality of edge detection kernels comprising four edge detection kernels, namely a horizontal, a vertical, a 45 degree and a −45 degree kernel, said kernels sensitive to horizontal, vertical, +45 degrees and −45 degrees edges, respectively, each said edge detection kernel comprising four edge detection kernel pixels encircling and neighboring said image pixel, namely a top-left, a top-right, a bottom-left and a bottom-right kernel pixel of said image, and wherein, in order to calculate the resulting intensity value as a total of the intensity values of all said edge detection kernel pixels of said kernel, said horizontal kernel multiplying said bottom-left and bottom-right kernel pixel intensity values with −1, said vertical kernel multiplying said top-right and bottom-right kernel pixel intensity values with −1, said 45 degree kernel multiplying said top-right and bottom-left kernel pixel intensity values with zero and said bottom-right pixel intensity value with −1, and said −45 degree kernel multiplying said top-left and bottom-right kernel pixel intensity values with zero and said bottom-left pixel intensity value with −1.

Claim 10

An image processor for reconstructing the missing color components for image pixels of an image from an associated image sensor array with three color components, each image pixel having a single color component pixel intensity value, said image processor comprising: means for determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, wherein each edge detection kernel comprises a plurality of edge detection kernel pixels neighboring said image pixel; and means for reconstructing the missing color components of each image pixel having: edge direction detection means for detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernels, and color reconstruction means for determining, for each image pixel, an intensity value of each said missing color component and for saving all color components for each image pixel in a reconstructed array, said plurality of edge detection kernels comprising four edge detection kernels, namely a horizontal, a vertical, a 45 degree and a −45 degree kernel, said kernels sensitive to horizontal, vertical, +45 degrees and −45 degrees edges, respectively, each said edge detection kernel comprising four edge detection kernel pixels encircling and neighboring said image pixel, namely a top-left, a top-right, a bottom-left and a bottom-right kernel pixel of said image, and wherein, in order to calculate the resulting intensity value as a total of the intensity values of all said edge detection kernel pixels of said kernel, said horizontal kernel multiplying said bottom-left and bottom-right kernel pixel intensity values with −1, said vertical kernel multiplying said top-right and bottom-right kernel pixel intensity values with −1, said 45 degree kernel multiplying said top-right and bottom-left kernel pixel intensity values with zero and said bottom-right pixel intensity value with −1, and said −45 degree kernel multiplying said top-left and bottom-right kernel pixel intensity values with zero and said bottom-left pixel intensity value with −1.

Claim 11

An edge-dependent color interpolation method for reconstructing the missing color component for each pixel of an image received from an associated image sensor array, each image pixel having a single color component, said method comprising the steps of: determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, each edge detection kernel comprising a plurality of edge detection kernel pixels neighboring said image pixel; and reconstructing the missing color components of each image pixel, according to the steps: detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernel, and determining for each image pixel an intensity value of each said missing color component and saving all color components for each image pixel in a reconstructed array; if the edge is found, said edge direction detection step determining the side of the edge the image pixel belongs to by comparing an intensity value of the image pixel with average intensity values of the neighboring same-color pixels at each side of the determined edge, and said color reconstruction step determining the intensity value of each said missing color component as a hue-weighted intensity value of the neighboring image pixels of the same missing color, on the same side of the detected image edge, or, if the image pixel is on the edge, of the neighboring image pixels of the same missing color from the edge or a line parallel with and closest to the edge, and in the absence of an edge, said color reconstruction step determining the intensity value of each said missing color component as a hue-weighted average intensity value of the surrounding image pixels of the same missing color, said hue-weighing comprising the step of multiplying with the image pixel intensity value and dividing with an average intensity value of the image pixel and a plurality of neighboring image pixels of the same color.

Claim 12

An edge-dependent color interpolation method for reconstructing the missing color component for each pixel of an image received from an associated image sensor array, each image pixel having a single color component, said method comprising the steps of: determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, each edge detection kernel comprising a plurality of edge detection kernel pixels neighboring said image pixel; and reconstructing the missing color components of each image pixel, according to the steps: detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernel, and determining for each image pixel an intensity value of each said missing color component and saving all color components for each image pixel in a reconstructed array; if the edge is found, said edge direction detection step determining the side of the edge the image pixel belongs to by comparing an intensity value of the image pixel with average intensity values of the neighboring same-color pixels at each side of the determined edge, and said color reconstruction step determining the intensity value of each said missing color component as a hue-weighted intensity value of the neighboring image pixels of the same missing color, on the same side of the detected image edge, or, if the image pixel is on the edge, of the neighboring image pixels of the same missing color from the edge or a line parallel with and closest to the edge, and in the absence of an edge, said color reconstruction step determining the intensity value of each said missing color component as a hue-weighted average intensity value of the surrounding image pixels of the same missing color, said hue-weighing comprising the step of multiplying with the image pixel intensity value and dividing with an average intensity value of the image pixel and a plurality of neighboring image pixels of the same color.

Claim 13

The method according to claim 9 , wherein said edge direction detection step comparing the resulting intensity values obtained for each said edge detection kernel, wherein each said resulting intensity value is a total of the intensity values of all said edge detection kernel pixels of said kernel, and finding a maximum resulting intensity value higher than a predetermined threshold.

Claim 14

The method according to claim 9 , wherein said method is executed by a computer program embeddable within an image processor hardware.

Claim 15

The method according to claim 9 , wherein: said image sensor array being covered with a color filter array; said image sensor array and said color filter array having a half-pixel offset pixel pattern, wherein each image pixel in every second row is offset by a half-pixel in horizontal direction from image pixels in neighboring rows, in order to get a high resolution image; and a 45 degree kernel and a −45 degree kernel are sensitive to +30 degrees and −30 degrees edges, respectively.

Claim 16

The method according to claim 12 , wherein said reconstructed array also having a half-pixel offset, and wherein said method is executed by a computer program, said computer program further having a horizontal interpolation step and a vertical interpolation step.

Claim 17

The method according to claim 13 , wherein said computer program further comprising the following steps: comparing the intensity values of the same-color component of the horizontally neighboring pixels of the image pixel, and if the difference is higher than a threshold, using the vertical interpolation step to obtain intensity values of all color components for said image pixel by averaging intensity values of the corresponding color component of the vertically neighboring pixels.

Claim 18

The method according to claim 13 , wherein said computer program further comprising the following steps: comparing the intensity values of the same-color component of the horizontally neighboring pixels of the image pixel, and if the difference is lower than a threshold, using the horizontal interpolation step to obtain intensity values of all color components for said image pixel by averaging intensity values of the corresponding color component of the horizontally neighboring pixels.

Claim 19

An edge-dependent color interpolation method for reconstructing the missing color component for each pixel of an image received from an associated image sensor array, each image pixel having a single color component, said method comprising the steps of: determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, each edge detection kernel comprising a plurality of edge detection kernel pixels neighboring said image pixel; and reconstructing the missing color components of each image pixel, according to the steps: detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernel, and determining for each image pixel an intensity value of each said missing color component and saving all color components for each image pixel in a reconstructed array; said plurality of edge detection kernels comprising four edge detection kernels, namely a horizontal, a vertical, a 45 degree and a −45 degree kernel, said kernels sensitive to horizontal, vertical, +45 degrees and −45 degrees edges, respectively, each said edge detection kernel comprising four edge detection kernel pixels encircling said image pixel, namely a top-left, a top-right, a bottom-left and a bottom-right kernel pixel of said image, and wherein, in order to calculate the resulting intensity value as the total of the intensity values of all said edge detection kernel pixels of said kernel, said horizontal kernel multiplying said bottom-left and bottom-right kernel pixel intensity values with −1, said vertical kernel multiplying said top-right and bottom-right kernel pixel intensity values with −1, said 45 degree kernel multiplying said top-right and bottom-left kernel pixel intensity values with zero and said bottom-right pixel intensity value with −1, and said −45 degree kernel multiplying said top-left and bottom-right kernel pixel intensity values with zero and said bottom-left pixel intensity value with −1.

Claim 20

An edge-dependent color interpolation method for reconstructing the missing color component for each pixel of an image received from an associated image sensor array, each image pixel having a single color component, said method comprising the steps of: determining for each image pixel a plurality of edge detection kernels having the same color as the image pixel, each edge detection kernel comprising a plurality of edge detection kernel pixels neighboring said image pixel; and reconstructing the missing color components of each image pixel, according to the steps: detecting direction of edges of the image and a side of the edge the image pixel belongs to, according to a highest valid resulting intensity value of said edge detection kernel, and determining for each image pixel an intensity value of each said missing color component and saving all color components for each image pixel in a reconstructed array; said plurality of edge detection kernels comprising four edge detection kernels, namely a horizontal, a vertical, a 45 degree and a −45 degree kernel, said kernels sensitive to horizontal, vertical, +45 degrees and −45 degrees edges, respectively, each said edge detection kernel comprising four edge detection kernel pixels encircling said image pixel, namely a top-left, a top-right, a bottom-left and a bottom-right kernel pixel of said image, and wherein, in order to calculate the resulting intensity value as the total of the intensity values of all said edge detection kernel pixels of said kernel, said horizontal kernel multiplying said bottom-left and bottom-right kernel pixel intensity values with −1, said vertical kernel multiplying said top-right and bottom-right kernel pixel intensity values with −1, said 45 degree kernel multiplying said top-right and bottom-left kernel pixel intensity values with zero and said bottom-right pixel intensity value with −1, and said −45 degree kernel multiplying said top-left and bottom-right kernel pixel intensity values with zero and said bottom-left pixel intensity value with −1.

Claim 21

An edge-dependent color reconstruction method implemented in an image processor receiving pixel intensity values for an image from an associated three-color image sensor array, each image pixel having a single color component, said method comprising the steps of: determining for each image pixel a plurality of closest neighboring same-color pixels; calculating resulting intensity values for each of a horizontal combination, vertical combination, right-to-left-down combination, and right-to-left-up combination of said neighboring same-color pixels; determining if an edge exists by comparing the resulting intensity values to obtain a maximum resulting intensity value, and comparing the maximum resulting intensity value with a threshold value; and reconstructing and saving all color components for each image pixel in a reconstructed array, according to the steps: if an edge exists: determining the direction of an edge by determining the combination that has the maximum resulting intensity value, determining the side of the edge the image pixel belongs to by comparing the intensity value of the image pixel to an average color intensity value of the neighboring same-color pixels on each side of the edge, retrieving the neighboring pixels with the missing color components from the same edge side, and reconstructing the missing color components for the image pixel using the retrieved neighboring color components according to equation: missing color component=(average intensity value of retrieved neighboring same-color pixel components)*(intensity of image pixel)/(average intensity of the same-color pixels from the determined side), and in the absence of an edge: reconstructing the missing color components for the image pixel using the neighboring color components according to equation: missing color component=(average intensity value of neighboring same-color pixel components)*(intensity of image pixel)/(average intensity of the image pixel and the neighboring same-color pixels).

Claim 22

An edge-dependent color reconstruction method implemented in an image processor receiving pixel intensity values for an image from an associated three-color image sensor array, each image pixel having a single color component, said method comprising the steps of: determining for each image pixel a plurality of closest neighboring same-color pixels; calculating resulting intensity values for each of a horizontal combination, vertical combination, right-to-left-down combination, and right-to-left-up combination of said neighboring same-color pixels; determining if an edge exists by comparing the resulting intensity values to obtain a maximum resulting intensity value, and comparing the maximum resulting intensity value with a threshold value; and reconstructing and saving all color components for each image pixel in a reconstructed array, according to the steps: if an edge exists: determining the direction of an edge by determining the combination that has the maximum resulting intensity value, determining the side of the edge the image pixel belongs to by comparing the intensity value of the image pixel to an average color intensity value of the neighboring same-color pixels on each side of the edge, retrieving the neighboring pixels with the missing color components from the same edge side, and reconstructing the missing color components for the image pixel using the retrieved neighboring color components according to equation: missing color component=(average intensity value of retrieved neighboring same-color pixel components)*(intensity of image pixel)/(average intensity of the same-color pixels from the determined side), and in the absence of an edge: reconstructing the missing color components for the image pixel using the neighboring color components according to equation: missing color component=(average intensity value of neighboring same-color pixel components)*(intensity of image pixel)/(average intensity of the image pixel and the neighboring same-color pixels).

Claim 23

The method according to claim 17 , wherein said method is executed by a computer program embedded in the image processor chip, comprised within a digital camera with the image sensor array.

Claim 24

The method according to claim 17 , wherein said method is executed by a computer program embedded in the image processor, comprised within a host computer accompanying a digital camera with the image sensor array.

Claim 25

The method according to claim 17 , wherein: said horizontal combination is a horizontal kernel, said vertical combination is a vertical kernel, said right-to-left-down combination is a 45 degree kernel, and said right-to-left-up combination is a −45 degree kernel, said edge detection kernels sensitive to horizontal, vertical, +45 degrees and −45 degrees edges, respectively, each said edge detection kernel comprising four edge detection kernel pixels encircling and neighboring said image pixel, namely a top-left, a top-right, a bottom-left and a bottom-right kernel pixel of said image, and wherein, in order to calculate the resulting intensity value as a total of the intensity values of all said edge detection kernel pixels of said kernel, said horizontal kernel multiplying said bottom-left and bottom-right kernel pixel intensity values with −1, said vertical kernel multiplying said top-right and bottom-right kernel pixel intensity values with −1, said 45 degree kernel multiplying said top-right and bottom-left kernel pixel intensity values with zero and said bottom-right pixel intensity value with −1, and said −45 degree kernel multiplying said top-left and bottom-right kernel pixel intensity values with zero and said bottom-left pixel intensity value with −1.

Claim 26

The method according to claim 20 , wherein: said image sensor array being covered with a color filter array; said image sensor array and said color filter array having a half-pixel offset pixel pattern, wherein each image pixel in every second row is offset by a half-pixel in horizontal direction from image pixels in neighboring rows, in order to get a high resolution image; a 45 degree kernel and a −45 degree kernel are sensitive to +30 degrees and −30 degrees edges, respectively; said method is executed by a computer with a computer program; and said computer program further having a horizontal interpolation step and a vertical interpolation step, and being embeddable within the computer hardware.

Claim 27

The method according to claim 21 , wherein said computer program further comprising the following steps: comparing the intensity values of the same-color component of the horizontally neighboring pixels of the image pixel; and if the difference is higher than a threshold, using the vertical interpolation step to obtain intensity values of all color components for said image pixel by averaging intensity values of the corresponding color component of the vertically neighboring pixels, and if the difference is lower than a threshold, using the horizontal interpolation step to obtain intensity values of all color components for said image pixel by averaging intensity values of the corresponding color component of the horizontally neighboring pixels.