Color interpolation for image sensors using a local linear regression method

Claim 2

An image processing system for demosaicing an image represented by raw sensor values generated by pixels within an array of pixels having at least two color planes, each of said pixels within said array of pixels producing a respective one of said raw sensor values in one of the at least two color planes for said image, said image processing system comprising: a buffer for storing at least a group of said raw sensor values produced by said pixels within a region of said image, said region being less than said image; and a processor connected to receive said group of raw sensor values stored in said buffer, determine regression coefficients representing the linear relations between the color planes for said region using at least said group of raw sensor values and calculate missing color sensor values associated with said pixels in said region of said image using said regression coefficients, said processor configured to use correlation coefficients between different ones of the color planes in said region to determine said regression coefficients.

Claim 3

The image processing system of claim 1 , wherein said regression coefficients comprise a slope value and an intercept value representing the linear relationship between two of the color planes in said region.

Claim 4

The image processing system of claim 2 , wherein said regression coefficients comprise a first slope value and a first intercept value for use in calculating said missing color sensor values in a first one of the color planes from said raw sensor values in a second one of the color planes in said region and a second slope value and a second intercept value for use in calculating said missing color sensor values in the second color plane from said raw sensor values in the first color plane in said region.

Claim 5

The image processing system of claim 1 , wherein a subset of said group of raw sensor values in said region is used to determine said regression coefficients.

Claim 6

The image processing system of claim 1 , wherein said processor is further configured to calculate correlation coefficients between each of the color planes in said region, said correlation coefficients between two of the color planes including a separate correlation coefficient for each potential position of said pixels in a first one of the color planes relative to the position of said pixels in a second one of the color planes.

Claim 7

The image processing system of claim 5 , wherein said processor is further configured to select a pixel position for the first color plane associated with a maximum one of said correlation coefficients between the first and second color planes, and each of said raw sensor values in the first color plane generated from said pixels within said selected pixel position relative to said pixels in the second color plane are used to determine said regression coefficients between the first and second color planes.

Claim 8

An image processing system for demosaicing an image represented by raw sensor values generated by pixels within an array of pixels having at least two color planes, each of said pixels within said array of pixels producing a respective one of said raw sensor values in one of the at least two color planes for said image, said image processing system comprising: a buffer for storing at least a group of said raw sensor values produced by said pixels within a region of said image, said region being less than said image; and a processor connected to receive said group of raw sensor values stored in said buffer, determine regression coefficients representing the linear relations between the color planes for said region using at least said group of raw sensor values and calculate missing color sensor values associated with said pixels in said region of said image using said regression coefficients, said processor configured to use a blurred image associated with said region of said image to determine said regression coefficients.

Claim 9

An image processing system for demosaicing an image represented by raw sensor values generated by pixels within an array of pixels having at least two color planes, each of said pixels within said array of pixels producing a respective one of said raw sensor values in one of the at least two color planes for said image, said image processing system comprising: a buffer for storing at least a group of said raw sensor values produced by said pixels within a region of said image, said region being less than said image; and a processor connected to receive said group of raw sensor values stored in said buffer, determine regression coefficients representing the linear relations between the color planes for said region using at least said group of raw sensor values and calculate missing color sensor values associated with said pixels in said region of said image using said regression coefficients, said processor configured to use a blurred image associated with said region of said image to determine said regression coefficients.

Claim 10

The image processing system of claim 7 , wherein said regression coefficients are determined using the high spatial frequency components of said group of raw sensor values.

Claim 11

The image processing system of claim 7 , wherein said processor is further configured to calculate interpolated missing color sensor values for said region form said group of raw sensor values to produce a first demosaiced image, generate said blurred image for said region of said image through a convolution of said interpolated missing color sensor values and said group of raw sensor values with a lowpass kernel, and produce a difference image by subtracting said blurred image from said first demosaiced image, and said regression coefficients are determined using said difference image.

Claim 12

The image processing system of claim 9 , wherein said processor is further configured to produce a second demosaiced image using said regression coefficients calculated from said difference image and output a final demosaiced image for said region by adding said blurred image to said second demosaiced image.

Claim 13

An image processing system for demosaicing an image represented by raw sensor values generated by pixels within an array of pixels having at least two color planes, each of said pixels within said array of pixels producing a respective one of said raw sensor values in one of the at least two color planes for said image, said image processing system comprising: a buffer for storing at least a group of said raw sensor values produced by said pixels within a region of said image, said region being less than said image; and a processor connected to receive said group of raw sensor values stored in said buffer, determine regression coefficients representing the linear relations between the color planes for said region using at least said group of raw sensor values and calculate missing color sensor values associated with said pixels in said region of said image using said regression coefficients, said processor configured to calculate interpolated missing color sensor values for said region from said group of raw sensor values and compare said interpolated missing color sensor values with said associated calculated missing color sensor values for each of said pixels within said region to detect grid artifacts produced by said calculated missing color sensor values.

Claim 14

An image processing system for demosaicing an image represented by raw sensor values generated by pixels within an array of pixels having at least two color planes, each of said pixels within said array of pixels producing a respective one of said raw sensor values in one of the at least two color planes for said image, said image processing system comprising: a buffer for storing at least a group of said raw sensor values produced by said pixels within a region of said image, said region being less than said image; and a processor connected to receive said group of raw sensor values stored in said buffer, determine regression coefficients representing the linear relations between the color planes for said region using at least said group of raw sensor values and calculate missing color sensor values associated with said pixels in said region of said image using said regression coefficients, said processor configured to calculate interpolated missing color sensor values for said region from said group of raw sensor values and compare said interpolated missing color sensor values with said associated calculated missing color sensor values for each of said pixels within said region to detect grid artifacts produced by said calculated missing color sensor values.

Claim 15

The image processing system of claim 11 , wherein said processor is further configured to detect and reduce grid artifacts produced by said calculated missing color sensor values.

Claim 16

The image processing system of claim 11 , wherein said processor is further configured to determine a difference value between said interpolated missing color sensor value in a first one of the color planes and said associated calculated missing color sensor value in the first color plane for a select one of said pixels within said region, said processor being further configured to replace said calculated missing color sensor value with said interpolated missing color sensor value for the first color plane of said select pixel when said difference value exceeds a threshold amount.

Claim 17

The image processing system of claim 13 , wherein said processor is further configured to detect edges located within said region of said image, said calculated missing color sensor value being used regardless of said difference value when said select pixel lies on one of said edges.

Claim 20

The method of claim 15 , wherein said step of determining said regression coefficients further comprises: calculating a slope value and an intercept value representing the linear relationship between two of the color planes in said region.

Claim 21

The method of claim 16 , wherein said step of calculating said missing color sensor values further comprises: calculating said missing color sensor values in a first one of the color planes from said raw sensor values in a second one of the color planes in said region using a first slope value and a first intercept value; and calculating said missing color sensor values in the second color plane from said raw sensor values in the first color plane in said region a second slope value and a second intercept value.

Claim 22

The method of claim 15 , wherein said step of determining said regression coefficients further comprises: calculating correlation coefficients between each of the color planes in said region, said correlation coefficients between two of the color planes including a separate correlation coefficient for each potential position of said pixels in a first one of the color planes relative to the position of said pixels in a second one of the color planes.

Claim 23

The method of claim 18 , wherein said step of determining said regression coefficients comprises: selecting a pixel position for the first color plane associated with a maximum one of said correlation coefficients between the first and second color planes; and determining said regression coefficients between the first and second color planes using each of said raw sensor values in the first color plane generated from said pixels within said selected pixel position relative to said pixels in the second color plane.

Claim 26

The method of claim 20 , wherein said step of determining said regression coefficients further comprises: determining said regression coefficients using the high spatial frequency components of said group of raw sensor values.

Claim 27

The method of claim 20 , wherein said step of determining said regression coefficients further comprises: calculating interpolated missing color sensor values for said region from said group of raw sensor values to produce a first demosaiced image; generating said blurred image for said region of said image through a convolution of said interpolated missing color sensor values and said group of raw sensor values with a lowpass kernel; producing a difference image by subtracting said blurred image from said first demosaiced image; and determining said regression coefficients using said difference image.

Claim 28

The method of claim 22 , further comprising: producing a second demosaiced image using said regression coefficients calculated from said difference image; and outputting a final demosaiced image for said region by adding said blurred image to said second demosaiced image.

Claim 31

The method of claim 24 , further comprising: determining a difference value between said interpolated missing color sensor value in a first one of the color planes and said associated calculated missing color sensor value in the first color plane for a select one of said pixels within said region; and replacing said calculated missing color sensor value with said interpolated missing color sensor value for the first color plane of said select pixel when said difference value exceeds a threshold amount.

Claim 32

The method of claim 25 , further comprising: detecting edges located within said region of said image; and using said calculated missing color sensor value regardless of said difference value when said select pixel lies on one of said edges.