Patent US5604856A successfully refreshed from Google Patents.
US5604856A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Motion compensated noise reduction method and system for computer generated images
Assignee: Microsoft Technology Licensing LLC
•
Filing: 1994-10-13
•
Granted: 1997-02-18
•
Expires: 2014-10-13
Abstract
Abstract A system and method are provided to reduce noise in computer generated images. A source object is stochastically sampled to generate a frame image which appears in motion through multiple successive frames. The motion of the frame image is predicted and the samples taken at the predicted location in various projected frames are used to generate an improved, antialiased, noise-reduced image in a single reference frame. By using stochastically obtained samples from multiple frames, the noise reduction technique provides many additional samples without any additional cost of creating them.
Inventors
- 👤 Brian Guenter
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-02 20:13
CPC Classifications
G; G06; G06T; G06T5/00; G06T5/70; G06T5/20; G06T5/50; G06T2200/00; G06T2200/12; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/20; G06T2207/20172; G06T2207/20182
Claims Analysis
Total Claims: 38
Claim 1
Independent Claim
230 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; deriving an intensity of the reference pixel in the reference frame by using the intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point in the visual display screen when the projected frame is depicted on the visual display screen, the intensity of the reference pixel comprises multiple illumination components selected from a group comprising intrinsic reflectance, shadow, specular reflectance, and specular spot; breaking the intensity of the one or more adjacent pixel intensities in the projected frame into multiple illumination components which correspond to the illumination components of the reference pixel; and the deriving step comprises independently deriving each illumination component of the reference pixel by using each illumination component in the intensity of the one or more adjacent pixels.
Claim 2
Independent Claim
230 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; deriving an intensity of the reference pixel in the reference frame by using the intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point in the visual display screen when the projected frame is depicted on the visual display screen, the intensity of the reference pixel comprises multiple illumination components selected from a group comprising intrinsic reflectance, shadow, specular reflectance, and specular spot; breaking the intensity of the one or more adjacent pixel intensities in the projected frame into multiple illumination components which correspond to the illumination components of the reference pixel; and the deriving step comprises independently deriving each illumination component of the reference pixel by using each illumination component in the intensity of the one or more adjacent pixels.
Claim 3
Independent Claim
161 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to multiple projected locations in multiple projected frames; deriving an intensity of the reference pixel in the reference frame by using intensities of multiple pixels in the projected frames which are adjacent to the projected location of the moving image point; and varying in number the multiple frames into which the motion of the frame image point is predicted based upon a number of pixels which are usable from the projected frames.
Claim 4
Independent Claim
161 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to multiple projected locations in multiple projected frames; deriving an intensity of the reference pixel in the reference frame by using intensities of multiple pixels in the projected frames which are adjacent to the projected location of the moving image point; and varying in number the multiple frames into which the motion of the frame image point is predicted based upon a number of pixels which are usable from the projected frames.
Claim 5
Independent Claim
172 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; analyzing any extent that the computer generated image is distorted due to its motion from the reference frame to the projected frame; and deriving an intensity of the reference pixel in the reference frame by using the intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point in the visual display screen when the projected frame is depicted on the visual display screen.
Claim 6
Independent Claim
172 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; analyzing any extent that the computer generated image is distorted due to its motion from the reference frame to the projected frame; and deriving an intensity of the reference pixel in the reference frame by using the intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point in the visual display screen when the projected frame is depicted on the visual display screen.
Claim 7
Dependent Claim
39 words
A method according to claim 3 further comprising the step of performing a two dimensional vector analysis to analyze for any distortion in the computer generated image due to its motion from the reference frame to the projected frame.
Claim 8
Dependent Claim
80 words
A method according to claim 3 further comprising the additional steps: performing a two dimensional vector analysis to analyze for any distortion in the computer generated image due to its motion from the reference frame to the projected frame; computing eigenvalues of the distorted unit vectors in the reference frame; and avoiding use of the pixel intensity from the projected frame when deriving the pixel intensity of the reference pixel in the event that the eigenvalues exceed a selected threshold.
Claim 9
Independent Claim
217 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; deriving an intensity of the reference pixel in the reference frame by using the intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point in the visual display screen when the projected frame is depicted on the visual display screen; determining if the one or more pixels in the projected frame correspond to the same computer generated image to which the frame image point corresponds; and filtering the intensity of the one or more pixels in the projected frame which correspond to the same computer generated image through a low-pass filter prior to using the intensity to derive an intensity of the reference pixel in the reference frame.
Claim 10
Independent Claim
217 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the method comprising the following steps: providing a frame image point on a computer generated image in a reference frame, the frame image point corresponding to a reference pixel in the visual display screen when the reference frame is depicted on the visual display screen; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; deriving an intensity of the reference pixel in the reference frame by using the intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point in the visual display screen when the projected frame is depicted on the visual display screen; determining if the one or more pixels in the projected frame correspond to the same computer generated image to which the frame image point corresponds; and filtering the intensity of the one or more pixels in the projected frame which correspond to the same computer generated image through a low-pass filter prior to using the intensity to derive an intensity of the reference pixel in the reference frame.
Claim 11
Independent Claim
291 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the computer generated image being formed by sampling a mathematical function representative of a source object to yield intensity values for corresponding pixels of the display screen, the samples being used to form a series of frames wherein the computer generated image appears in motion across the visual display screen during the series of frames which appear in sequence, the method comprising the following steps: providing a frame image point on the computer generated image in a reference frame, the frame image point having pixel datum for a corresponding reference pixel in the visual display screen when the reference frame is depicted on the visual display screen, the pixel datum comprising an object number indicative of the source object, multiple color components, and a depth value; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; determining whether the frame image point in the projected frame is occluded by another computer generated image; analyzing any extent that the computer generated image is distorted due to its motion from the reference frame to the projected frame; identifying one or more pixels in the projected frame which are adjacent to the projected location of the moving frame image point; filtering the intensity of the one or more pixels in the projected frame through a low-pass filter; and summing the filtered intensity of the one or more pixels in the projected frame with the intensity of the reference pixel in the reference frame to derive an improved intensity of the reference pixel.
Claim 12
Independent Claim
291 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the computer generated image being formed by sampling a mathematical function representative of a source object to yield intensity values for corresponding pixels of the display screen, the samples being used to form a series of frames wherein the computer generated image appears in motion across the visual display screen during the series of frames which appear in sequence, the method comprising the following steps: providing a frame image point on the computer generated image in a reference frame, the frame image point having pixel datum for a corresponding reference pixel in the visual display screen when the reference frame is depicted on the visual display screen, the pixel datum comprising an object number indicative of the source object, multiple color components, and a depth value; predicting motion of the frame image point from its reference location in the reference frame to a projected location in a projected frame; determining whether the frame image point in the projected frame is occluded by another computer generated image; analyzing any extent that the computer generated image is distorted due to its motion from the reference frame to the projected frame; identifying one or more pixels in the projected frame which are adjacent to the projected location of the moving frame image point; filtering the intensity of the one or more pixels in the projected frame through a low-pass filter; and summing the filtered intensity of the one or more pixels in the projected frame with the intensity of the reference pixel in the reference frame to derive an improved intensity of the reference pixel.
Claim 13
Dependent Claim
45 words
A method according to claim 7 wherein the method is prematurely stopped in the event one of the following occurs: (1) the frame image point is occluded by another computer generated image, and (2) the computer generated image is distorted beyond a selected threshold amount.
Claim 14
Dependent Claim
82 words
A method according to claim 7 wherein the one or more pixels in the projected frame are actuated in accordance with associated pixel datum which individually comprises an object number, multiple color components, and a depth value; and the filtering and summing steps comprise filtering the individual color components through a low-pass filter and summing the filtered color components of the one or more pixels in the projected frame with the corresponding color components of the reference pixel in the reference frame.
Claim 15
Dependent Claim
94 words
A method according to claim 7 wherein the intensity of the reference pixel comprises multiple illumination components, the method further comprising: breaking the intensity of the one or more pixel intensities in the projected frame into multiple illumination components which correspond to the illumination components of the reference pixel; and the filtering and summing steps comprise filtering the individual illumination components through a low-pass filter and summing the filtered illumination components of the one or more pixels in the projected frame with the corresponding illumination components of the reference pixel in the reference frame.
Claim 16
Dependent Claim
33 words
A method according to claim 7 wherein the occlusion determination step comprises comparing the object number of the pixel datum with object numbers of pixel datum associated with pixels in the projected frame.
Claim 17
Dependent Claim
24 words
A method according to claim 7 wherein the occlusion determination step comprises a depth analysis determination using the depth value of the pixel datum.
Claim 18
Independent Claim
528 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the computer generated image being formed by sampling a mathematical function representative of a source object to yield intensity values for corresponding pixels of the visual display screen, the samples being used to form a series of frames wherein the computer generated image appears in motion across the visual display screen during the series of frames which appear in sequence, the method comprising the following steps: (a) providing a source point on the source object, the source point being mapped through a transformation T i into a reference frame image point RFIP i in a reference frame i, the reference frame image point RFIP i corresponding to a reference pixel p i in the visual display screen when the reference frame i is depicted on the visual display screen; (b) predicting motion of the reference frame image point RFIP i in the reference frame i to a projected frame image point PFIP k in a projected frame k, the projected frame image point PFIP k also relating to the source point on the source object, the projected frame image point PFIP k corresponding to the reference frame image point RFIP i via a motion prediction relationship PFIP k =T k T i -1 RFIP i which accounts for the motion of the computer generated image across the visual display screen from one position in the reference frame i to another position in the projected frame k, the transformation T k of the motion prediction relationship mapping the source point on the source object into the reference frame k, the projected frame image point PFIP k being positioned within the projected frame k in an area defined by four nearest pixels p k1 -p k4 ; (c) determining whether the projected frame image point PFIP k is occluded in the projected frame k by another computer generated image; (d) selecting a number of neighboring pixels p n1 -p nm surrounding the reference pixel p i in the reference frame i, the neighboring pixels p n1 -p nm defining a reference bounding periphery in the reference frame i; (e) transforming the reference bounding periphery in the reference frame i into the reference frame k using a distortion transformation P which accounts for any distortion of the moving computer generated image from one position in the reference frame i to another position in the projected frame k; (f) defining a projected bounding periphery within the projected frame k which encompasses the transformed and distorted reference bounding periphery; (g) identifying pixels in the projected frame k that (1) lie within the projected bounding periphery and (2) are being used to depict the computer generated image, the identified pixels having associated pixel intensities; (h) filtering the pixel intensities of the identified pixels in the projected frame k using a selected filter to produce filtered pixel intensities; and (i) using the filtered pixel intensities from the projected frame k to derive a pixel intensity of the reference pixel p i in the reference frame i.
Claim 19
Independent Claim
528 words
A method for reducing noise in a computer generated image that is created for viewing on a visual display screen, the visual display screen having a matrix of pixels, the computer generated image being formed by sampling a mathematical function representative of a source object to yield intensity values for corresponding pixels of the visual display screen, the samples being used to form a series of frames wherein the computer generated image appears in motion across the visual display screen during the series of frames which appear in sequence, the method comprising the following steps: (a) providing a source point on the source object, the source point being mapped through a transformation T i into a reference frame image point RFIP i in a reference frame i, the reference frame image point RFIP i corresponding to a reference pixel p i in the visual display screen when the reference frame i is depicted on the visual display screen; (b) predicting motion of the reference frame image point RFIP i in the reference frame i to a projected frame image point PFIP k in a projected frame k, the projected frame image point PFIP k also relating to the source point on the source object, the projected frame image point PFIP k corresponding to the reference frame image point RFIP i via a motion prediction relationship PFIP k =T k T i -1 RFIP i which accounts for the motion of the computer generated image across the visual display screen from one position in the reference frame i to another position in the projected frame k, the transformation T k of the motion prediction relationship mapping the source point on the source object into the reference frame k, the projected frame image point PFIP k being positioned within the projected frame k in an area defined by four nearest pixels p k1 -p k4 ; (c) determining whether the projected frame image point PFIP k is occluded in the projected frame k by another computer generated image; (d) selecting a number of neighboring pixels p n1 -p nm surrounding the reference pixel p i in the reference frame i, the neighboring pixels p n1 -p nm defining a reference bounding periphery in the reference frame i; (e) transforming the reference bounding periphery in the reference frame i into the reference frame k using a distortion transformation P which accounts for any distortion of the moving computer generated image from one position in the reference frame i to another position in the projected frame k; (f) defining a projected bounding periphery within the projected frame k which encompasses the transformed and distorted reference bounding periphery; (g) identifying pixels in the projected frame k that (1) lie within the projected bounding periphery and (2) are being used to depict the computer generated image, the identified pixels having associated pixel intensities; (h) filtering the pixel intensities of the identified pixels in the projected frame k using a selected filter to produce filtered pixel intensities; and (i) using the filtered pixel intensities from the projected frame k to derive a pixel intensity of the reference pixel p i in the reference frame i.
Claim 20
Dependent Claim
69 words
A method according to claim 13 wherein: the four nearest projected pixels p k1 -p k4 are used to display one or more computer generated images in the projected frame k; and the occlusion determination step (d) comprises determining whether the one or more computer generated images being depicted by the projected pixels p k1 -p k4 are different than the computer generated image corresponding to the source object.
Claim 21
Dependent Claim
97 words
A method according to claim 13 wherein: the frame image points have associated depth values and a pixel onto which one or more frame image points map depicts the frame image point with the least depth value among the one or more frame image points; and the occlusion determination step (d) comprises conducting a depth analysis test between a depth value associated with the projected frame image point PFIP k and any depth values of any frame image points from other computer generated images which also map onto the four nearest pixels p k1 -p k4 .
Claim 22
Dependent Claim
17 words
A method according to claim 15 wherein the depth analysis test is a z-buffer depth analysis test.
Claim 23
Dependent Claim
156 words
A method according to claim 13 wherein: the four nearest projected pixels p k1 -p k4 are used to display one or more computer generated images in the projected frame k; frame image points have associated depth values and a pixel onto which one or more frame image points map depicts the frame image point with the least depth value among the one or more frame image points; and the occlusion determination step (d) comprises the following sequential steps: determining whether the one or more computer generated images being depicted by the projected pixels p k1 -p k4 are different than the computer generated image corresponding to the source object; and conducting a depth analysis test between a depth value associated with the projected frame image point PFIP k and any depth values of any frame image points from other computer generated images which also map onto the four nearest pixels p k1 -p k4 .
Claim 24
Dependent Claim
20 words
A method according to claim 13 wherein the projected bounding periphery is defined by pixels in the projected frame k.
Claim 25
Dependent Claim
22 words
A method according to claim 13 further comprising the step of calculating the distortion transformation P using a two dimensional vector analysis.
Claim 26
Dependent Claim
77 words
A method according to claim 13 further comprising the additional steps: calculating the distortion transformation P using a two dimensional vector analysis which examines change in unit vectors originating at the reference frame image point RFIP i in the reference frame i to the same, but distorted, unit vectors originating at the projected frame image point PFIP k in the reference frame k; and computing the eigenvalues of the distorted unit vectors in the reference frame k.
Claim 27
Dependent Claim
33 words
A method according to claim 20 further comprising the step of discarding use of the pixel intensity values from the projected frame k in the event that the eigenvalues exceed a selected threshold.
Claim 28
Dependent Claim
52 words
A method according to claim 13 further comprising the step of transforming the identified pixels from the projected frame k back to the reference frame i using an inverse distortion transformation P -1 ; and the filtering step (h) comprises filtering the pixel intensities of the identified pixels through a low-pass filter.
Claim 29
Dependent Claim
50 words
A method according to claim 13 wherein the using step (i) comprises averaging the filtered pixel intensities from the projected frame k with the pixel intensity of the reference pixel p i in the reference frame i to derive an improved pixel intensity of the reference pixel p i .
Claim 30
Dependent Claim
96 words
A method according to claim 13 wherein the intensity of the reference pixel p i comprises multiple color components, the method further comprising: breaking the pixel intensities of the identified pixels into multiple color components which correspond to the color components of the reference pixel p i ; the filtering step (h) comprises filtering each color component in the pixel intensities of the identified pixels; the using step (i) comprises using each filtered color component in the pixel intensities of the identified pixels to derive the corresponding color component of the reference pixel p i .
Claim 31
Dependent Claim
16 words
A method according to claim 13 wherein the steps (a)-(i) are repeated for multiple projected frames.
Claim 32
Dependent Claim
14 words
A method according to claim 25 wherein the multiple projected frames vary in number.
Claim 33
Dependent Claim
95 words
A method according to claim 13 wherein the intensity of the reference pixel p i comprises multiple illumination components, the method further comprising: breaking the pixel intensities of the identified pixels into multiple illumination components which correspond to the illumination of the reference pixel p i ; the filtering step (h) comprises filtering each illumination component in the pixel intensities of the identified pixels; the using step (i) comprises using each filtered illumination component in the pixel intensities of the identified pixels to derive the corresponding illumination component of the reference pixel p i .
Claim 34
Dependent Claim
44 words
A method according to claim 13 wherein the method is stopped in the event one of the following occurs: (1) the frame image point is occluded by another computer generated image, and (2) the computer generated image is distorted beyond a selected threshold amount.
Claim 35
Independent Claim
141 words
A system for reducing noise in a computer generated image that can be converted into a pixel display, the computer generated image comprising multiple frame image points in a reference frame, the frame image points corresponding to reference pixels in a visual display screen, the system comprising: a motion predictor to predict motion of a frame image point from its reference location in a reference frame to a projected location in a projected frame; a distortion analyzer which analyzes any extent that the computer generated image is distorted due to its motion from the reference frame to the projected frame; and an intensity processor which derives an intensity of the reference pixel in the reference frame by using an intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point.
Claim 36
Independent Claim
141 words
A system for reducing noise in a computer generated image that can be converted into a pixel display, the computer generated image comprising multiple frame image points in a reference frame, the frame image points corresponding to reference pixels in a visual display screen, the system comprising: a motion predictor to predict motion of a frame image point from its reference location in a reference frame to a projected location in a projected frame; a distortion analyzer which analyzes any extent that the computer generated image is distorted due to its motion from the reference frame to the projected frame; and an intensity processor which derives an intensity of the reference pixel in the reference frame by using an intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point.
Claim 37
Independent Claim
173 words
A system for reducing noise in a computer generated image that can be converted into a pixel display, the computer generated image comprising multiple frame image points in a reference frame, the frame image points corresponding to reference pixels in a visual display screen, the system comprising: a motion predictor to predict motion of a frame image point from its reference location in a reference frame to a projected location in a projected frame; and an intensity processor which derives an intensity of the reference pixel in the reference frame by using an intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point, the intensity processor determining if the one or more pixels in the projected frame correspond to the same computer generated image to which the frame image point corresponds, the intensity processor including a low-pass filter which filters the intensity of the one or more adjacent pixels in the projected frame which correspond to the same computer generated image.
Claim 38
Independent Claim
173 words
A system for reducing noise in a computer generated image that can be converted into a pixel display, the computer generated image comprising multiple frame image points in a reference frame, the frame image points corresponding to reference pixels in a visual display screen, the system comprising: a motion predictor to predict motion of a frame image point from its reference location in a reference frame to a projected location in a projected frame; and an intensity processor which derives an intensity of the reference pixel in the reference frame by using an intensity of one or more pixels in the projected frame which are adjacent to the projected location of the moving image point, the intensity processor determining if the one or more pixels in the projected frame correspond to the same computer generated image to which the frame image point corresponds, the intensity processor including a low-pass filter which filters the intensity of the one or more adjacent pixels in the projected frame which correspond to the same computer generated image.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-02-18 | 2000-08-19 | 2001-02-18 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-02-18 | 2004-08-19 | 2005-02-18 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-02-18 | 2008-08-19 | 2009-02-18 | $7700.00 | Window Closed / Lapsed |