US6064393A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method for measuring the fidelity of warped image layer approximations in a real-time graphics rendering pipeline
Assignee: Microsoft Technology Licensing LLC
•
Filing: 1997-07-30
•
Granted: 2000-05-16
•
Expires: 2017-07-30
Abstract
Abstract A layered graphics rendering pipeline measures image fidelity ("fiducials") to determine how accurately a transformed image layer approximates a rendering of a 3D object. The graphics rendering pipeline approximates the change in position or color of 3D object by transforming a rendering of the 3D object from a previous frame. The pipeline uses the fiducials to control rendering of factored scene elements to independent image layers. The pipeline then combines the layers to compute frames of animation. The types of fiducials include sampling, visibility, and photometric fiducials. The sampling fiducial measures the distortion of an image sample when warped to screen coordinates. The visibility fiducial measures the change in visibility of a scene element since a previous rendering of the scene element. The photometric fiducial measures either the change in lighting from the time of the previous rendering to the current time, or it measures the difference between warped color samples and actual color samples of the scene element for a current frame.
Inventors
- 👤 Jerome E. Lengyel
- 👤 John Snyder
- 👤 James T. Kajiya
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-31 22:50
CPC Classifications
G; G06; G06T; G06T15/00; G06T15/005; G06T15/04
Claims Analysis
Total Claims: 70
Claim 1
Independent Claim
160 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer that approximates motion of 3D object geometry, comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing sampling distortion of the 2D transform; using the sampling distortion to determine whether to re-render the scene element for the current frame; approximating motion of the scene element by using the sprite transform to transform an initial rendering of the scene element at the initial frame to a new position in the current frame; compositing the image layers, including at least one transformed layer, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 2
Dependent Claim
160 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer that approximates motion of 3D object geometry, comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing sampling distortion of the 2D transform; using the sampling distortion to determine whether to re-render the scene element for the current frame; approximating motion of the scene element by using the sprite transform to transform an initial rendering of the scene element at the initial frame to a new position in the current frame; compositing the image layers, including at least one transformed layer, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 3
Dependent Claim
32 words
The method of claim 1 wherein the scene elements are rendered to separate image layers, each in a corresponding sprite coordinate space, and each at a non-fixed location in output device coordinates.
Claim 4
Dependent Claim
22 words
The method of claim 2 wherein at least one corresponding sprite space has a different spatial resolution from the output device coordinates.
Claim 5
Dependent Claim
21 words
The method of claim 2 wherein at least one corresponding sprite space has a different orientation from the output device coordinates.
Claim 6
Dependent Claim
21 words
The method of claim 1 wherein the step of computing the sampling distortion includes computing singular values for the 2D transform.
Claim 7
Dependent Claim
38 words
The method of claim 1 wherein the 2D transform is an affine transform that is capable of computing translation, rotation, scaling and shear of an image layer to approximate motion of object geometry depicted in the image layer.
Claim 8
Dependent Claim
47 words
The method of claim 6 including: computing sampling distortion for image samples in a variety of locations across the initial rendering of the first scene element; using the sampling distortion for the image samples across the initial rendering to determine whether to re-render the first scene element.
Claim 9
Dependent Claim
14 words
A computer readable medium including computer-executable instructions for performing the steps of claim 1.
Claim 10
Dependent Claim
184 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing visibility distortion caused by approximating motion of the scene element with the 2D transform by determining transitions between back-facing to front-facing surfaces of characteristic geometry of the scene element from the initial frame to the current frame; using the visibility distortion to determine whether to re-render the scene element for the current frame; approximating motion of the scene element by using the 2D transform to transform an initial rendering of the scene element at the initial frame to a new position in the current frame; compositing the image layers, including one or more transformed layers, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 11
Dependent Claim
184 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing visibility distortion caused by approximating motion of the scene element with the 2D transform by determining transitions between back-facing to front-facing surfaces of characteristic geometry of the scene element from the initial frame to the current frame; using the visibility distortion to determine whether to re-render the scene element for the current frame; approximating motion of the scene element by using the 2D transform to transform an initial rendering of the scene element at the initial frame to a new position in the current frame; compositing the image layers, including one or more transformed layers, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 12
Dependent Claim
17 words
The method of claim 9 wherein the image layers correspond to non-fixed portions of the output image.
Claim 13
Dependent Claim
58 words
The method of claim 9 including: computing 2D transforms for each of the scene elements to approximate the motion of the scene elements from a previous frame to the current frame; computing visibility distortion for each of the scene elements; determining whether to re-render the scene elements based on the visibility distortion of each of the scene elements.
Claim 14
Dependent Claim
76 words
The method of claim 9 further including: clipping characteristic geometry of the scene elements to an expanded viewing frustum that extends beyond screen boundaries of the output image; tracking position of clipped points of the characteristic geometry to determine whether the clipped points, when projected to the viewing frustum, are located within the screen boundaries; determining whether to re-render the image layers when the clipped points corresponding to the image layers are within the screen boundaries.
Claim 15
Dependent Claim
15 words
A computer readable medium having computer-executable instructions for performing the steps recited in claim 9.
Claim 16
Dependent Claim
171 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence by transforming an image layer representing an initial rendering of the scene element; measuring change in lighting of the scene element to compute photometric distortion of the 2D transform; using the photometric distortion to determine whether to re-render the first scene element; approximating motion of the scene element by using the 2D transform to transform an initial rendering of the scene element at the initial frame to a new position in the current frame; to construct frames in the animation sequence, compositing the image layers, including at least one transformed layer, into an output image in output device coordinates.
Claim 17
Dependent Claim
171 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence by transforming an image layer representing an initial rendering of the scene element; measuring change in lighting of the scene element to compute photometric distortion of the 2D transform; using the photometric distortion to determine whether to re-render the first scene element; approximating motion of the scene element by using the 2D transform to transform an initial rendering of the scene element at the initial frame to a new position in the current frame; to construct frames in the animation sequence, compositing the image layers, including at least one transformed layer, into an output image in output device coordinates.
Claim 18
Dependent Claim
42 words
The method of claim 14 wherein the step of computing the photometric distortion includes sampling lighting at characteristic points of the scene element at the initial frame and the current frame, and determining the change in color values at the characteristic points.
Claim 19
Dependent Claim
35 words
The method of claim 14 wherein the step of computing the photometric distortion includes determining change in position of a light source relative to the scene element from the initial frame to the current frame.
Claim 20
Dependent Claim
36 words
The method of claim 16 wherein the step of computing the photometric distortion includes determining angular change in position of a light source relative to the scene element from the initial frame to the current frame.
Claim 21
Dependent Claim
35 words
The method of claim 16 wherein the step of computing the photometric distortion includes determining change in distance of a light source relative to the scene element from the initial frame to the current frame.
Claim 22
Dependent Claim
37 words
The method of claim 16 wherein the step of computing the photometric distortion includes determining change in position of a viewpoint of the scene relative to the scene element from the initial frame to the current frame.
Claim 23
Dependent Claim
23 words
The method of claim 14 further including performing a color warp to approximate lighting change from the initial rendering to the current frame.
Claim 24
Dependent Claim
15 words
A computer-readable medium having computer executable instructions for performing the steps recited in claim 14.
Claim 25
Dependent Claim
203 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements, where each scene element includes a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a color warp that approximates change in lighting of a scene element from an initial frame to a current frame in an animation sequence by modifying color values in an image layer representing an initial rendering of the scene element; measuring a difference between warped color values computed by performing the color warp on the initial rendering and color values of the scene element for the current frame to compute photometric distortion; using the photometric distortion to determine whether to re-render the scene element; approximating the change in lighting of the scene element by using the color warp to modify the color values in the initial rendering of the scene element at the initial frame to new color values for the current frame; compositing the image layers, including at least one layer modified using a color warp, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 26
Dependent Claim
203 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements, where each scene element includes a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers; computing a color warp that approximates change in lighting of a scene element from an initial frame to a current frame in an animation sequence by modifying color values in an image layer representing an initial rendering of the scene element; measuring a difference between warped color values computed by performing the color warp on the initial rendering and color values of the scene element for the current frame to compute photometric distortion; using the photometric distortion to determine whether to re-render the scene element; approximating the change in lighting of the scene element by using the color warp to modify the color values in the initial rendering of the scene element at the initial frame to new color values for the current frame; compositing the image layers, including at least one layer modified using a color warp, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 27
Dependent Claim
40 words
The method of claim 22 wherein the step of computing the photometric distortion includes sampling lighting at characteristic points of the scene element and determining how the sampled lighting differs from lighting at warped samples corresponding to the characteristic points.
Claim 28
Dependent Claim
37 words
The method of claim 22 wherein the step of computing the photometric distortion includes determining the change in position of a light source relative to the first scene element from the initial frame to the current frame.
Claim 29
Dependent Claim
21 words
The method of claim 22 wherein the color warp comprises a single color multiplier applied to samples in the initial rendering.
Claim 30
Dependent Claim
47 words
The method of claim 23 wherein the color warp comprises an array of multipliers applied to samples in the initial rendering, where the values of the multipliers in the array vary with location and are based on lighting sampled at the characteristic points of the scene element.
Claim 31
Dependent Claim
30 words
The method of claim 23 wherein the color warp comprises at least one multiplier and at least one offset applied to samples of the initial rendering to approximate lighting changes.
Claim 32
Dependent Claim
15 words
A computer-readable medium having computer executable instructions for performing the steps recited in claim 22.
Claim 33
Dependent Claim
166 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer in an output image comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers, including rendering a scene element to a 2D sprite space at a different resolution than a resolution of the output image; computing a 2D transform that transforms the scene element from the 2D sprite space to the screen space of a display device; computing sampling distortion of the 2D transform; using the sampling distortion to determine whether to render the scene element at a different spatial resolution; using the 2D transform to transform a rendering of the scene element from the 2D sprite space to the screen space; compositing the image layers, including at least one transformed layer, into an output image in screen space to construct each frame in the animation sequence.
Claim 34
Dependent Claim
166 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer in an output image comprising: factoring a scene into scene elements, where the scene elements include a 3D object or set of 3D objects in the scene; independently rendering the scene elements to separate image layers, including rendering a scene element to a 2D sprite space at a different resolution than a resolution of the output image; computing a 2D transform that transforms the scene element from the 2D sprite space to the screen space of a display device; computing sampling distortion of the 2D transform; using the sampling distortion to determine whether to render the scene element at a different spatial resolution; using the 2D transform to transform a rendering of the scene element from the 2D sprite space to the screen space; compositing the image layers, including at least one transformed layer, into an output image in screen space to construct each frame in the animation sequence.
Claim 35
Dependent Claim
140 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing sampling distortion, photometric distortion, geometric distortion and visibility distortion of a transformed image layer representing the scene element; using the sampling distortion, photometric distortion, geometric distortion and visibility distortion to determine whether to re-render the scene element; approximating motion of the scene element by transforming an initial rendering of the scene element at the initial frame to a new position in the current frame; compositing the image layers into a output image in output device coordinates to construct frames in the animation sequence.
Claim 36
Dependent Claim
140 words
In a real time graphics rendering pipeline, a method for measuring fidelity of a transformed image layer comprising: factoring a scene into scene elements; independently rendering the scene elements to separate image layers; computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing sampling distortion, photometric distortion, geometric distortion and visibility distortion of a transformed image layer representing the scene element; using the sampling distortion, photometric distortion, geometric distortion and visibility distortion to determine whether to re-render the scene element; approximating motion of the scene element by transforming an initial rendering of the scene element at the initial frame to a new position in the current frame; compositing the image layers into a output image in output device coordinates to construct frames in the animation sequence.
Claim 37
Independent Claim
63 words
A method for measuring fidelity of a transformed image layer, comprising: computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing a sampling distortion measure of the 2D transform; and using the sampling distortion measure to determine accuracy of approximating motion of the scene element with the 2D transform.
Claim 38
Independent Claim
63 words
A method for measuring fidelity of a transformed image layer, comprising: computing a 2D transform that approximates motion of a scene element from an initial frame to a current frame in an animation sequence; computing a sampling distortion measure of the 2D transform; and using the sampling distortion measure to determine accuracy of approximating motion of the scene element with the 2D transform.
Claim 39
Dependent Claim
65 words
The method of claim 31 further including: approximating motion of the scene element by using the 2D transform to transform the initial rendering of the scene element at the initial frame to a new position in the current frame; and compositing image layers, including at least one transformed layer, into an output image in output device coordinates to construct each frame in the animation sequence.
Claim 40
Dependent Claim
22 words
The method of claim 31 wherein the step of computing the sampling distortion measure includes computing singular values for the 2D transform.
Claim 41
Dependent Claim
38 words
The method of claim 31 wherein the 2D transform is an affine transform that is capable of computing translation, rotation, scaling and shear of an image layer to approximate motion of object geometry depicted in the image layer.
Claim 42
Dependent Claim
26 words
The method of claim 31 including: computing sampling distortion for image samples in a variety of locations across the initial rendering of the first scene element.
Claim 43
Dependent Claim
14 words
A computer readable medium including computer-executable instructions for performing the steps of claim 31.
Claim 44
Independent Claim
37 words
A method for measuring fidelity between instances of a scene element comprising: computing a visibility distortion measure of the scene element; and using the visibility distortion measure to determine distortion of an approximation of the scene element.
Claim 45
Independent Claim
37 words
A method for measuring fidelity between instances of a scene element comprising: computing a visibility distortion measure of the scene element; and using the visibility distortion measure to determine distortion of an approximation of the scene element.
Claim 46
Dependent Claim
38 words
The method of claim 37 wherein the step of computing the visibility distortion measure includes: determining transitions between back-facing to front-facing surfaces or between front-facing to back-facing surfaces of characteristic geometry of the instances of the scene element.
Claim 47
Dependent Claim
42 words
The method of claim 37 wherein the step of computing the visibility distortion measure includes: determining whether a clipped point of the scene element intersects with screen boundaries of a display device on which a rendering of the scene element is displayed.
Claim 48
Dependent Claim
50 words
The method of claim 37 wherein the step of computing the visibility distortion measure includes: determining transitions between front-facing to back-facing surfaces or between back-facing to front-facing surfaces of characteristic polygons of the instances of the scene element and weighting the transitions by screen area occupied by the characteristic polygons.
Claim 49
Dependent Claim
35 words
The method of claim 37 further including: approximating motion of the scene element by using a 2D transform to transform an initial rendering of a first instance of the scene element to a new position.
Claim 50
Dependent Claim
77 words
The method of claim 37 further including: clipping characteristic geometry of the scene element to an expanded viewing frustum that extends beyond screen boundaries of an output image; tracking position of a clipped point of the characteristic geometry to determine whether the clipped point, when projected to the viewing frustum, is located within the screen boundaries; determining whether to re-render the scene element when the clipped point corresponding to the scene element is within the screen boundaries.
Claim 51
Dependent Claim
15 words
A computer readable medium having computer-executable instructions for performing the steps recited in claim 37.
Claim 52
Independent Claim
37 words
A method for measuring fidelity between instances of a scene element comprising: computing a photometric distortion measure of the scene element; and using the photometric distortion measure to determine distortion of an approximation of the scene element.
Claim 53
Independent Claim
37 words
A method for measuring fidelity between instances of a scene element comprising: computing a photometric distortion measure of the scene element; and using the photometric distortion measure to determine distortion of an approximation of the scene element.
Claim 54
Dependent Claim
21 words
The method of claim 44 further including: using the photometric distortion measure to determine whether to re-render the first scene element.
Claim 55
Dependent Claim
31 words
The method of claim 44 further including: to construct frames in the animation sequence, compositing image layers, including at least one transformed layer, into an output image in output device coordinates.
Claim 56
Dependent Claim
47 words
The method of claim 44 wherein the step of computing the photometric distortion measure includes sampling lighting at characteristic points of a first and second instance of the scene element, and determining the change in color values at the characteristic points between the first and second instances.
Claim 57
Dependent Claim
38 words
The method of claim 44 wherein the step of computing the photometric distortion measure includes determining change in position of a light source relative to the scene element between a first and second instance of the scene element.
Claim 58
Dependent Claim
50 words
The method of claim 44 further including: approximating motion of the scene element by using a 2D transform to transform an initial rendering of the scene element to a new position; using the photometric distortion measure to determine distortion of approximating motion of the scene element with the 2D transform.
Claim 59
Dependent Claim
24 words
The method of claim 44 further including performing a color warp to approximate lighting change between first and second instances of the scene element.
Claim 60
Dependent Claim
92 words
The method of claim 44 including: computing a color warp that approximates change in lighting of the scene element between first and second instances of the scene element by modifying color values in an image layer representing an initial rendering of the scene element; measuring a difference between warped color values computed by performing the color warp on the initial rendering and color values of the second instance of the scene element to compute the photometric distortion measure; and using the photometric distortion measure to determine whether to re-render the scene element.
Claim 61
Dependent Claim
38 words
The method of claim 51 further including: approximating the change in lighting of the scene element by using the color warp to modify the color values in the initial rendering of the scene element to new color values.
Claim 62
Dependent Claim
15 words
A computer-readable medium having computer executable instructions for performing the steps recited in claim 44.
Claim 63
Independent Claim
85 words
A method for measuring fidelity of a transformed image layer comprising: computing a color warp that approximates change in lighting of a scene element from an initial frame to a current frame in an animation sequence by modifying color values in an image layer representing an initial rendering of the scene element; and measuring a difference between warped color values computed by performing the color warp on the initial rendering and color values of the scene element for the current frame to compute photometric distortion.
Claim 64
Independent Claim
85 words
A method for measuring fidelity of a transformed image layer comprising: computing a color warp that approximates change in lighting of a scene element from an initial frame to a current frame in an animation sequence by modifying color values in an image layer representing an initial rendering of the scene element; and measuring a difference between warped color values computed by performing the color warp on the initial rendering and color values of the scene element for the current frame to compute photometric distortion.
Claim 65
Dependent Claim
46 words
The method of claim 54 further including: approximating the change in lighting of the scene element by using the color warp to modify the color values in the initial rendering of the scene element at the initial frame to new color values for the current frame.
Claim 66
Dependent Claim
96 words
A graphics rendering system comprising: a preprocessor for computing a sprite transform used to approximate motion of a scene element and for computing the fidelity of a transformed image layer by measuring sampling, lighting, or visibility distortion of the transformed image layer as a result of the sprite transform being applied to a rendering of the scene element; a rendering sub-system for rendering a scene element to an image layer, and for transforming the image layer by applying the sprite transform to the image layer to approximate motion of the scene element in frames of animation.
Claim 67
Dependent Claim
96 words
A graphics rendering system comprising: a preprocessor for computing a sprite transform used to approximate motion of a scene element and for computing the fidelity of a transformed image layer by measuring sampling, lighting, or visibility distortion of the transformed image layer as a result of the sprite transform being applied to a rendering of the scene element; a rendering sub-system for rendering a scene element to an image layer, and for transforming the image layer by applying the sprite transform to the image layer to approximate motion of the scene element in frames of animation.
Claim 68
Dependent Claim
37 words
The graphics rendering system of claim 56 wherein the rendering system is operable to render more than one scene element to separate image layers and is operable to composite the image layers into a final output image.
Claim 69
Independent Claim
61 words
A method for measuring the fidelity of an image layer that is re-used in an animation sequence comprising: computing a visibility or photometric distortion measure of a scene element of the image layer; and using the visibility or the photometric distortion measure to determine the accuracy of re-using the image layer rather than re-rendering the scene element for the image layer.
Claim 70
Independent Claim
61 words
A method for measuring the fidelity of an image layer that is re-used in an animation sequence comprising: computing a visibility or photometric distortion measure of a scene element of the image layer; and using the visibility or the photometric distortion measure to determine the accuracy of re-using the image layer rather than re-rendering the scene element for the image layer.
References Cited (Backward Citations)
US5684935A
Cited
US5808617A
Cited
US5852443A
Cited
US5864342A
Cited
US5880737A
Cited
US5886701A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-05-16 | 2003-11-15 | 2004-05-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-05-16 | 2007-11-15 | 2008-05-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-05-16 | 2011-11-15 | 2012-05-16 | $7700.00 | Window Closed / Lapsed |