US6340994B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
System and method for using temporal gamma and reverse super-resolution to process images for use in digital display systems
Assignee: RPX Corp
•
Filing: 1999-03-26
•
Granted: 2002-01-22
•
Expires: 2019-03-26
Abstract
Abstract An image processing system including a display output processor using Temporal Gamma Processing (TGP) and Reverse Super-resolution (RSR) techniques to process images. TGP assures that the time-related representation of an image is as accurate as possible, and thus, based on a previous frame value and a known transfer function of the display modulation system, adjusts its output values to provide a desired output value during display of a desired frame. RSR performs a superset of the frame rate conversion process for converting between disparate input and output frame rates. RSR, improving display quality when intended display images have apparent resolution higher than can be supported by an image modulator, sequences lower resolution images at higher frame rates to simulate higher resolution outputs. RSR also spatially filters the lower resolution images into transitional images by shifting the pixel matrix of each image frame such that the weighted filter center of each sequenced frame is constant or such that motion artifacts are not generated. Additionally, RSR, to prevent motion artifacts, enables moving the viewed images at the display refresh screen rate and uses motion-adaptive filters and motion tracking information to track image objects.
Inventors
- 👤 Neal Margulis
- 👤 Chad Fogg
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-10 00:44
CPC Classifications
H; H04; H04N; H04N9/00; H04N9/64; G; G09; G09G; G09G3/00; G09G3/20; G09G5/00; G09G5/02; H04N19/00; H04N19/46; H04N19/50; H04N19/503; H04N19/51; H04N19/59; H04N19/60; H04N19/61; H04N19/85; H04N19/86; H04N5/00; H04N5/14; H04N5/21; H04N9/12; H04N9/31; H04N9/3179; H04N9/3185; G09G2320/00; G09G2320/02; G09G2320/0271; G09G2320/0276; G09G2340/00; G09G2340/04; G09G2340/0407; G09G2340/0435; H04N5/74
Claims Analysis
Total Claims: 37
Claim 1
Independent Claim
139 words
An image processing apparatus for processing bitstream information to provide the video stream image data for use in a display system having a display device, comprising: a databus; a display input processor (DIP) coupled to said databus, said DIP comprising a data input connector and a first plurality of processing modules configured to receive said bitstream data through said data input connector and generate DIP outputs therefrom; a display output processor (DOP) coupled to said databus, said DOP comprising a second plurality of processing modules configured to process said DIP outputs and provide DOP outputs therefrom, said second plurality comprising a temporal gamma processing (TGP) module; and a buffer memory, coupled to said databus, configured for storing said DIP outputs and said DOP outputs and for providing stored DOP outputs as video stream image data to said display device.
Claim 2
Dependent Claim
139 words
An image processing apparatus for processing bitstream information to provide the video stream image data for use in a display system having a display device, comprising: a databus; a display input processor (DIP) coupled to said databus, said DIP comprising a data input connector and a first plurality of processing modules configured to receive said bitstream data through said data input connector and generate DIP outputs therefrom; a display output processor (DOP) coupled to said databus, said DOP comprising a second plurality of processing modules configured to process said DIP outputs and provide DOP outputs therefrom, said second plurality comprising a temporal gamma processing (TGP) module; and a buffer memory, coupled to said databus, configured for storing said DIP outputs and said DOP outputs and for providing stored DOP outputs as video stream image data to said display device.
Claim 3
Dependent Claim
39 words
The apparatus of claim 1 wherein said temporal gamma processing module is configured to use a desired intensity value and a previous frame intensity value to determine an output intensity value for each color component of said DIP inputs.
Claim 4
Dependent Claim
40 words
The apparatus of claim 2 further comprising a plurality of look-up tables, the plurality configured such that said desired intensity value determines the depth of said look-up tables and said previous frame intensity value determines the number of look-up tables.
Claim 5
Dependent Claim
27 words
The apparatus of claim 3 wherein said temporal gamma processing module is configured to use at least one of said look-up tables for performing color gamma correction.
Claim 6
Dependent Claim
27 words
The apparatus of claim 3 where in s aid temporal gamma processing module is configured to use a formula for determining the values of said DOP outputs.
Claim 7
Dependent Claim
138 words
An image processing apparatus for processing bitstream information to provide video stream image data for use in a display system having a display device, comprising: a databus; a display input processor (DIP) coupled to said databus, said DIP comprising a data input connector and a first plurality of processing modules configured to receive said bitstream data through said data input connector and generate DIP outputs therefrom; a display output processor (DOP) coupled to said databus, said DOP comprising a second plurality of processing modules configured to process said DIP outputs and provide DOP outputs therefrom, said second plurality comprising a reverse super-resolution (RSR) module; and a buffer memory, coupled to said databus, configured for storing said DIP outputs and said DOP outputs and for providing stored DOP outputs as the video stream image data to said display device.
Claim 8
Dependent Claim
138 words
An image processing apparatus for processing bitstream information to provide video stream image data for use in a display system having a display device, comprising: a databus; a display input processor (DIP) coupled to said databus, said DIP comprising a data input connector and a first plurality of processing modules configured to receive said bitstream data through said data input connector and generate DIP outputs therefrom; a display output processor (DOP) coupled to said databus, said DOP comprising a second plurality of processing modules configured to process said DIP outputs and provide DOP outputs therefrom, said second plurality comprising a reverse super-resolution (RSR) module; and a buffer memory, coupled to said databus, configured for storing said DIP outputs and said DOP outputs and for providing stored DOP outputs as the video stream image data to said display device.
Claim 9
Dependent Claim
46 words
The apparatus of claim 6 wherein said second plurality of processing modules further comprises a display modulator, said display modulator configured to provide modulated data to an image modulator for forming display images on said display device, said image modulator configured for a specific display device.
Claim 10
Dependent Claim
32 words
The apparatus of claim 7 wherein said RSR module is configured to improve display images that have a higher spatial resolution than can be supported by the number of pixels being modulated.
Claim 11
Dependent Claim
22 words
The apparatus of claim 8 wherein each pixel of said display modulator comprises a matrix of emitting elements modulating said display images.
Claim 12
Dependent Claim
24 words
The apparatus of claim 9 wherein said matrix of emitting elements adjusts the position of image outputs during multiple output frames without reloading pixels.
Claim 13
Dependent Claim
18 words
The apparatus of claim 8 wherein said first plurality of processing modules comprises an image reconstruction (IR) module.
Claim 14
Dependent Claim
23 words
The apparatus of claim 11 wherein said IR module is configured for providing an output frame having resolution higher than an input frame.
Claim 15
Dependent Claim
50 words
The apparatus of claim 12 wherein said IR module is further configured to use an input frame to construct intermediate frames having resolution higher than the input frame and to use motion estimation vectors to correlate the intermediate frames adjacent frames and restoring said input frames from said intermediate frames.
Claim 16
Dependent Claim
27 words
The apparatus of claim 13 wherein said IR module is configured to tag said motion estimation vectors with image error terms based on an image matching criteria.
Claim 17
Dependent Claim
33 words
The apparatus of claim 14 wherein said IR module is configured to restore said input frames from said intermediate frames using motion estimation vectors having a low value for said image error terms.
Claim 18
Dependent Claim
51 words
The apparatus of claim 14 wherein said IR module is configured to use input frames configured as pixel blocks having sub-blocks, and is further configured to track motion of said input frames on a sub-block basis if said motion estimation vectors are tagged with a low value for said error terms.
Claim 19
Dependent Claim
20 words
The apparatus of claim 14 wherein said IR module is configured to use motion estimation vectors from an input bitstream.
Claim 20
Dependent Claim
24 words
The apparatus of claim 7 wherein the data modulated by said display modulator is pixel data for forming display images on said display device.
Claim 21
Dependent Claim
91 words
The apparatus of claim 6 wherein said RSR module is further configured to receive said video frequency or image data as a sequence of input frames having a transfer rate of X frames per second and to RSR filter each input frame of said sequence of input frames and to provide therefrom a sequence of Y number of RSR frames having a transfer rate of Z frames per second where Z=X*Y, and wherein the Y number of RSR frames comprise a start RSR frame and at least one subsequent RSR frame.
Claim 22
Independent Claim
84 words
A method for processing bitstream information to form images for use in a display system having a display device, comprising: receiving said bitstream information into a display processor (DIP); processing said received bitstream information to generate DIP outputs; receiving said DIP outputs into a display output processor (DOP); processing said DIP outputs with a temporal gamma processing (TGP) module to generate DOP outputs; directing said DOP outputs to a buffer memory module; and providing images based on said DOP outputs to said display device.
Claim 23
Independent Claim
84 words
A method for processing bitstream information to form images for use in a display system having a display device, comprising: receiving said bitstream information into a display processor (DIP); processing said received bitstream information to generate DIP outputs; receiving said DIP outputs into a display output processor (DOP); processing said DIP outputs with a temporal gamma processing (TGP) module to generate DOP outputs; directing said DOP outputs to a buffer memory module; and providing images based on said DOP outputs to said display device.
Claim 24
Dependent Claim
25 words
The method of claim 20 wherein processing with said TGP module comprises determining a output intensity value for each color component of said DIP outputs.
Claim 25
Dependent Claim
84 words
The method of claim 21 , wherein determining said output intensity value comprises using a desired intensity value and a previous frame intensity value to determine said output intensity value for each color component of said DIP outputs, wherein the using comprises looking up a value within a plurality of look-up tables, the depth of said plurality of look-up tables being a function of the desired intensity value and the number of tables with the plurality being a function of the previous frame intensity.
Claim 26
Dependent Claim
29 words
The method of claim 22 wherein processing with said TGP module comprises using a formula for determining an output intensity value for each color component of said DIP outputs.
Claim 27
Dependent Claim
27 words
The method of claim 21 wherein processing said DIP outputs further comprises processing the outputs with a reverse super-resolution (RSR) module and a display modulator (DM) module.
Claim 28
Independent Claim
100 words
A method for processing bitstream information to for m images for use in a display system having a display device, comprising: receiving said bitstream information into a display processor (DIP); processing said received bitstream information to generate DIP outputs; receiving said DIP outputs into a display output processor (DOP), said DOP comprising a reverse super-resolution (RSR) module and a display modulator (DM) module; processing said DIP outputs with said reverse super-resolution (RSR) module and said display modulator (DM) module; directing said DOP outputs to a buffer memory module; and providing images based on said DOP outputs to said display device.
Claim 29
Independent Claim
100 words
A method for processing bitstream information to for m images for use in a display system having a display device, comprising: receiving said bitstream information into a display processor (DIP); processing said received bitstream information to generate DIP outputs; receiving said DIP outputs into a display output processor (DOP), said DOP comprising a reverse super-resolution (RSR) module and a display modulator (DM) module; processing said DIP outputs with said reverse super-resolution (RSR) module and said display modulator (DM) module; directing said DOP outputs to a buffer memory module; and providing images based on said DOP outputs to said display device.
Claim 30
Dependent Claim
65 words
The method of claim 25 wherein processing said DIP outputs with said RSR module comprises sequencing frames of low resolution data having a first transfer rate, input frames, and converting the each input frame into a number of RSR frames having a second transfer rate higher than the first rate, the number of RSR frames representing an RSR image block having a simulated higher resolution.
Claim 31
Dependent Claim
26 words
The method of claim 26 wherein the second transfer rate is equal to the product of the first transfer rate and the number of RSR frames.
Claim 32
Dependent Claim
50 words
The method of claim 25 wherein processing said DIP outputs with said RSR module to form RSR frames further comprises shifting a pixel matrix that represents each RSR image block by the same pixel amount, where the pixel amount can be either a number of pixels or a pixel fraction.
Claim 33
Dependent Claim
38 words
The method of claim 28 wherein shifting said pixel matrix comprises shifting such that a matrix pixel weighted center of each input frame is constant for each of the number of RSR frames formed from the input frame.
Claim 34
Independent Claim
99 words
A method for temporal gamma processing, comprising: providing a first color intensity value; providing a second color intensity value; providing a number of look-up tables, each table comprising a number of adjusted color intensity values, wherein the number of tables, L, is a function of the second color intensity value and the number of adjusted color intensity values, M, is a function of the first color intensity value,; using the second intensity value to select one of said tables; and using said first color intensity value to select one of said adjusted color intensity values in said selected table.
Claim 35
Independent Claim
99 words
A method for temporal gamma processing, comprising: providing a first color intensity value; providing a second color intensity value; providing a number of look-up tables, each table comprising a number of adjusted color intensity values, wherein the number of tables, L, is a function of the second color intensity value and the number of adjusted color intensity values, M, is a function of the first color intensity value,; using the second intensity value to select one of said tables; and using said first color intensity value to select one of said adjusted color intensity values in said selected table.
Claim 36
Dependent Claim
46 words
The method of claim 30 wherein said first color intensity value comprises a desired output frame color intensity value and said second color intensity value comprises a previous frame color intensity value which corresponds to a previous value of said desired output frame color intensity value.
Claim 37
Dependent Claim
14 words
The method of claim 30 wherein each of said tables further comprises mapping values.
References Cited (Backward Citations)
US4843380A
Cited
US5543819A
Cited
US5092671A
Cited
US5579027A
Cited
US5633956A
Cited
US5369432A
Cited
US5426471A
Cited
US5754260A
Cited
US5666164A
Cited
US5515111A
Cited
US5696848A
Cited
US5559676A
Cited
US5668569A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2005-01-21 | 2005-07-23 | 2006-01-22 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2009-01-21 | 2009-07-23 | 2010-01-22 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2013-01-21 | 2013-07-23 | 2014-01-22 | $7700.00 | Window Closed / Lapsed |