US5155594A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Hierarchical encoding method and apparatus employing background references for efficiently communicating image sequences
Assignee: Polycom LLC
•
Filing: 1991-05-10
•
Granted: 1992-10-13
•
Expires: 2011-05-10
Abstract
Abstract A method and apparatus for transmitting a sequence of image frames by encoding interframe error data features the steps of compiling a spatially decomposed image of a background of the sequence of image frames, spatially decomposing a warped image of a previous frame, and spatially decomposing a new input image. The spatially decomposed input image is compared with the spatially decomposed background image and with the spatially decomposed warped image. An error signal defining the spatially decomposed input image is generated based on these comparisons.
Inventors
- 👤 Jeffrey Bernstein
- 👤 Bernd Girod
- 👤 Xiancheng Yuan
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-26 02:07
CPC Classifications
G; G06; G06T; G06T3/00; G06T3/40; G06T3/4023; H; H04; H04N; H04N19/00; H04N19/10; H04N19/169; H04N19/186; H04N19/50; H04N19/503; H04N19/51; H04N19/53; H04N19/577; H04N19/60; H04N19/61; H04N19/63
Claims Analysis
Total Claims: 26
Claim 1
Independent Claim
93 words
A method for encoding interframe error data, in an image transmission system, for transmitting a sequence of image frames, said method comprising: compiling a spatially decomposed image of a background of the sequence of image frames, spatially decomposing a warped image of a previous frame, spatially decomposing a new input image, comparing said spatially decomposed input image with said spatially decomposed background image and said spatially decomposed warped image, and generating an error signal defining said spatially decomposed input image based on said spatially decomposed warped image and said spatially decomposed background image.
Claim 2
Independent Claim
93 words
A method for encoding interframe error data, in an image transmission system, for transmitting a sequence of image frames, said method comprising: compiling a spatially decomposed image of a background of the sequence of image frames, spatially decomposing a warped image of a previous frame, spatially decomposing a new input image, comparing said spatially decomposed input image with said spatially decomposed background image and said spatially decomposed warped image, and generating an error signal defining said spatially decomposed input image based on said spatially decomposed warped image and said spatially decomposed background image.
Claim 3
Dependent Claim
66 words
The method of claim 1 further comprising the steps of comparing said current image data to previous image data representing a previous image frame, based on said comparison, generating a motion signal representing the displacement of each of a plurality of blocks of said previous image, transmitting said motion signal to said remote receiver to allow said receiver to prepare said warped image pyramid data structure.
Claim 4
Independent Claim
255 words
A method for transmitting a sequence of image frames comprising the steps of: maintaining a background pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said background pyramid data structure representing a background of said sequence of image frames, preparing a warped image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said warped image pyramid data structure representing an estimate of a current image frame based on at least one previous image frame, decimating current input image data representing said current image frame to generate a current image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, for at least one decimation level, based on said warped image pyramid data structure, preparing a first prediction image component for each of a plurality of spatial components at said decimation level, comparing said first prediction image component with a corresponding spatial component of said current image pyramid data structure to form a first error signal, based on said background pyramid data structure, preparing a second prediction image component for each of a plurality a spatial components at said decimation level, comparing said second prediction image component with a corresponding spatial component of said current image pyramid data structure to form a second error signal, and selecting one of said first prediction image component and said second prediction image component, and transmitting electrical signals representing said selection to a receiver.
Claim 5
Independent Claim
255 words
A method for transmitting a sequence of image frames comprising the steps of: maintaining a background pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said background pyramid data structure representing a background of said sequence of image frames, preparing a warped image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said warped image pyramid data structure representing an estimate of a current image frame based on at least one previous image frame, decimating current input image data representing said current image frame to generate a current image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, for at least one decimation level, based on said warped image pyramid data structure, preparing a first prediction image component for each of a plurality of spatial components at said decimation level, comparing said first prediction image component with a corresponding spatial component of said current image pyramid data structure to form a first error signal, based on said background pyramid data structure, preparing a second prediction image component for each of a plurality a spatial components at said decimation level, comparing said second prediction image component with a corresponding spatial component of said current image pyramid data structure to form a second error signal, and selecting one of said first prediction image component and said second prediction image component, and transmitting electrical signals representing said selection to a receiver.
Claim 6
Dependent Claim
58 words
The method of claim 3 wherein selecting one of said first and second prediction components comprises the steps of: for each of said first and second error signals estimating the number of bits required to encode the error signal, and selecting from said first and second error signals, the error signal requiring the smallest estimated number of bits.
Claim 7
Dependent Claim
93 words
The method of claim 4 wherein: estimating the number of bits comprises the steps of: for each error signal, calculating an error energy E equal to the mean squared error signal, determining the number of bits N required to notify the receiver if the associated error signal is selected, and computing a weighted combination of E and N as representative of the number of bits required to encode said associated error signal, and wherein: selecting from said first and second error signals comprises selecting the error signal which yields the lowest weighted combination.
Claim 8
Dependent Claim
38 words
The method of claim 5 wherein transmitting electrical signals representing said selection comprises: transmitting a code indicating which of said first or second error signals was selected, encoding said selected error signal, and transmitting said encoded error signal.
Claim 9
Dependent Claim
76 words
The method of claim 3 further comprising the steps of: for each decimation level, based on a default image pyramid data structure, preparing a third prediction image component for each of a plurality of spatial components at said decimation level, comparing said third prediction image component with a corresponding spatial component of said current image pyramid data structure to form a third error signal, and selecting one of said first, second and third prediction image components.
Claim 10
Dependent Claim
61 words
The method of claim 7 wherein selecting one of said first, second and third prediction components comprises the steps of: for each of said first, second and third error signals, estimating the number of bits required to encode the error signal, and selecting from said first, second and third error signals, the error signal requiring the smallest estimated number of bits.
Claim 11
Dependent Claim
94 words
The method of claim 8 wherein: estimating the number of bits comprises the steps of: for each error signal, calculating an error energy E equal to the mean squared error signal, determining the number of bits N required to notify the receiver if the associated error signal is selected, and computing a weighted combination of E and N as representative of the number of bits required to encode said associated error signal, and wherein: selecting from said first, second and third error signals comprises selecting the error signal which yields the lowest weighted combination.
Claim 12
Dependent Claim
130 words
The method of claim 3 wherein said step of maintaining a background pyramid structure comprises the steps of: preparing a reconstructed image pyramid data structure representative of a receiver image data structure prepared by said receiver in response to said transmitted electrical signals, for each component of said reconstructed image pyramid data structure, determining the number of frames since said component was last changed, generating a weighting factor which is proportional to said determined number of frames, and for each component of said background pyramid data structure, computing a new component value equal to a weighted sum of the background component value and the value of a corresponding component of said reconstructed image pyramid data structure, the relative weight accorded said background component value being determined by said weighting factor.
Claim 13
Dependent Claim
26 words
The method of claim 10 further comprising the step of: dynamically adjusting the weighting factor during actual use of the method with a connected communication channel.
Claim 14
Dependent Claim
103 words
An image encoding device for encoding interframe error data, in an image transmission system, for transmitting a sequence of image frames, comprising: means for compiling a spatially decomposed image of a background of the sequence of image frames, means for spatially decomposing a warped image of a previous frame, means for spatially decomposing a new input image, means for comparing said spatially decomposed input image with said spatially decomposed background image and said spatially decomposed warped image, and means for generating an error signal defining said spatially decomposed input image based on said spatially decomposed warped image and said spatially decomposed background image.
Claim 15
Dependent Claim
103 words
An image encoding device for encoding interframe error data, in an image transmission system, for transmitting a sequence of image frames, comprising: means for compiling a spatially decomposed image of a background of the sequence of image frames, means for spatially decomposing a warped image of a previous frame, means for spatially decomposing a new input image, means for comparing said spatially decomposed input image with said spatially decomposed background image and said spatially decomposed warped image, and means for generating an error signal defining said spatially decomposed input image based on said spatially decomposed warped image and said spatially decomposed background image.
Claim 16
Dependent Claim
246 words
An image encoding device for transmitting a sequence of image frames comprising: means for maintaining a background pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said background pyramid data structure representing a background of said sequence of image frames, means for preparing a warped image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said warped image pyramid data structure representing an estimate of a current image frame based on at least one previous image frame, means for decimating current input image data representing said current image frame to generate a current image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, for at least one decimation level, means for preparing a first prediction image component for each of a plurality of spatial components at said decimation level based on said warped image pyramid data structure, means for comparing said first prediction image component with a corresponding spatial component of said current image pyramid data structure to form a second error signal, means for selecting one of said first prediction image component with a corresponding spatial component of said current image pyramid data structure to form a second error signal, means for selecting one of said first prediction image component and said second prediction image component, and means for transmitting electrical signals representing said selection to a receiver.
Claim 17
Dependent Claim
246 words
An image encoding device for transmitting a sequence of image frames comprising: means for maintaining a background pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said background pyramid data structure representing a background of said sequence of image frames, means for preparing a warped image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, said warped image pyramid data structure representing an estimate of a current image frame based on at least one previous image frame, means for decimating current input image data representing said current image frame to generate a current image pyramid data structure having a plurality of decimation levels, each decimation level comprising at least one spatial component, for at least one decimation level, means for preparing a first prediction image component for each of a plurality of spatial components at said decimation level based on said warped image pyramid data structure, means for comparing said first prediction image component with a corresponding spatial component of said current image pyramid data structure to form a second error signal, means for selecting one of said first prediction image component with a corresponding spatial component of said current image pyramid data structure to form a second error signal, means for selecting one of said first prediction image component and said second prediction image component, and means for transmitting electrical signals representing said selection to a receiver.
Claim 18
Dependent Claim
70 words
The image encoding device of claim 13 further comprising means for comparing said current image data to previous image data representing a previous image frame, based on said comparison, generating a motion signal representing the displacement of each of a plurality of blocks of said previous image, and means for transmitting said motion signal to said remote receiver to allow said receiver to prepare said warped image pyramid data structure.
Claim 19
Dependent Claim
60 words
The image encoding device of claim 13 wherein said means for selecting one of said first and second prediction components comprises: means for estimating the number of bits required to encode each of said first and second error signals, and means for selecting from said first and second error signals, the error signal requiring the smallest estimated number of bits.
Claim 20
Dependent Claim
106 words
The image encoding device of claim 15 wherein said means for estimating the number of bits comprises: means for calculating, for each error signal, an error energy E equal to the mean squared error signal, means for determining the number of bits N required to notify the receiver if the associated error signal is selected, and means for computing a weighted combination of E and N as representative of the number of bits required to encode said associated error signal, and wherein: said means for selecting from said first and second error signals comprises means for selecting the error signal which yields the lowest weighted combination.
Claim 21
Dependent Claim
49 words
The image encoding device of claim 15 wherein said means for transmitting electrical signals representing said selection comprises: means for transmitting a code indicating which of said first or second error signals was selected, means for encoding said selected error signal, and means for transmitting said encoded error signal.
Claim 22
Dependent Claim
80 words
The image encoding device of claim 13 further comprising: for each decimation level, means for preparing a third prediction image component for each of a plurality of spatial components at said decimation level based on a default image pyramid data structure, means for comparing said third prediction image component with a corresponding spatial component of said current image pyramid data structure to form a third error signal, means for selecting one of said first, second and third prediction image components.
Claim 23
Dependent Claim
67 words
The image encoding device of claim 18 wherein said means for selecting one of said first, second and third prediction components comprises: means for estimating, for each of said first, second and third error signals, the number of bits required to encode the error signal, and means for selecting from said first, second and third error signals, the error signal requiring the smallest estimated number of bits.
Claim 24
Dependent Claim
107 words
The image encoding device of claim 19 wherein said means for estimating the number of bits comprises: means for calculating, for each error signal, an error energy E equal to the mean squared error signal, means for determining the number of bits N required to notify the receiver if the associated error signal is selected, and means for computing a weighted combination of E and N as representative of the number of bits required to encode said associated error signal, and wherein said means for selecting from said first, second and third error signals comprises means for selecting the error signal which yields the lowest weighted combination.
Claim 25
Dependent Claim
137 words
The image encoding device of claim 13 wherein said means for maintaining a background pyramid structure comprises: means for preparing a reconstructed image pyramid data structure representative of a receiver image data structure prepared by said receiver in response to said transmitted electrical signals, means for determining, for each component of said reconstructed image pyramid data structure, the number of frames since said component was last changed, means for generating a weighting factor which is proportional to said determined number of frames, and means for computing, for each component of said background pyramid data structure, a new component value equal to a weighted sum of the background component value and the value of a corresponding component of said reconstructed image pyramid data structure, the relative weight accorded said background component value being determined by said weighting factor.
Claim 26
Dependent Claim
27 words
The image encoding device of claim 21 further comprising: means for dynamically adjusting the weighting factor during actual use of the method with a connected communication channel.
References Cited (Backward Citations)
US4802006A
Cited
US4718104A
Cited
US4717957A
Cited
US4703350A
Cited
US4727422A
Cited
US4689673A
Cited
US4794455A
Cited
US4816914A
Cited
US4833535A
Cited
US4864396A
Cited
US4849810A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1995-10-13 | 1996-04-13 | 1996-10-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 1999-10-13 | 2000-04-13 | 2000-10-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2003-10-13 | 2004-04-13 | 2004-10-13 | $7700.00 | Window Closed / Lapsed |