US5617334A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Multi-viewpoint digital video coder/decoder and method
Assignee: Columbia University
•
Filing: 1995-07-21
•
Granted: 1997-04-01
•
Expires: 2015-07-21
Abstract
Abstract The multi-viewpoint video encoder disclosed herein comprises a depth estimator, a predictor connected to the depth estimator, and a comparator connected to the predictor. In addition, the multi-viewpoint video encoder has an output, preferably including a multiplexer for multiplexing the first image, the depth map, the second viewpoint vector and the prediction errors into a signal. The multi-viewpoint video encoder also includes a depth map encoder/compressor. The depth map is compressed according to a video compression standard, preferably compatible with the MPEG-2 standard. The multi-viewpoint video encoder further includes a first image encoder. The first image is encoded according to a video coding standard, preferably compatible with the MPEG-2 standard. In this manner, an MPEG-2 monitor can display the first image video without any further modifications. Many of the elements described above are already found in MPEG-2 encoders. Accordingly, a multi-viewpoint video encoder only requires the addition of the depth estimator and the predictor mentioned above. A multi-viewpoint video decoder as well as methods to encode and decode multi-viewpoint video are also disclosed herein.
Inventors
- 👤 Belle L. Tseng
- 👤 Dimitris Anastassiou
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 01:03
CPC Classifications
G; G06; G06T; G06T9/00; G06T9/008; G06T9/004; H; H04; H04N; H04N13/00; H04N13/10; H04N13/106; H04N13/161; H04N19/00; H04N19/50; H04N19/597; H04N13/189; H04N13/194; H04N13/20; H04N13/204; H04N13/239; H04N13/243; H04N13/286; H04N13/296; H04N13/30; H04N2013/0074; H04N2013/0081; H04N2013/0096
Claims Analysis
Total Claims: 55
Claim 1
Independent Claim
22 words
A multi-viewpoint video encoder comprising: a depth estimator; a predictor connected to the depth estimator; and a comparator connected to the predictor.
Claim 2
Dependent Claim
22 words
A multi-viewpoint video encoder comprising: a depth estimator; a predictor connected to the depth estimator; and a comparator connected to the predictor.
Claim 3
Dependent Claim
11 words
The multi-viewpoint video encoder of claim 1, further comprising an output.
Claim 4
Dependent Claim
30 words
The multi-viewpoint video encoder of claim 2, wherein the output further comprises a multiplexer for multiplexing the first image, the depth map, the second viewpoint vector and the prediction errors.
Claim 5
Dependent Claim
13 words
The multi-viewpoint video encoder of claim 1, further comprising a depth map compressor.
Claim 6
Dependent Claim
19 words
The multi-viewpoint video encoder of claim 4, wherein the depth map is compressed according to a video compression standard.
Claim 7
Dependent Claim
18 words
The multi-viewpoint video encoder of claim 5, wherein the video compression standard is compatible with the MPEG-2 standard.
Claim 8
Dependent Claim
13 words
The multi-viewpoint video encoder of claim 1, further comprising a first image encoder.
Claim 9
Dependent Claim
22 words
The multi-viewpoint video encoder of claim 7, wherein the first image encoder encodes the first image according to a video coding standard.
Claim 10
Dependent Claim
18 words
The multi-viewpoint video encoder of claim 8, wherein the video coding standard is compatible with the MPEG-2 standard.
Claim 11
Dependent Claim
28 words
A multi-viewpoint video encoder comprising: an MPEG-2 encoder; a depth estimator connected to the MPEG-2 encoder; and a predictor connected to the depth estimator and the MPEG-2 encoder.
Claim 12
Dependent Claim
28 words
A multi-viewpoint video encoder comprising: an MPEG-2 encoder; a depth estimator connected to the MPEG-2 encoder; and a predictor connected to the depth estimator and the MPEG-2 encoder.
Claim 13
Dependent Claim
20 words
The multi-viewpoint video encoder of claim 10, wherein the predictor further comprises a manipulator, a combiner, and a vector selector.
Claim 14
Dependent Claim
23 words
A multi-viewpoint video decoder comprising: a receiver; a predictor connected to the receiver; and a reconstructor connected to the predictor and the receiver.
Claim 15
Dependent Claim
23 words
A multi-viewpoint video decoder comprising: a receiver; a predictor connected to the receiver; and a reconstructor connected to the predictor and the receiver.
Claim 16
Dependent Claim
17 words
The multi-viewpoint video decoder of claim 12, wherein the predictor further comprises a manipulator and a combiner.
Claim 17
Dependent Claim
13 words
The multi-viewpoint video decoder of claim 12, further comprising a depth map decompressor.
Claim 18
Dependent Claim
16 words
A multi-viewpoint video decoder comprising: an MPEG-2 decoder; and a predictor connected to the MPEG-2 decoder.
Claim 19
Dependent Claim
16 words
A multi-viewpoint video decoder comprising: an MPEG-2 decoder; and a predictor connected to the MPEG-2 decoder.
Claim 20
Dependent Claim
17 words
The multi-viewpoint video decoder of claim 15, wherein the predictor further comprises a manipulator and a combiner.
Claim 21
Dependent Claim
14 words
A multi-viewpoint video decoder comprising: a receiver; and a predictor connected to the receiver.
Claim 22
Dependent Claim
14 words
A multi-viewpoint video decoder comprising: a receiver; and a predictor connected to the receiver.
Claim 23
Dependent Claim
16 words
The multi-video decoder of claim 17, wherein the predictor further comprises a manipulator and a combiner.
Claim 24
Dependent Claim
20 words
The multi-viewpoint video decoder of claim 17, further comprising a depth map decompressor connected between the predictor and the receiver.
Claim 25
Dependent Claim
14 words
The multi-viewpoint video decoder of claim 17, further comprising a desired viewpoint vector input.
Claim 26
Dependent Claim
15 words
The multi-viewpoint video decoder of claim 17, further comprising: a constructor connected to the predictor.
Claim 27
Dependent Claim
13 words
The multi-viewpoint video decoder of claim 21, the constructor further comprising a memory.
Claim 28
Dependent Claim
16 words
A multi-viewpoint video decoder comprising: an MPEG-2 decoder; and a predictor connected to the MPEG-2 decoder.
Claim 29
Dependent Claim
16 words
A multi-viewpoint video decoder comprising: an MPEG-2 decoder; and a predictor connected to the MPEG-2 decoder.
Claim 30
Dependent Claim
16 words
The multi-video decoder of claim 23, wherein the predictor further comprises a manipulator and a combiner.
Claim 31
Dependent Claim
15 words
The multi-viewpoint video decoder of claim 23, further comprising a constructor connected to the predictor.
Claim 32
Dependent Claim
14 words
The multi-viewpoint video decoder of claim 25, further comprising a desired viewpoint vector input.
Claim 33
Independent Claim
75 words
A method for encoding multi-viewpoint video, comprising the steps of: selecting a first image having a first viewpoint vector; forming a depth map for the first image; selecting a second image having a second viewpoint vector; predicting a predicted second image having the second viewpoint vector by manipulating the first image and the depth map to reflect the second viewpoint vector; and calculating prediction errors for reconstructing the second image from the predicted second image.
Claim 34
Independent Claim
75 words
A method for encoding multi-viewpoint video, comprising the steps of: selecting a first image having a first viewpoint vector; forming a depth map for the first image; selecting a second image having a second viewpoint vector; predicting a predicted second image having the second viewpoint vector by manipulating the first image and the depth map to reflect the second viewpoint vector; and calculating prediction errors for reconstructing the second image from the predicted second image.
Claim 35
Dependent Claim
29 words
The method of encoding multi-viewpoint video of claim 27, further comprising the step of transmitting the first image, the depth map, the second viewpoint vector and the prediction errors.
Claim 36
Dependent Claim
32 words
The method of encoding multi-viewpoint video of claim 28, wherein the transmission step comprises multiplexing the first image, the depth map, the second viewpoint vector and the prediction errors into a signal.
Claim 37
Dependent Claim
25 words
The method of encoding multi-viewpoint video of claim 27, wherein the prediction errors calculation step comprises comparing the second image and the predicted second image.
Claim 38
Dependent Claim
18 words
The method of encoding multi-viewpoint video of claim 27, further comprising the step of compressing the depth map.
Claim 39
Dependent Claim
23 words
The method of encoding multi-viewpoint video of claim 31, wherein the depth map compression step is performed according to a video compression standard.
Claim 40
Dependent Claim
20 words
The method of encoding multi-viewpoint video of claim 32, wherein the video compression standard is compatible with the MPEG-2 standard.
Claim 41
Dependent Claim
18 words
The method of encoding multi-viewpoint video of claim 27, further comprising the step of encoding the first image.
Claim 42
Dependent Claim
22 words
The method of encoding multi-viewpoint video of claim 34, wherein the image encoding step is performed according to a video coding standard.
Claim 43
Dependent Claim
20 words
The method of encoding multi-viewpoint video of claim 35, wherein the video coding standard is compatible with the MPEG-2 standard.
Claim 44
Independent Claim
73 words
A method for decoding multi-viewpoint video, comprising the steps of: receiving a first image having a first viewpoint, a depth map, a second viewpoint vector and prediction errors; forming a predicted second image having the second viewpoint vector by manipulating the first image and the depth map to reflect the second viewpoint vector; and reconstructing a second image having the second viewpoint vector by combining the prediction errors and the predicted second image.
Claim 45
Independent Claim
73 words
A method for decoding multi-viewpoint video, comprising the steps of: receiving a first image having a first viewpoint, a depth map, a second viewpoint vector and prediction errors; forming a predicted second image having the second viewpoint vector by manipulating the first image and the depth map to reflect the second viewpoint vector; and reconstructing a second image having the second viewpoint vector by combining the prediction errors and the predicted second image.
Claim 46
Independent Claim
55 words
A method for decoding multi-viewpoint video, comprising the steps of: receiving a first image having a first viewpoint vector, a depth map, a second viewpoint vector and prediction errors; and constructing a predicted second image having a desired viewpoint vector by manipulating the first image and the depth map to reflect the desired viewpoint vector.
Claim 47
Independent Claim
55 words
A method for decoding multi-viewpoint video, comprising the steps of: receiving a first image having a first viewpoint vector, a depth map, a second viewpoint vector and prediction errors; and constructing a predicted second image having a desired viewpoint vector by manipulating the first image and the depth map to reflect the desired viewpoint vector.
Claim 48
Dependent Claim
19 words
The method of decoding multi-viewpoint video of claim 38, further comprising the step of obtaining the desired viewpoint vector.
Claim 49
Dependent Claim
18 words
The method of decoding multi-viewpoint video of claim 38, further comprising the step of decompressing the depth map.
Claim 50
Dependent Claim
48 words
The method of decoding multi-viewpoint video of claim 38, further comprising the steps of: constructing a second image having the desired viewpoint vector by combining a first stored mesh, a second stored mesh, a first stored reconstructed image, a second stored reconstructed image, and the predicted second image.
Claim 51
Dependent Claim
27 words
The method of decoding multi-viewpoint video of claim 41, wherein the first stored reconstructed image is a nearest past stored reconstructed image having the desired viewpoint vector.
Claim 52
Dependent Claim
26 words
The method of decoding multi-viewpoint video of claim 42, wherein the first stored mesh is a stored mesh respective to the nearest past stored reconstructed image.
Claim 53
Dependent Claim
27 words
The method of decoding multi-viewpoint video of claim 41, wherein the second stored reconstructed image is a nearest stored future reconstructed image having the desired viewpoint vector.
Claim 54
Dependent Claim
26 words
The method of decoding multi-viewpoint video of claim 44, wherein the second stored mesh is a stored mesh respective to the nearest stored future reconstructed image.
Claim 55
Dependent Claim
83 words
The method of decoding multi-viewpoint video of claim 41, wherein the second image construction step further comprises the steps of: combining the first stored mesh and the first stored reconstructed image to form a first mesh image; combining the second stored mesh and the second stored reconstructed image to form a second mesh image; warping the first and second mesh images to form an intermediate mesh image; and constructing the second image by combining the intermediate mesh image with the predicted second image.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2000-03-31 | 2000-09-30 | 2001-04-01 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2004-03-31 | 2004-09-30 | 2005-04-01 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2008-03-31 | 2008-09-30 | 2009-04-01 | $7700.00 | Window Closed / Lapsed |