EP0883300A2
Expired / Lapsed
EP
⏱️ 0.0 years remaining
Temporal and spatial scaleable coding for video object planes
Assignee: Motorola Mobility LLC
•
Filing: 1998-05-26
•
Granted: —
•
Expires: 2018-05-26
Abstract
Abstract Temporal and spatial scaling of video images including video object planes (VOPs) (117, 118, 119, 405, 415, 420, 430, 520, 522, 524, 526, 532, 542, 705, 730, 750, 760, 780, 790, 805, 815, 820, 830, 850, 860, 880, 890) in an input digital video sequence is provided. Coding efficiency is improved by adaptively compressing scaled field mode video. Upsampled VOPs (450, 490, 522, 542, 750, 790) in the enhancement layer are reordered to provide a greater correlation with the input video sequence based on a linear criteria. The resulting residue is coded using a spatial transformation such as the DCT. A motion compensation scheme is used for coding enhancement layer VOPs (450, 460, 480, 490, 522, 524, 526, 542, 750, 760, 780, 790, 850, 860, 880, 890) by scaling motion vectors which have already been determined for the base layer VOPs (405, 415, 420, 430, 520, 532, 705, 730, 805, 815, 820, 830). A reduced search area whose center is defined by the scaled motion vectors is provided. The motion compensation scheme is suitable for use with scaled frame mode or field mode video. Various processor configurations achieve particular scaleable coding results. Applications of scaleable coding include stereoscopic video, picture-in-picture, preview access channels, and ATM communications.
Inventors
- 👤 Xuemin Chen
- 👤 Ajay Luthra
- 👤 Ganesh Rajan
- 👤 Mandayam Narasimhan
Bibliographic Details
Application Number
—
Jurisdiction
EP
Data Source
google_patents
Last Synchronized
2026-11-10 01:46
CPC Classifications
H; H04; H04N; H04N19/00; H04N19/20; G; G06; G06T; G06T3/00; G06T3/40; H04N13/00; H04N13/10; H04N13/106; H04N13/161; H04N13/194; H04N19/29; H04N19/50; H04N19/503; H04N19/51; H04N19/577; H04N19/597; H04N7/00; H04N7/24; H04N7/52; H04N13/15; H04N13/167; H04N13/189; H04N13/20; H04N13/286; H04N19/30; H04N19/31; H04N19/33; H04N2013/0074; H04N2013/0085; H04N2013/0092
Claims Analysis
Total Claims: 45
Claim 1
Independent Claim
131 words
A method for scaling an input video sequence
comprising video object planes (VOPs) for communication
in a corresponding base layer and enhancement layer,
said VOPs in said input video sequence having an
associated spatial resolution and temporal resolution,
comprising the steps of: downsampling pixel data of a first particular one
of said VOPs of said input video sequence to provide a
first base layer VOP having a reduced spatial
resolution; upsampling pixel data of at least a portion of said
first base layer VOP to provide a first upsampled VOP in
said enhancement layer; and differentially encoding said first upsampled VOP
using said first particular one of said VOPs of said
input video sequence for communication in said
enhancement layer at a temporal position corresponding
to said first base layer VOP.
Claim 2
Dependent Claim
93 words
The method of claim 1, wherein said VOPs in
said input video sequence are field mode VOPs, and said
differentially encoding step comprises the further steps
of: reordering lines of said pixel data of said first
upsampled VOP in a field mode if said lines of pixel
data meet a reordering criteria; then determining a residue according to a difference
between pixel data of said first upsampled VOP and pixel
data of said first particular one of said VOPs of said
input video sequence; and spatially transforming said residue to provide
transform coefficients.
Claim 3
Dependent Claim
48 words
The method of claim 2, wherein: said lines of pixel data of said first upsampled
VOP meet said reordering criteria when a sum of
differences of luminance values of opposite-field lines
is greater than a sum of differences of luminance data
of same-field lines and a bias term.
Claim 4
Independent Claim
124 words
The method of any of the preceding claims,
comprising the further steps of: downsampling pixel data of a second particular one
of said VOPs of said input video sequence to provide a
second base layer VOP having a reduced spatial
resolution; upsampling pixel data of at least a portion of said
second base layer VOP to provide a second upsampled VOP
in said enhancement layer which corresponds to said
first upsampled VOP; using at least one of said first and second base
layer VOPs to predict an intermediate VOP corresponding
to said first and second upsampled VOPs; and encoding said intermediate VOP for communication in
said enhancement layer at a temporal position which is
intermediate to that of said first and second upsampled
VOPs.
Claim 5
Dependent Claim
65 words
The method of claim 4, wherein: said enhancement layer has a higher temporal
resolution than said base layer; and said base and enhancement layer are adapted to
provide at least one of: (a) a picture-in-picture (PIP) capability wherein a
PIP image is carried in said base layer, and (b) a preview access channel capability wherein a
preview access image is carried in said base layer.
Claim 6
Independent Claim
36 words
The method of any of the preceding claims,
wherein: said base layer is adapted to carry higher
priority, lower bit rate data, and said enhancement
layer is adapted to carry lower priority, higher bit
rate data.
Claim 7
Independent Claim
131 words
A method for scaling an input video sequence
comprising video object planes (VOPs) for communication
in a corresponding base layer and enhancement layer,
said VOPs in said input video sequence having an
associated spatial resolution and temporal resolution,
comprising the steps of: providing a first particular one of said VOPs of
said input video sequence for communication in said base
layer as a first base layer VOP. downsampling pixel data of at least a portion of
said first base layer VOP for communication in said
enhancement layer as a first downsampled VOP at a
temporal position corresponding to said first base layer
VOP; downsampling corresponding pixel data of said first
particular one of said VOPs to provide a comparison VOP;
and differentially encoding said first downsampled VOP
using said comparison VOP.
Claim 8
Dependent Claim
62 words
The method of claim 7, comprising the further
steps of: differentially encoding said first base layer VOP
using said first particular one of said VOPs by: determining a residue according to a difference
between pixel data of said first base layer VOP and
pixel data of said first particular one of said VOPs; and spatially transforming said residue to provide
transform coefficients.
Claim 9
Dependent Claim
61 words
The method of claim 8, wherein said VOPs in
said input video sequence are field mode VOPs, and said
first base layer VOP is differentially encoded by the
steps of: reordering lines of said pixel data of said first
base layer VOP in a field mode prior to said determining
step if said lines of pixel data meet a reordering
criteria.
Claim 10
Dependent Claim
49 words
The method of claim 9, wherein: said lines of pixel data of said first base layer
VOP meet said reordering criteria when a sum of
differences of luminance values of opposite-field lines
is greater than a sum of differences of luminance data
of same-field lines and a bias term.
Claim 11
Independent Claim
154 words
The method of any of claims 7 to 10,
comprising the further steps of: providing a second particular one of said VOPs of
said input video sequence for communication in said base
layer as a second base layer VOP; downsampling pixel data of at least a portion of
said second base layer VOP for communication in said
enhancement layer as a second downsampled VOP at a
temporal position corresponding to said second base
layer VOP; downsampling corresponding pixel data of said
second particular one of said VOPs to provide a
comparison VOP; differentially encoding said second downsampled VOP
using said Comparison VOP; using at least one of said first and second base
layer VOPs to predict an intermediate VOP corresponding
to said first and second downsampled VOPs; and encoding said intermediate VOP for communication in
said enhancement layer at a temporal position which is
intermediate to that of said first and second upsampled
VOPs.
Claim 12
Independent Claim
43 words
The method of any of claims 7 to 11, wherein: the base and enhancement layers are adapted to
provide a stereoscopic video capability in which image
data in the enhancement layer has a lower spatial
resolution than image data in the base layer.
Claim 13
Independent Claim
133 words
A method for coding a bi-directionally
predicted video object plane (B-VOP), comprising the
steps of: scaling an input video sequence comprising video
object planes (VOPs) for communication in a
corresponding base layer and enhancement layer; providing first and second base layer VOPs in said
base layer which correspond to said input video sequence
VOPs; said second base layer VOP being predicted from
said first base layer VOP according to a motion vector
MV p ; providing said B-VOP in said enhancement layer at a
temporal position which is intermediate to that of said
first and second base layer VOPs; and encoding said B-VOP using at least one of: (a) a forward motion vector MV f and (b) a backward motion vector MV B ,
obtained by scaling said motion vector MV p .
Claim 14
Dependent Claim
108 words
The method of claim 13, wherein: a temporal distance TR p separates said first and
second base layer VOPs; a temporal distance TR B separates said first base
layer VOP and said B-VOP; m/n is a ratio of the spatial resolution of the
first and second base layer VOPs to the spatial
resolution of the B-VOP; and at least one of: (a) said forward motion vector MV f is determined
according to the relationship MV f =(m/n)·TR B ·MV p /TR p ; and (b) said backward motion vector MV b is determined
according to the relationship MV b =(m/n)·(TR B -TR p )·MV p /TR p .
Claim 15
Dependent Claim
67 words
The method of claim 13 or 14, comprising the
further step of: encoding said B-VOP using at least one of: (a) a search region of said first base layer VOP
whose center is determined according to said forward
motion vector MV f ; and (b) a search region of said second base layer VOP
whose center is determined according to said backward
motion vector MV B .
Claim 16
Independent Claim
181 words
A method for recovering an input video
sequence comprising video object planes (VOPs) which was
scaled and communicated in a corresponding base layer
and enhancement layer, said VOPs in said input video
sequence having an associated spatial resolution and
temporal resolution, wherein: pixel data of a first particular one of said VOPs
of said input video sequence is downsampled and carried
as a first base layer VOP having a reduced spatial
resolution; pixel data of at least a portion of said first base
layer VOP is upsampled and carried as a first upsampled
VOP in said enhancement layer at a temporal position
corresponding to said first base layer VOP; and said first upsampled VOP is differentially encoded
using said first particular one of said VOPs of said
input video sequence; said method comprising the steps of: upsampling said pixel data of said first base layer
VOP to restore said associated spatial resolution; and processing said first upsampled VOP and said first
base layer VOP with said restored associated spatial
resolution to provide an output video signal with said
associated spatial resolution.
Claim 17
Dependent Claim
92 words
The method of claim 16, wherein: said VOPs in said input video sequence are field
mode VOPs; and said first upsampled VOP is differentially encoded
by reordering lines of said pixel data of said first
upsampled VOP in a field mode if said lines of pixel
data meet a reordering criteria, then determining a
residue according to a difference between pixel data of
said first upsampled VOP and pixel data of said first
particular one of said VOPs of said input video
sequence, and spatially transforming said residue to
provide transform coefficients.
Claim 18
Dependent Claim
48 words
The method of claim 17, wherein: said lines of pixel data of said first upsampled
VOP meet said reordering criteria when a sum of
differences of luminance values of opposite-field lines
is greater than a sum of differences of luminance data
of same-field lines and a bias term.
Claim 19
Independent Claim
122 words
The method of any of claims 16 to 18, wherein: a second particular one of said VOPs of said input
video sequence is downsampled to provide a second base
layer VOP having a reduced spatial resolution; pixel data of at least a portion of said second
base layer VOP is upsampled to provide a second
upsampled VOP in said enhancement layer which
corresponds to said first upsampled VOP; at least one of said first and second base layer
VOPs is used to predict an intermediate VOP
corresponding to said first and second upsampled VOPs;
and said intermediate VOP is encoded for communication
in said enhancement layer at a temporal position which
is intermediate to that of said first and second
upsampled VOPs.
Claim 20
Dependent Claim
65 words
The method of claim 19, wherein: said enhancement layer has a higher temporal
resolution than said base layer; and said base and enhancement layer are adapted to
provide at least one of: (a) a picture-in-picture (PIP) capability wherein a
PIP image is carried in said base layer, and (b) a preview access channel capability wherein a
preview access image is carried in said base layer.
Claim 21
Independent Claim
37 words
The method of any of claims 16 to 20, wherein: said base layer is adapted to carry higher
priority, lower bit rate data, and said enhancement
layer is adapted to carry lower priority, higher bit
rate data.
Claim 22
Independent Claim
184 words
A method for recovering an input video
sequence comprising video object planes (VOPs) which was
scaled and communicated in a corresponding base layer
and enhancement layer, said VOPs in said input video
sequence having an associated spatial resolution and
temporal resolution, wherein: a first particular one of said VOPs of said input
video sequence is provided in said base layer as a first
base layer VOP; pixel data of at least a portion of said first base
layer VOP is downsampled and carried in said enhancement
layer as a first downsampled VOP at a temporal position
corresponding to said first base layer VOP; corresponding pixel data of said first particular
one of said VOPs is downsampled to provide a comparison
VOP; and said first downsampled VOP is differentially
encoded using said comparison VOP; said method comprising the steps of: upsampling said pixel data of said first
downsampled VOP to restore said associated spatial
resolution; and processing said first enhancement layer VOP with
said restored associated spatial resolution and said
first base layer VOP to provide an output video signal
with said associated spatial resolution.
Claim 23
Dependent Claim
59 words
The method of claim 22, wherein: said first base layer VOP is differentially
encoding using said first particular one of said VOPs by
determining a residue according to a difference between
pixel data of said first base layer VOP and pixel data
of said first particular one of said VOPs, and spatially
transforming said residue to provide transform
coefficients.
Claim 24
Dependent Claim
58 words
The method of claim 23, wherein: said VOPs in said input video sequence are field
mode VOPs, and said first base layer VOP is
differentially encoded by reordering lines of said pixel
data of said first base layer VOP in a field mode prior
to determining said residue if said lines of pixel data
meet a reordering criteria
Claim 25
Dependent Claim
49 words
The method of claim 24, wherein: said lines of pixel data of said first base layer
VOP meet said reordering criteria when a sum of
differences of luminance values of opposite-field lines
is greater than a sum of differences of luminance data
of same-field lines and a bias term.
Claim 26
Independent Claim
154 words
The method of any of claims 22 to 25, wherein: a second particular one of said VOPs of said input
video sequence is provided in said base layer as a
second base layer VOP; pixel data of at least a portion of said second
base layer VOP is downsampled and carried in said
enhancement layer as a second downsampled VOP at a
temporal position corresponding to said second base
layer VOP; corresponding pixel data of said second particular
one of said VOPs is downsampled to provide a comparison
VOP; said second downsampled VOP is differentially
encoded using said comparison VOP; at least one of said first and second base layer
VOPs is used to predict an intermediate VOP
corresponding to said first and second upsampled VOPs;
and said intermediate VOP is encoded for communication
in said enhancement layer at a temporal position which
is intermediate to that of said first and second
upsampled VOPs.
Claim 27
Independent Claim
43 words
The method of any of claims 22 to 26, wherein: said base and enhancement layer are adapted to
provide a stereoscopic video capability in which image
data in said enhancement layer has a lower spatial
resolution than image data in said base layer.
Claim 28
Independent Claim
190 words
A method for recovering an input video
sequence comprising video object planes (VOPs) which was
scaled and communicated in a corresponding base layer
and enhancement layer in a data stream, said VOPs in
said input video sequence having an associated spatial
resolution and temporal resolution, wherein: first and second base layer VOPs are provided in
said base layer which correspond to said input video
sequence VOPs; said second base layer VOP is predicted from said
first base layer VOP according to a motion vector MV p ; a bi-directionally predicted video object plane (B-VOP)
is provided in said enhancement layer at a temporal
position which is intermediate to that of said first and
second base layer VOPs; and said B-VOP is encoded using a forward motion vector
MV f and a backward motion vector MV B which are obtained
by scaling said motion vector MV p ; said method comprising the steps of: recovering said forward motion vector MV f and said
backward motion vector MV B from said data stream; and decoding said B-VOP using said forward motion
vector MV f and said backward motion vector MV B .
Claim 29
Dependent Claim
108 words
The method of claim 28, wherein: a temporal distance TR p separates said first and
second base layer VOPs; a temporal distance TR B separates said first base
layer VOP and said B-VOP; m/n is a ratio of the spatial resolution of the
first and second base layer VOPs to the spatial
resolution of the B-VOP; and at least one of: (a) said forward motion vector MV f is determined
according to the relationship MV f =(m/n)·TR B ·MV p /TR p ; and (b) said backward motion vector MV b is determined
according to the relationship MV b =(m/n)·(TR B -TR p )·MV p /TR p .
Claim 30
Dependent Claim
64 words
The method of claim 28 or 29, wherein: said B-VOP is encoded using at least one of: (a) a search region of said first base layer VOP
whose center is determined according to said forward
motion vector MV f ; and (b) a search region of said second base layer VOP
whose center is determined according to said backward
motion vector MV B .
Claim 31
Dependent Claim
183 words
A decoder apparatus for recovering an input
video sequence comprising video object planes (VOPs)
which was scaled and communicated in a corresponding
base layer and enhancement layer, said VOPs in said
input video sequence having an associated spatial
resolution and temporal resolution, wherein: pixel data of a first particular one of said VOPs
of said input video sequence is downsampled and carried
as a first base layer VOP having a reduced spatial
resolution; pixel data of at least a portion of said first base
layer VOP is upsampled and carried as a first upsampled
VOP in said enhancement layer at a temporal position
corresponding to said first base layer VOP; and said first upsampled VOP is differentially encoded
using said first particular one of said VOPs of said
input video sequence; said apparatus comprising: means for upsampling said pixel data of said first
base layer VOP to restore said associated spatial
resolution; and means for processing said first upsampled VOP and
said first base layer VOP with said restored associated
spatial resolution to provide an output video signal
with said associated spatial resolution.
Claim 32
Dependent Claim
92 words
The apparatus of claim 31, wherein: said VOPs in said input video sequence are field
mode VOPs; and said first upsampled VOP is differentially encoded
by reordering lines of said pixel data of said first
upsampled VOP in a field mode if said lines of pixel
data meet a reordering criteria, then determining a
residue according to a difference between pixel data of
said first upsampled VOP and pixel data of said first
particular one of said VOPs of said input video
sequence, and spatially transforming said residue to
provide transform coefficients.
Claim 33
Dependent Claim
50 words
The apparatus of claim 31 or 32, wherein: said lines of pixel data of said first upsampled
VOP meet said reordering criteria when a sum of
differences of luminance values of opposite-field lines
is greater than a sum of differences of luminance data
of same-field lines and a bias term.
Claim 34
Independent Claim
122 words
The apparatus of any of claims 31 to 33,
wherein: a second particular one of said VOPs of said input
video sequence is downsampled to provide a second base
layer VOP having a reduced spatial resolution; pixel data of at least a portion of said second
base layer VOP is upsampled to provide a second
upsampled VOP in said enhancement layer which
corresponds to said first upsampled VOP; at least one of said first and second base layer
VOPs is used to predict an intermediate VOP
corresponding to said first and second upsampled VOPs;
and said intermediate VOP is encoded for communication
in said enhancement layer at a temporal position which
is intermediate to that of said first and second
upsampled VOPs.
Claim 35
Dependent Claim
65 words
The apparatus of claim 34, wherein: said enhancement layer has a higher temporal
resolution than said base layer; and said base and enhancement layers are adapted to
provide at least one of: (a) a picture-in-picture (PIP) capability wherein a
PIP image is carried in said base layer, and (b) a preview access channel capability wherein a
preview access image is carried in said base layer.
Claim 36
Independent Claim
37 words
The apparatus of any of claims 31 to 35,
wherein: said base layer is adapted to carry higher
priority, lower bit rate data, and said enhancement
layer is adapted to carry lower priority, higher bit
rate data.
Claim 37
Dependent Claim
185 words
A decoder apparatus for recovering an input
video sequence comprising video object planes (VOPs)
which was scaled and communicated in a corresponding
base layer and enhancement layer, said VOPs in said
input video sequence having an associated spatial
resolution and temporal resolution, wherein: a first particular one of said VOPs of said input
video sequence is provided in said base layer as a first
base layer VOP; pixel data of at least a portion of said first base
layer VOP is downsampled and carried in said enhancement
layer as a first downsampled VOP at a temporal position
corresponding to said first base layer VOP; corresponding pixel data of said first particular
one of said VOPs is downsampled to provide a comparison
VOP; and said first downsampled VOP is differentially
encoded using said comparison VOP; said apparatus comprising: means for upsampling said pixel data of said first
downsampled VOP to restore said associated spatial
resolution; and means for processing said first enhancement layer
VOP with said restored spatial resolution and said first
base layer VOP to provide an output video signal with
said associated spatial resolution.
Claim 38
Dependent Claim
54 words
The apparatus of claim 37, wherein: said first downsampled VOP is differentially
encoding by determining a residue according to a
difference between pixel data of said first downsampled
VOP and pixel data of said first particular one of said
VOPs of said input video sequence, and spatially
transforming said residue to provide transform
coefficients.
Claim 39
Dependent Claim
58 words
The apparatus of claim 38, wherein: said VOPs in said input video sequence are field
mode VOPs, and said first base layer VOP is
differentially encoded by reordering lines of said pixel
data of said first base layer VOP in a field mode prior
to determining said residue if said lines of pixel data
meet a reordering criteria.
Claim 40
Dependent Claim
49 words
The apparatus of claim 39, wherein: said lines of pixel data of said first base layer
VOP meet said reordering criteria when a sum of
differences of luminance values of opposite-field lines
is greater than a sum of differences of luminance data
of same-field lines and a bias term.
Claim 41
Independent Claim
122 words
The apparatus of any of claims 37 to 40,
wherein: a second particular one of said VOPs of said input
video sequence is provided for communication in said
base layer as a second base layer VOP; pixel data of at least a portion of said second
base layer VOP is downsampled to provide a second
downsampled VOP in said enhancement layer which
corresponds to said first upsampled VOP; at least one of said first and second base layer
VOPs is used to predict an intermediate VOP
corresponding to said first and second downsampled VOPs;
and said intermediate VOP is encoded for communication
in said enhancement layer at a temporal position which
is intermediate to that of said first and second
upsampled VOPs.
Claim 42
Independent Claim
43 words
The apparatus of any of claims 37 to 41,
wherein: said base and enhancement layer are adapted to
provide a stereoscopic video capability in which image
data in said enhancement layer has a lower spatial
resolution than image data in said base layer.
Claim 43
Dependent Claim
192 words
A decoder apparatus for recovering an input
video sequence comprising video object planes (VOPs)
which was scaled and communicated in a corresponding
base layer and enhancement layer in a data stream, said
VOPs in said input video sequence having an associated
spatial resolution and temporal resolution, wherein: first and second base layer VOPs which correspond
to said input video sequence VOPs are provided in said
base layer; said second base layer VOP is predicted from said
first base layer VOP according to a motion vector MV p ; a bi-directionally predicted video object plane (B-VOP)
is provided in said enhancement layer at a temporal
position which is intermediate to that of said first and
second base layer VOPs; and said B-VOP is encoded using a forward motion vector
MV f and a backward motion vector MV B which are obtained
by scaling said motion vector MV p ; said apparatus comprising: means for recovering said forward motion vector MV f and said backward motion vector MV B from said data
stream; and means for decoding said B-VOP using said forward
motion vector MV f and said backward motion vector MV B .
Claim 44
Dependent Claim
108 words
The apparatus of claim 43, wherein: a temporal distance TR p separates said first and
second base layer VOPs; a temporal distance TR B separates said first base
layer VOP and said B-VOP; m/n is a ratio of the spatial resolution of the
first and second base layer VOPs to the spatial
resolution of the B-VOP; and at least one of: (a) said forward motion vector MV f is determined
according to the relationship MV f =(m/n)·TR B ·MV p /TR p ; and (b) said backward motion vector MV b is determined
according to the relationship MV b =(m/n)·(TR B -TR p )·MV p /TR p .
Claim 45
Dependent Claim
64 words
The apparatus of claim 43 or 44, wherein: said B-VOP is encoded using at least one of: (a) a search region of said first base layer VOP
whose center is determined according to said forward
motion vector MV f ; and (b) a search region of said second base layer VOP
whose center is determined according to said backward
motion vector MV B .
References Cited (Backward Citations)
No citation records indexed.
USPTO Annuity & Maintenance Windows
Maintenance schedule is calculated for granted US utility patents.