US6148026A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Mesh node coding to enable object based functionalities within a motion compensated transform video coder
Assignee: Sharp Laboratories of America Inc
•
Filing: 1997-12-29
•
Granted: 2000-11-14
•
Expires: 2017-12-29
Abstract
Abstract A method and apparatus for coding video data permits coding of video information with optional, enhanced functionalities. Video data is coded as base layer data and enhancement layer data. The base layer data includes convention motion compensated transform encoded texture and motion vector data. Optional enhancement layer data contains mesh node vector data. Mesh node vector data of the enhancement layer may be predicted based on motion vectors of the base layer. Thus, simple decoders may decode the base layer data and obtain a basic representation of the coded video data. However, more powerful decoders may decode both the base layer and enhanced layer to obtain decoded video permitting functionalities. An embodiment of the present invention provides a back channel that permits a decoder to affect how mesh node coding is performed in the encoder. The decoder may command the encoder to reduce or eliminate encoding of mesh node motion vectors. The back channel finds application in single layer systems and two layer systems.
Inventors
- 👤 Atul Puri
- 👤 A. Murat Tekalp
- 👤 Petrus J. L. Van Beek
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 12:47
CPC Classifications
H; H04; H04N; H04N19/00; H04N19/20; H04N19/29; H04N19/50; H04N19/503; H04N19/51; H04N19/537; H04N19/54
Claims Analysis
Total Claims: 49
Claim 1
Independent Claim
55 words
A video coder that codes video data as coded output, the coded output including block based texture data, block based motion data and mesh node based motion data, the coder responsive to a predetermined command received over a back channel by reducing a number of mesh nodes represented in the mesh node based motion data.
Claim 2
Dependent Claim
55 words
A video coder that codes video data as coded output, the coded output including block based texture data, block based motion data and mesh node based motion data, the coder responsive to a predetermined command received over a back channel by reducing a number of mesh nodes represented in the mesh node based motion data.
Claim 3
Dependent Claim
32 words
The video coder of claim 1, wherein, when the command requests the mesh node motion data to be eliminated, the coder ceases to code video data as mesh node based motion data.
Claim 4
Dependent Claim
15 words
The video coder of claim 1, wherein the video coder is a single layer coder.
Claim 5
Dependent Claim
46 words
The video coder of claim 1, wherein the video coder is a dual layer coder, the block based texture data and block based motion data being includes in coded base layer data and the mesh node based motion data being included in coded enhancement layer data.
Claim 6
Dependent Claim
76 words
A video decoder that decodes coded video data received over a channel, the coded video data including block based texture data, block based motion data and mesh node based motion data, the decoder determining whether the mesh node motion data is necessary for video decoding and, when the mesh node motion data is unnecessary, generating a back channel command representing a request to reduce a number of mesh nodes represented by the node based motion data.
Claim 7
Dependent Claim
76 words
A video decoder that decodes coded video data received over a channel, the coded video data including block based texture data, block based motion data and mesh node based motion data, the decoder determining whether the mesh node motion data is necessary for video decoding and, when the mesh node motion data is unnecessary, generating a back channel command representing a request to reduce a number of mesh nodes represented by the node based motion data.
Claim 8
Dependent Claim
17 words
The video decoder of claim 5, wherein the command requests elimination of the mesh node motion data.
Claim 9
Dependent Claim
15 words
The video decoder of claim 5, wherein the video decoder is a single layer decoder.
Claim 10
Dependent Claim
46 words
The video decoder of claims 5, wherein the video decoder is a dual layer decoder, the block based texture data and block based motion data being included in coded base layer data and the mesh node based motion data being included in coded enhancement layer data.
Claim 11
Dependent Claim
96 words
A video coder that codes input video data, comprising: a base layer encoder that receives the input video data and outputs coded base layer data representative of the input video data, the coded base layer data including block based texture data and block based motion data, an enhancement layer encoder that receives the input video data and outputs coded enhanced layer data including mesh node based motion information, a multiplexer that merges the coded base layer data and coded enhanced layer data into a unitary data stream and outputs the unitary data stream to a channel.
Claim 12
Dependent Claim
96 words
A video coder that codes input video data, comprising: a base layer encoder that receives the input video data and outputs coded base layer data representative of the input video data, the coded base layer data including block based texture data and block based motion data, an enhancement layer encoder that receives the input video data and outputs coded enhanced layer data including mesh node based motion information, a multiplexer that merges the coded base layer data and coded enhanced layer data into a unitary data stream and outputs the unitary data stream to a channel.
Claim 13
Dependent Claim
22 words
The video coder of claim 9, wherein the base layer encoder outputs the block based motion data to the enhanced layer encoder.
Claim 14
Dependent Claim
24 words
The video coder of claim 9, wherein the multiplexer routes contents of a back channel contained within the channel to the enhancement layer encoder.
Claim 15
Dependent Claim
29 words
The video coder of claim 11, wherein responsive to a predetermined command, the enhancement layer encoder reduces a number of mesh nodes represented by the mesh node motion data.
Claim 16
Dependent Claim
18 words
The video coder of claim 11, wherein responsive to a predetermined command, the enhancement layer encoder is disabled.
Claim 17
Dependent Claim
15 words
The video coder of claim 9, wherein the input video data constitutes video object planes.
Claim 18
Dependent Claim
23 words
The video coder of claim 9, wherein a representation of the input video data may be generated from the coded base layer data.
Claim 19
Dependent Claim
128 words
A video decoder that decodes coded video data, the coded video data including block based motion data and block based texture data included in coded base layer data and mesh based motion data included in coded enhancement layer data, the decoder comprising: a base layer decoder that decodes the coded base layer data into decoded video data and motion data, an enhancement layer decoder that decodes the enhancement layer data, a demutliplexer that receives the coded video data from a channel, routes the coded base layer data to the base layer decoder and routes the coded enhancement layer data to the enhancement layer decoder, and a compositor that receives decoded outputs from the base layer encoder and the enhancement layer decoder and generates a synthesized video output therefrom.
Claim 20
Dependent Claim
128 words
A video decoder that decodes coded video data, the coded video data including block based motion data and block based texture data included in coded base layer data and mesh based motion data included in coded enhancement layer data, the decoder comprising: a base layer decoder that decodes the coded base layer data into decoded video data and motion data, an enhancement layer decoder that decodes the enhancement layer data, a demutliplexer that receives the coded video data from a channel, routes the coded base layer data to the base layer decoder and routes the coded enhancement layer data to the enhancement layer decoder, and a compositor that receives decoded outputs from the base layer encoder and the enhancement layer decoder and generates a synthesized video output therefrom.
Claim 21
Dependent Claim
22 words
The video decoder of claim 16, wherein the base layer decoder outputs the block based motion data to the enhanced layer decoder.
Claim 22
Dependent Claim
31 words
The video decoder of claim 16, wherein the compositor generates commands and outputs the commands to the demultiplexer and the demultiplexer places the commands in a back channel in the channel.
Claim 23
Dependent Claim
30 words
The video decoder of claim 18, wherein the compositor generates a command representing a request to reduce a number of mesh nodes represented by the mesh node based motion data.
Claim 24
Dependent Claim
23 words
The video decoder of claim 18, wherein the compositor generates a command representing a request to eliminate the mesh node based motion data.
Claim 25
Dependent Claim
17 words
The video decoder of claim 16, wherein compositor's synthesized video output is generated as video object planes.
Claim 26
Dependent Claim
110 words
A video decoder that decodes coded video data, the coded video data including block based motion data and block based texture data included in coded base layer data and mesh based motion data included in coded enhancement layer data, the decoder comprising: a base layer decoder that decodes the coded base layer data into decoded video data and motion data, a demutliplexer that receives the coded video data from a channel and routes the coded base layer data to the base layer decoder and ignores the coded enhancement layer data, and a compositor that receives a decoded output from the base layer encoder and generates a synthesized video output therefrom.
Claim 27
Dependent Claim
110 words
A video decoder that decodes coded video data, the coded video data including block based motion data and block based texture data included in coded base layer data and mesh based motion data included in coded enhancement layer data, the decoder comprising: a base layer decoder that decodes the coded base layer data into decoded video data and motion data, a demutliplexer that receives the coded video data from a channel and routes the coded base layer data to the base layer decoder and ignores the coded enhancement layer data, and a compositor that receives a decoded output from the base layer encoder and generates a synthesized video output therefrom.
Claim 28
Dependent Claim
128 words
In a two-layer video coder that encodes input video data as coded video data, the coded video data including a base layer of block based texture data and block based motion data and an enhancement layer of mesh node based motion data, a method of coding the enhancement layer data, comprising: generating mesh nodes from the input video data, receiving the block based motion data, for at least one mesh node: identifying a block that is spatially related to the mesh node, predicting a motion vector for the mesh node based upon a motion vector of the spatially related block, comparing an actual motion vector of the mesh node against the predicted motion vector, and encoding a difference between the actual motion vector and the predicted motion vector.
Claim 29
Dependent Claim
128 words
In a two-layer video coder that encodes input video data as coded video data, the coded video data including a base layer of block based texture data and block based motion data and an enhancement layer of mesh node based motion data, a method of coding the enhancement layer data, comprising: generating mesh nodes from the input video data, receiving the block based motion data, for at least one mesh node: identifying a block that is spatially related to the mesh node, predicting a motion vector for the mesh node based upon a motion vector of the spatially related block, comparing an actual motion vector of the mesh node against the predicted motion vector, and encoding a difference between the actual motion vector and the predicted motion vector.
Claim 30
Dependent Claim
52 words
The method of claim 23, wherein the predicting step includes steps of: identifying a motion vector of the spatially related block, identifying motion vectors of blocks neighboring the spatially related block, and predicting the motion vector of the mesh node as a predetermined function of the spatially co-located block and its neighbors.
Claim 31
Dependent Claim
24 words
The method of claim 24, wherein the neighboring blocks include blocks above, below, to the left and to the right of the related block.
Claim 32
Dependent Claim
16 words
The method of claim 24, wherein the neighboring blocks including all blocks circumscribing the related block.
Claim 33
Dependent Claim
28 words
The method of claim 24, wherein the neighboring blocks are the blocks circumscribing the related block and are closest to the mesh node's position within the related block.
Claim 34
Dependent Claim
13 words
The method of claim 24, wherein the predetermined function is an averaging function.
Claim 35
Dependent Claim
14 words
The method of claim 24, wherein the predetermined function is a weighted averaging function.
Claim 36
Dependent Claim
13 words
The method of claim 24, wherein the predetermined function is a median function.
Claim 37
Dependent Claim
17 words
The method of claim 23, wherein a block is an 8 pixel by 8 pixel rectangular area.
Claim 38
Dependent Claim
19 words
The method of claim 23, wherein a block is a macroblock, a 16 pixel by 16 pixel rectangular area.
Claim 39
Dependent Claim
110 words
In a two-layer video decoder that decodes coded video data, the coded video data including a base layer of block based texture data and block based motion data and an enhancement layer of residual mesh node based motion data, a method of decoding the mesh node based motion data, comprising: receiving the coded enhancement layer data, for at least one mesh node: for at least one mesh node: identifying a block that is spatially related to the mesh node, predicting a motion vector for the mesh node based upon a motion vector of the spatially related block, and adding residual motion data of the node to the predicted motion vector.
Claim 40
Dependent Claim
110 words
In a two-layer video decoder that decodes coded video data, the coded video data including a base layer of block based texture data and block based motion data and an enhancement layer of residual mesh node based motion data, a method of decoding the mesh node based motion data, comprising: receiving the coded enhancement layer data, for at least one mesh node: for at least one mesh node: identifying a block that is spatially related to the mesh node, predicting a motion vector for the mesh node based upon a motion vector of the spatially related block, and adding residual motion data of the node to the predicted motion vector.
Claim 41
Dependent Claim
52 words
The method of claim 33, wherein the predicting step includes steps of: identifying a motion vector of the spatially related block, identifying motion vectors of blocks neighboring the spatially related block, and predicting the motion vector of the mesh node as a predetermined function of the spatially co-located block and its neighbors.
Claim 42
Dependent Claim
24 words
The method of claim 34, wherein the neighboring blocks include blocks above, below, to the left and to the right of the related block.
Claim 43
Dependent Claim
16 words
The method of claim 34, wherein the neighboring blocks including all blocks circumscribing the related block.
Claim 44
Dependent Claim
28 words
The method of claim 34, wherein the neighboring blocks are the blocks circumscribing the related block and are closest to the mesh node's position within the related block.
Claim 45
Dependent Claim
13 words
The method of claim 34, wherein the predetermined function is an averaging function.
Claim 46
Dependent Claim
14 words
The method of claim 34, wherein the predetermined function is a weighted averaging function.
Claim 47
Dependent Claim
13 words
The method of claim 34, wherein the predetermined function is a median finction.
Claim 48
Dependent Claim
17 words
The method of claim 33, wherein a block is an 8 pixel by 8 pixel rectangular area.
Claim 49
Dependent Claim
19 words
The method of claim 33, wherein a block is a macroblock, a 16 pixel by 16 pixel rectangular area.
References Cited (Backward Citations)
US5923777A
Cited
US5936671A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-11-14 | 2004-05-15 | 2004-11-14 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-11-14 | 2008-05-15 | 2008-11-14 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-11-14 | 2012-05-15 | 2012-11-14 | $7700.00 | Window Closed / Lapsed |