Joint rate control for stereoscopic video coding

Claim 1

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: determining a complexity measure for each of said channels; wherein said channels comprise successive image frames of said stereoscopic video sequence, including a first channel comprising a base layer, and a second channel comprising an enhancement layer; allocating a number of bits representing said video sequence between said channels according to each channel's complexity; grouping a number of frames of the first and second channels into successive respective groups of pictures (GOPs); and forming successive joint groups of pictures (GOPs) from corresponding GOPs of the first and second channels; wherein: each joint GOP includes the same number of I, P and B pictures from the first channel, and P and B pictures from the second channel; and said allocating step allocates the same target number of bits to each joint GOP.

Claim 2

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: determining a complexity measure for each of said channels; wherein said channels comprise successive image frames of said stereoscopic video sequence, including a first channel comprising a base layer, and a second channel comprising an enhancement layer; allocating a number of bits representing said video sequence between said channels according to each channel's complexity; grouping a number of frames of the first and second channels into successive respective groups of pictures (GOPs); and forming successive joint groups of pictures (GOPs) from corresponding GOPs of the first and second channels; wherein: each joint GOP includes the same number of I, P and B pictures from the first channel, and P and B pictures from the second channel; and said allocating step allocates the same target number of bits to each joint GOP.

Claim 18

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: determining a complexity measure for each of said channels; and allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.

Claim 19

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: determining a complexity measure for each of said channels; and allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.

Claim 24

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: (a) determining a complexity measure for each of said channels; (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; the successive image frames of a first channel comprise I, P and B pictures; the successive image frames of a second channel comprise only P and B pictures; a complexity measure C I is maintained for I pictures; a complexity measure C 1P is maintained for the first channel P pictures; a complexity measure C 1B is maintained for the first channel B pictures; a complexity measure C 2P is maintained for the second channel P pictures; and a complexity measure C 2B is maintained for the second channel B pictures; (c) grouping a number of frames of each channel into a respective group of pictures (GoP) for the channel; and (d) forming a joint GoP from corresponding GoPs of said first and second channels; wherein said allocating step is responsive to a target number of bits (T m ,n) determined as: ##EQU37## where: C m ,n is the complexity measure for the picture type of frame (m,n), m is a channel index and n is a frame index, subscripts I, P and B denote, respectively, I, P and B pictures, subscripts 1 and 2 denote, respectively, the first and second channels, N I , N 1P , N 1B , N 2P and N 2B are, respectively, the number of remaining I pictures, first-channel P pictures, first-channel B pictures, second-channel P pictures and second-channel B pictures in the joint GoP, w denotes a quality balance weighting factor, K denotes a quantization parameter weighting factor, and T r is the remaining number of bits in the joint GoP, defined as T r =T r -R m' ,n', where -- R m' ,n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).

Claim 25

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: (a) determining a complexity measure for each of said channels; (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; the successive image frames of a first channel comprise I, P and B pictures; the successive image frames of a second channel comprise only P and B pictures; a complexity measure C I is maintained for I pictures; a complexity measure C 1P is maintained for the first channel P pictures; a complexity measure C 1B is maintained for the first channel B pictures; a complexity measure C 2P is maintained for the second channel P pictures; and a complexity measure C 2B is maintained for the second channel B pictures; (c) grouping a number of frames of each channel into a respective group of pictures (GoP) for the channel; and (d) forming a joint GoP from corresponding GoPs of said first and second channels; wherein said allocating step is responsive to a target number of bits (T m ,n) determined as: ##EQU37## where: C m ,n is the complexity measure for the picture type of frame (m,n), m is a channel index and n is a frame index, subscripts I, P and B denote, respectively, I, P and B pictures, subscripts 1 and 2 denote, respectively, the first and second channels, N I , N 1P , N 1B , N 2P and N 2B are, respectively, the number of remaining I pictures, first-channel P pictures, first-channel B pictures, second-channel P pictures and second-channel B pictures in the joint GoP, w denotes a quality balance weighting factor, K denotes a quantization parameter weighting factor, and T r is the remaining number of bits in the joint GoP, defined as T r =T r -R m' ,n', where -- R m' ,n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).

Claim 26

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: determining a complexity measure for each of said channels; and allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices; the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel; and the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.

Claim 27

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: determining a complexity measure for each of said channels; and allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices; the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel; and the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.

Claim 31

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: (a) determining a complexity measure for each of said channels; and (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating step is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and (c) controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; said target bit rate is maintained within said lower and upper bounds according to the parameters ##EQU40## the quantity 2N is the number of frames in a joint group of pictures (GOP), ##EQU41## is the number of bits transmitted for channel m during the nth frame period, ##EQU42## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said video decoder; and all of the terms in the lower bound are determinable at the video encoder.

Claim 32

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: (a) determining a complexity measure for each of said channels; and (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating step is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and (c) controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; said target bit rate is maintained within said lower and upper bounds according to the parameters ##EQU40## the quantity 2N is the number of frames in a joint group of pictures (GOP), ##EQU41## is the number of bits transmitted for channel m during the nth frame period, ##EQU42## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said video decoder; and all of the terms in the lower bound are determinable at the video encoder.

Claim 33

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.

Claim 34

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and the complexity (C) of an image frame comprises the product of an average quantization parameter (Q) used for that frame and the number of bits (R) generated when that frame is quantized using the quantization parameter Q.

Claim 35

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; the successive image frames of a first channel comprise I, P and B pictures; the successive image frames of a second channel comprise only P and B pictures; a complexity measure C I is maintained for I pictures; a complexity measure C 1P is maintained for the first channel P pictures; a complexity measure C 1B is maintained for the first channel B pictures; a complexity measure C 2P is maintained for the second channel P pictures; and a complexity measure C 2B is maintained for the second channel B pictures; means for grouping a number of frames of each channel into a respective group of pictures (GoP) for the channel; and means for forming a joint GoP from corresponding GoPs of said first and second channels, said joint GoP comprising 2N frames; wherein said allocating means is responsive to a target number of bits (T m ,n) determined as: ##EQU43## where: C m ,n is the complexity measure for the picture type of frame (m,n), m is a channel index and n is a frame index, subscripts I, P and B denote, respectively, I, P and B pictures, subscripts 1 and 2 denote, respectively, the first and second channels, N I , N 1P , N 1B , N 2P and N 2B are, respectively, the number of remaining I pictures, first-channel P pictures, first-channel B pictures second-channel P pictures and second-channel B pictures in the joint GoP, w denotes a quality balance weighting factor, K denotes a quantization parameter weighting factor, and T r is the remaining number of bits in the joint GoP, defined as T r =T r -R m' ,n', where -- R m' ,n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).

Claim 36

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; the successive image frames of a first channel comprise I, P and B pictures; the successive image frames of a second channel comprise only P and B pictures; a complexity measure C I is maintained for I pictures; a complexity measure C 1P is maintained for the first channel P pictures; a complexity measure C 1B is maintained for the first channel B pictures; a complexity measure C 2P is maintained for the second channel P pictures; and a complexity measure C 2B is maintained for the second channel B pictures; means for grouping a number of frames of each channel into a respective group of pictures (GoP) for the channel; and means for forming a joint GoP from corresponding GoPs of said first and second channels, said joint GoP comprising 2N frames; wherein said allocating means is responsive to a target number of bits (T m ,n) determined as: ##EQU43## where: C m ,n is the complexity measure for the picture type of frame (m,n), m is a channel index and n is a frame index, subscripts I, P and B denote, respectively, I, P and B pictures, subscripts 1 and 2 denote, respectively, the first and second channels, N I , N 1P , N 1B , N 2P and N 2B are, respectively, the number of remaining I pictures, first-channel P pictures, first-channel B pictures second-channel P pictures and second-channel B pictures in the joint GoP, w denotes a quality balance weighting factor, K denotes a quantization parameter weighting factor, and T r is the remaining number of bits in the joint GoP, defined as T r =T r -R m' ,n', where -- R m' ,n' is the number of bits generated for a frame (m',n') coded just before frame (m,n).

Claim 37

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices; the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel; and the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.

Claim 38

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence, each image frame being divided into a plurality of slices; the complexity measure of a channel is based on the complexity (C) of the image frame slices carried by the channel; and the complexity (C) of an image frame slice comprises the product of a quantization parameter (Q) used for that slice and the number of bits (R) generated when that slice is quantized using the quantization parameter Q.

Claim 39

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating means is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and means for controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; said target bit rate is maintained within said lower and upper bounds according to the parameters ##EQU44## the quantity 2N is the number of frames in a joint group of pictures (GOP), ##EQU45## is the number of bits transmitted for channel m during the nth frame period, ##EQU46## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said video decoder; and all of the terms in the lower bound are determinable at the video encoder.

Claim 40

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating means is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and means for controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; said target bit rate is maintained within said lower and upper bounds according to the parameters ##EQU44## the quantity 2N is the number of frames in a joint group of pictures (GOP), ##EQU45## is the number of bits transmitted for channel m during the nth frame period, ##EQU46## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said video decoder; and all of the terms in the lower bound are determinable at the video encoder.

Claim 41

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: (a) determining a complexity measure for each of said channels; and (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating step is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and (c) controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; and said target bit rate is maintained within said lower and upper bounds according to the parameters: ##EQU47## where the quantity R c m ,n is the number of bits transmitted for channel m during the nth frame period, B c m ,n is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said decoder buffer.

Claim 42

A method for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising the steps of: (a) determining a complexity measure for each of said channels; and (b) allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating step is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and (c) controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; and said target bit rate is maintained within said lower and upper bounds according to the parameters: ##EQU47## where the quantity R c m ,n is the number of bits transmitted for channel m during the nth frame period, B c m ,n is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said decoder buffer.

Claim 46

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating means is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and means for controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; and said target bit rate is maintained within said lower and upper bounds according to the parameters: ##EQU48## where the quantity ##EQU49## is the number of bits transmitted for channel m during the nth frame period, ##EQU50## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said decoder buffer.

Claim 47

An apparatus for balancing a quality ratio between a plurality of channels of a stereoscopic video sequence, comprising: means for determining a complexity measure for each of said channels; and means for allocating a number of bits representing said video sequence between said channels according to each channel's complexity; wherein: said channels carry successive image frames of said stereoscopic video sequence; the complexity measure of a channel is based on the complexity (C) of the image frames carried by the channel; and said allocating means is responsive to a target bit rate T m ,n assigned to each frame based on the complexity (C) of the frame, where m is a channel index and n is a frame index; and means for controlling said target bit rate to prevent the underflow and overflow of buffers in a video encoder processing said video image and a video decoder receiving said video image; wherein: said target bit rate is adjusted in accordance with variable lower and upper bounds for each frame; and said target bit rate is maintained within said lower and upper bounds according to the parameters: ##EQU48## where the quantity ##EQU49## is the number of bits transmitted for channel m during the nth frame period, ##EQU50## is the fullness of a virtual encoder buffer for channel m at the frame instant of n, B d max is the maximum fullness of a decoder buffer of said video decoder, and N' is a decoding delay of said decoder buffer.