US6104441A
Expired / Lapsed
US
⏱️ 0.0 years remaining
System for editing compressed image sequences
Assignee: Hewlett Packard Development Co LP
•
Filing: 1998-04-29
•
Granted: 2000-08-15
•
Expires: 2018-04-29
Abstract
Abstract This disclosure provides for an image editing system that permits manipulation of compressed image formats without full decompression to the image domain. In particular, an editor manipulates discrete cosine transform ("DCT") coefficients for a dependent frame in a compressed image sequence to incorporate DCT representations of anchor frames upon which the dependent frame depends. Once converted, the frame can simply be reordered in the compressed domain without violating temporal dependencies, subject to bit rate matching. In one embodiment, an image sequence can be cut to leave remaining image frames which are then altered, so as to eliminate temporal dependencies upon eliminated frames. In another embodiment, an image sequence can be processed in the compressed domain for reverse-play. The editor provided by this disclosure permits real-time compressed domain editing applications, with reduced requirements for computational bandwidth.
Inventors
- 👤 Susie J. Wee
- 👤 Bhaskaran Vasudev
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-03 00:01
CPC Classifications
G; G11; G11B; G11B27/00; G11B27/02; G11B27/031; G11B27/034; G11B27/036; H; H04; H04N; H04N19/00; H04N19/40; H04N21/00; H04N21/20; H04N21/23; H04N21/234; H04N21/23424; H04N21/2343; H04N21/238; H04N21/40; H04N21/43; H04N21/44; H04N21/44016; G11B2220/00; G11B2220/20; G11B2220/25; G11B2220/2508; G11B2220/2512
Claims Analysis
Total Claims: 26
Claim 1
Independent Claim
173 words
A method of editing a compressed input image sequence and producing therefrom a compressed output image sequence, the input sequence including at least one group of images (that does not depend upon any other images for reconstruction from a compressed format), the group of images having at least one compressed dependent frame that contains data for reconstructing one image based upon data from a second image in the group of images, said method using a computing device and comprising: receiving with the compressed input image sequence; using the computing device to convert a dependent frame to a compressed data format that does not depend upon the second image without decompressing all images in the group of images to an image domain; editing data using the computing device, by altering temporal order of an image represented by the dependent frame with respect to at least one other image in the sequence; and using the computing device to produce the compressed output image sequence from the data which has been edited by the computing device.
Claim 2
Independent Claim
173 words
A method of editing a compressed input image sequence and producing therefrom a compressed output image sequence, the input sequence including at least one group of images (that does not depend upon any other images for reconstruction from a compressed format), the group of images having at least one compressed dependent frame that contains data for reconstructing one image based upon data from a second image in the group of images, said method using a computing device and comprising: receiving with the compressed input image sequence; using the computing device to convert a dependent frame to a compressed data format that does not depend upon the second image without decompressing all images in the group of images to an image domain; editing data using the computing device, by altering temporal order of an image represented by the dependent frame with respect to at least one other image in the sequence; and using the computing device to produce the compressed output image sequence from the data which has been edited by the computing device.
Claim 3
Dependent Claim
33 words
A method according to claim 1, further comprising using the computing device to produce the compressed output image sequence in a format that is compliant with a Moving Pictures Expert Group ("MPEG") Standard.
Claim 4
Dependent Claim
68 words
A method according to claim 1, wherein using the computing device to convert the dependent frame includes: determining discrete cosine transform ("DCT") coefficients for the dependent frame; and modifying the DCT coefficients to incorporate image data corresponding to the second image, to yield modified DCT coefficients; wherein the dependent frame is thereby modified so as to no longer depend upon data values from the second image for decompression.
Claim 5
Dependent Claim
26 words
A method according to claim 1, further comprising clipping first and second compressed input image sequences and splicing them together to form a third image sequence.
Claim 6
Dependent Claim
90 words
A method according to claim 4, wherein splicing includes: clipping a tail frame of the first sequence by converting the tail frame to eliminate any temporal dependencies upon future frames in the first sequence; clipping a lead frame of the second sequence by converting the lead frame to eliminate any temporal dependencies upon prior frames in the second sequence; and splicing the first and second sequences together to form a third sequence, by ordering an image represented by the lead frame immediately after an image represented by the tail frame.
Claim 7
Dependent Claim
37 words
A method according to claim 1, further comprising using the computing device to edit the compressed input image sequence such that the compressed output image sequence represents a reverse-play-order of images represented by the input image sequence.
Claim 8
Dependent Claim
162 words
A method according to claim 1, wherein the compressed input image sequence is compliant with a Moving Pictures Expert Group ("MPEG") Standard, including independent ("I") frames, prediction ("P") frames and bidirectional ("B") frames, and wherein using the computing device to convert the dependent frame to a compressed data format includes: using the computing device to perform a conversion upon the dependent frame, including one of conversion of a P frame to an I frame by removing dependence of the P frame upon data from any other frame, conversion of a B frame to a B frame having no dependence upon data from future frames, such that the B frame as converted only depends upon one or more prior frames in the sequence of data, and conversion of a B frame to a B frame having no dependence upon data from prior frames, such that the B frame as converted only depends upon one or more future frames in the sequence of data.
Claim 9
Dependent Claim
29 words
A method according to claim 1, wherein using the computing device to convert the dependent frame includes converting the dependent frame without decompressing any image to an image domain.
Claim 10
Independent Claim
203 words
An apparatus that converts an input sequence of compressed input images into an output sequence of compressed images, the input sequence including at least one group of images (that does not depend upon any other images for reconstruction from a compressed format), the group of images having at least one compressed dependent frame that contains data for reconstructing one image based upon data from a second image in the group of images, said apparatus comprising: memory that receives the input sequence; and an editor that receives user commands to edit the input sequence and that, responsive to user commands, alters the temporal order of at least one image from the compressed image sequence (including an image represented by the dependent frame) with respect to another image in the input sequence, the editor including one of firmware, hardware and software that converts the dependent frame to a compressed data format that does not depend upon the second image, without decompressing all images in the group of images to an image domain, alters the temporal order of at least one image in the sequence (including an image represented by the dependent frame), and formats the compressed output image sequence to reflect the altered temporal order.
Claim 11
Independent Claim
203 words
An apparatus that converts an input sequence of compressed input images into an output sequence of compressed images, the input sequence including at least one group of images (that does not depend upon any other images for reconstruction from a compressed format), the group of images having at least one compressed dependent frame that contains data for reconstructing one image based upon data from a second image in the group of images, said apparatus comprising: memory that receives the input sequence; and an editor that receives user commands to edit the input sequence and that, responsive to user commands, alters the temporal order of at least one image from the compressed image sequence (including an image represented by the dependent frame) with respect to another image in the input sequence, the editor including one of firmware, hardware and software that converts the dependent frame to a compressed data format that does not depend upon the second image, without decompressing all images in the group of images to an image domain, alters the temporal order of at least one image in the sequence (including an image represented by the dependent frame), and formats the compressed output image sequence to reflect the altered temporal order.
Claim 12
Dependent Claim
28 words
An apparatus according to claim 9, further comprising means for splicing a satellite feed of images with a second image sequence, to produce a third, composite image sequence.
Claim 13
Dependent Claim
34 words
An apparatus according to claim 9, further comprising means for cutting a first image sequence and splicing a remaining portion of the first image sequence to a second image sequence within a compressed domain.
Claim 14
Dependent Claim
24 words
An apparatus according to claim 9, further comprising means for formatting the compressed output image sequence to represent the input image sequence in reverse-play-order.
Claim 15
Independent Claim
83 words
An apparatus, comprising: means for receiving a compressed input image sequence; means for manipulating the compressed input image sequence in a compressed domain, without decompressing any image in the input image sequence to a image domain, including one of means for cutting the compressed input image sequence and splicing remaining images of the compressed image sequence to a second image sequence and means for reversing play-order of images represented by the input image sequence; and means for producing a compressed output image sequence.
Claim 16
Independent Claim
83 words
An apparatus, comprising: means for receiving a compressed input image sequence; means for manipulating the compressed input image sequence in a compressed domain, without decompressing any image in the input image sequence to a image domain, including one of means for cutting the compressed input image sequence and splicing remaining images of the compressed image sequence to a second image sequence and means for reversing play-order of images represented by the input image sequence; and means for producing a compressed output image sequence.
Claim 17
Independent Claim
318 words
An apparatus operative to receive compressed image data complying with a Moving Pictures Experts' Group (M.P.E.G.) Standard and including information representing a sequence of independent (I) frames, prediction (P) frames and bidirectional (B) frames, to edit temporal order of images represented by the sequence in response to a set of one or more commands, and to provide an output of image data that is compressed and compliant with a MPEG Standard, said apparatus comprising: machine readable media; and instructions stored on the machine readable media that instruct a machine to receive data representing I, P and B frames, perform, with respect to a particular frame selected in response to the set of one or more commands, a conversion including one of (a) conversion of a P frame to an I frame by removing dependence of the P frame upon data from any other frame, (b) conversion of a B frame to a B frame having no dependence upon data from future frames, such that the B frame as converted only depends upon one or more prior frames in the sequence of data, and (c) conversion of a B frame to a B frame having no dependence upon data from prior frames, such that the B frame as converted only depends upon one or more future frames in the sequence of data, and alter temporal order of at least one image represented by the sequence (including an image represented by the dependent frame) with respect to another image in response to the commands, and produce therefrom the output of compressed data, wherein the conversion is automatically selected (in response to the set of one or more commands and in response to whether the particular frame is an I, P or B frame) to eliminate a temporal dependency between any frame representing the at least one image which is to be altered in temporal order with respect to another image.
Claim 18
Independent Claim
318 words
An apparatus operative to receive compressed image data complying with a Moving Pictures Experts' Group (M.P.E.G.) Standard and including information representing a sequence of independent (I) frames, prediction (P) frames and bidirectional (B) frames, to edit temporal order of images represented by the sequence in response to a set of one or more commands, and to provide an output of image data that is compressed and compliant with a MPEG Standard, said apparatus comprising: machine readable media; and instructions stored on the machine readable media that instruct a machine to receive data representing I, P and B frames, perform, with respect to a particular frame selected in response to the set of one or more commands, a conversion including one of (a) conversion of a P frame to an I frame by removing dependence of the P frame upon data from any other frame, (b) conversion of a B frame to a B frame having no dependence upon data from future frames, such that the B frame as converted only depends upon one or more prior frames in the sequence of data, and (c) conversion of a B frame to a B frame having no dependence upon data from prior frames, such that the B frame as converted only depends upon one or more future frames in the sequence of data, and alter temporal order of at least one image represented by the sequence (including an image represented by the dependent frame) with respect to another image in response to the commands, and produce therefrom the output of compressed data, wherein the conversion is automatically selected (in response to the set of one or more commands and in response to whether the particular frame is an I, P or B frame) to eliminate a temporal dependency between any frame representing the at least one image which is to be altered in temporal order with respect to another image.
Claim 19
Dependent Claim
100 words
An apparatus according to claim 14, wherein the particular frame is part of a group of pictures (GOP) and wherein the instructions instruct a machine to perform the conversion by: converting the particular frame to an image domain; and re-converting, once a particular image represented by the particular frame has been altered in temporal order with respect to another image, the particular image to one of an I frame and P frame and B frame format; whereby the sequence of images has been edited to produce a MPEG compliant output without fully decompressing an entire GOP to an image domain.
Claim 20
Dependent Claim
28 words
An apparatus according to claim 14, wherein the instructions instruct a machine to perform the conversion in a transform-domain, without decompressing the particular frame to an image domain.
Claim 21
Dependent Claim
33 words
An apparatus according to claim 14, wherein the instructions instruct a machine to perform the conversion by directly altering motion vectors and discrete cosine transform coefficients, without performing an inverse discrete cosine transformation.
Claim 22
Dependent Claim
51 words
An apparatus according to claim 14, wherein said apparatus further provides capability to play a sequence of images backward and wherein: the instructions instruct a machine to alter temporal order by reversing temporal dependencies of each of a plurality of frames selected from the group of I, P and B frames.
Claim 23
Dependent Claim
132 words
An apparatus according to claim 14, wherein said apparatus further provides capability to splice a first sequence of images with a second sequence of images, and wherein: the instructions instruct a machine to clip a tail frame of the first sequence by converting the tail frame to eliminate any temporal dependencies upon future frames in the first sequence; the instructions instruct a machine to clip a head frame of the second sequence by converting the head frame to eliminate any temporal dependencies upon prior frames in the second sequence; and the instruction instruct a machine to splice the first and second sequences together to form a third sequence, in which an image represented by the head frame immediately follows an image represented by the tail frame when the third sequence is decompressed.
Claim 24
Dependent Claim
21 words
An apparatus according to claim 19, wherein the instructions further instruct a machine to change bit rates of the head frame.
Claim 25
Dependent Claim
37 words
An apparatus according to claim 19, wherein the instructions further instruct a machine to match bit rates of the head frame to the tail frame by further quantizing one of the head frame and the tail frame.
Claim 26
Dependent Claim
47 words
An apparatus according to claim 19, wherein the instructions further instruct a machine to match bit rates of the head frame to the tail frame by further converting one of (a) an I frame to a P frame and (b) a P frame to a B frame.
References Cited (Backward Citations)
US5377023A
Cited
US5050225A
Cited
US5491761A
Cited
US5157488A
Cited
US5168375A
Cited
US5414469A
Cited
US5408328A
Cited
US5666461A
Cited
US5631742A
Cited
US5379122A
Cited
US5408274A
Cited
US5912709A
Cited
US5467131A
Cited
US5701386A
Cited
US5793894A
Cited
US5732183A
Cited
US5710873A
Cited
US5917988A
Cited
US5802240A
Cited
US5812791A
Cited
US5864682A
Cited
US5708732A
Cited
US5739862A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-08-15 | 2004-02-14 | 2004-08-15 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-08-15 | 2008-02-14 | 2008-08-15 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-08-15 | 2012-02-14 | 2012-08-15 | $7700.00 | Window Closed / Lapsed |