US5262856A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Video image compositing techniques
Assignee: Massachusetts Institute of Technology
•
Filing: 1992-06-04
•
Granted: 1993-11-16
•
Expires: 2012-06-04
Abstract
Abstract A method of processing video data created for display in one aspect ratio in a plurality of different sequences of frames thereof, each frame in a sequence containing background data which represents at least a part of the same relatively stationary background data in the sequence. The background data in each frame of a sequence is stored and processed so as to produce combined background data representing the background image in the sequence in a different aspect ratio, without the presence of moving objects therein. Data representing moving objects in each frame of a sequence are stored together with selected portions of the background data which surround the moving objects in each frame. The data representing moving objects in each frame are correctly positioned in the combined background data, which is used for each frame of the sequence, by matching the selected portion of the background data surrounding the moving objects with corresponding portions of the combined background data to determine the correct positions of the moving objects in the background image. The background data and the matching portions of the moving objects are composited in accordance with such correct position determination to form a composite image for each frame in which the moving objects are correctly positioned in the combined background image.
Inventors
- 👤 Andrew B. Lippman
- 👤 Walter R. Bender
- 👤 Patrick G. McLean
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-30 20:04
CPC Classifications
H; H04; H04N; H04N7/00; H04N7/01; H04N7/0117; H04N7/0122; H04N19/00; H04N19/20; H04N19/23; H04N19/50; H04N19/503; H04N19/51; H04N19/527; H04N5/00; H04N5/222; H04N5/262; H04N5/272
Claims Analysis
Total Claims: 15
Claim 1
Independent Claim
156 words
A method for processing video data in a plurality of different sequences of frames thereof, each frame in a particular sequence of frames including relatively motionless background data representing at least a part of the entire relatively motionless background data in said sequence and moving object data representing moving objects in said sequence, said method comprising the steps of storing the background and moving object data in each frame of a particular sequence of frames; aligning common portions of the stored background data in each frame to provide stored combined background data which represents a composite relatively motionless background image which contains substantially the entire relatively motionless background data in said sequence and is at least as wide as the background image in any one frame of said sequence thereof; processing the aligned background data to remove the moving object data therefrom; storing said processed background data as combined background data representing said composite background image.
Claim 2
Independent Claim
156 words
A method for processing video data in a plurality of different sequences of frames thereof, each frame in a particular sequence of frames including relatively motionless background data representing at least a part of the entire relatively motionless background data in said sequence and moving object data representing moving objects in said sequence, said method comprising the steps of storing the background and moving object data in each frame of a particular sequence of frames; aligning common portions of the stored background data in each frame to provide stored combined background data which represents a composite relatively motionless background image which contains substantially the entire relatively motionless background data in said sequence and is at least as wide as the background image in any one frame of said sequence thereof; processing the aligned background data to remove the moving object data therefrom; storing said processed background data as combined background data representing said composite background image.
Claim 3
Dependent Claim
75 words
A method in accordance with claim 1 and further including the steps of processing the data in each frame of said sequence of frames to provide moving object data representing objects in each frame of said sequence of frames, which move relative to the relatively motionless background data over said sequence of frames, together with data representing selected portions of the background data which at least partially surround each moving object in each said frame.
Claim 4
Dependent Claim
127 words
A method in accordance with claim 2 and further including the steps of matching the selected portions of said background data which at least partially surround the object data representing the moving objects in each frame with corresponding portions of the combined background data to determine the correct position of said moving objects in each frame in said combined background data; and compositing said matching portions of the background at least partially surrounding said moving objects into the combined background data in each frame in accordance with said correct position determination so as to form composited image data for each frame of said sequence of frames in which said moving objects are correctly positioned in said combined background data in each frame of said sequence of frames.
Claim 5
Dependent Claim
145 words
A method in accordance with claim 3 and further including the steps of transmitting said combined background data representing said composite background image to a receiver for use in displaying said composite background image in each frame of said sequence of frames whereby each displayed frame includes at least a portion of substantially the same composite background image; transmitting said moving object data and the portions of background data at least partially surrounding said moving object data to a receiver for use in displaying in each frame of said sequence thereof the moving object data for each said frame composited into the combined background data for each frame; receiving at said receiver the transmitted combined background data and the transmitted moving object data with said portions of at least partially surrounding background data; and performing said matching step and said compositing step at said receiver.
Claim 6
Dependent Claim
110 words
A method in accordance with claim 3 wherein said matching step is performed at a transmitter and further includes the steps of transmitting said combined background data representing said composite background image, said moving object data with the portions of background data at least partially surrounding said moving object data, and information representing said determined correct position of said moving object in said combined background data for each frame of said sequence of frames to a receiver; receiving at said receiver said combined background data, said moving object data with said at least partially surrounding background data, and said correct position information; and performing said compositing step at said receiver.
Claim 7
Dependent Claim
77 words
A method in accordance with claim 1 wherein said aligning step includes storing said background data in each frame in a buffer so that common portions thereof are aligned when background data in a frame is shifted relative to background data in a previous frame; combining the stored background data in all of the aligned frames in a manner so as to produce combined background data representing the substantially entire background data in said sequence of frames.
Claim 8
Dependent Claim
62 words
A method in accordance with claim 6 and further including comparing background data in one frame relative to background data in a previous frame, and when the shift of the background data in said one frame is above a selected threshold, storing the background data in said one frame in said buffer with an offset from said previous frame representing said shift.
Claim 9
Dependent Claim
66 words
A method in accordance with claim 7 wherein, in said storing step, the background data in each frame is stored in a single widened buffer so that the background data in a frame which is shifted from the background data in a previous frame is stored in said widened buffer at a selected offset position therein, portions of said buffer at each frame containing no data.
Claim 10
Dependent Claim
74 words
A method in accordance with claim 7 wherein, in said storing step, background data is stored in a narrow buffer so that the background data in a frame which is shifted from background data in a previous frame is stored in said narrow buffer, there being no portions of said narrow buffer at each frame not containing said data, and further storing a coded number representing the offset of the data in each frame.
Claim 11
Dependent Claim
95 words
A method in accordance with claim 1 wherein said processing step includes removing pixels representing moving object data in each frame of said sequence of frames and storing said removed pixels in a moving object buffer to provide a sequence of stored frames containing only the moving objects in each frame of said sequence; storing the relatively motionless background data remaining when said moving object data is removed in each frame in a single widened composite background buffer, said stored relatively motionless background data representing substantially the entire background data in said sequence of frames.
Claim 12
Dependent Claim
38 words
A method in accordance with claim 3 wherein the number of frames in which said moving object data is composited into the combined background data depends on how often the moving object moves over said sequence of frames.
Claim 13
Dependent Claim
44 words
A method in accordance with claim 11 wherein the moving object data for objects which move relatively often over said sequence of frames are composited into said combined background data more frequently than the moving object data for objects which move relatively less often.
Claim 14
Dependent Claim
67 words
A method in accordance with claim 4 wherein the combined background data are transmitted once for a sequence of frames, moving object data for objects which move relatively often over the sequence of frames are transmitted relatively frequently for the sequence of frames, and moving object data for objects which move less often over the sequence of frames are transmitted less frequently for the sequence of frames.
Claim 15
Dependent Claim
61 words
A method in accordance with claim 13 wherein moving object data for objects which move the most frequently over the sequence of frames are transmitted at each frame of the sequence and moving object data for objects which do not move the most frequently over the sequence of frames are transmitted more than once but less frequently than at each frame.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 1996-11-15 | 1997-05-17 | 1997-11-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2000-11-15 | 2001-05-17 | 2001-11-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2004-11-15 | 2005-05-17 | 2005-11-16 | $7700.00 | Window Closed / Lapsed |