US5764803A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Motion-adaptive modelling of scene content for very low bit rate model-assisted coding of video sequences
Assignee: Nokia of America Corp
•
Filing: 1996-04-03
•
Granted: 1998-06-09
•
Expires: 2016-04-03
Abstract
Abstract A method and apparatus for video coding whereby a region of an image which includes a predetermined object such as a person's face in the foreground portion of the image is automatically determined. Specifically, the foreground portion of the image is identified, and one or more predetermined (geometric) shapes (e.g., ellipses) are compared with the shapes of objects found in the foreground portion of the image. The foreground portion of an image may be determined by performing a global motion estimation of the overall image to detect global image movement resulting, for example, from camera pan and zoom. That portion of the image whose movement is consistent with the estimated global motion may be identified as the background portion, with the remainder of the image identified as the foreground portion. The identified region of the image which includes the predetermined object and portions of the image which do not include the predetermined object may be coded with differing levels of coding accuracy (e.g., using different quantization levels), such that if the identified region contains, for example, a person's face, the quality of the coding of the face may be improved relative to the quality of the coding of other portions of the image.
Inventors
- 👤 Arnaud Eric Jacquin
- 👤 Wendi Beth Rabiner
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-02 23:02
CPC Classifications
H; H04; H04N; H04N19/00; H04N19/50; H04N19/503; H04N19/51; H04N19/527; H04N19/10; H04N19/102; H04N19/124; H04N19/126; H04N19/134; H04N19/154; H04N19/169; H04N19/17; H04N19/189; H04N19/196; H04N19/60; H04N19/61; H04N19/20
Claims Analysis
Total Claims: 22
Claim 1
Independent Claim
179 words
A method of coding a video signal, the video signal comprising a succession of frames, a given one of the frames comprising an image including a foreground portion and a background portion, the foreground portion of the image of the given frame including a predetermined object, the method comprising the steps of: automatically determining the foreground portion of the image of the given frame; automatically determining a region of interest within the foreground portion of the image of the given frame, said region of interest including at least a portion of the predetermined object, by matching one or more predetermined shapes to the predetermined object in the foreground portion of the image of the given frame; coding the determined region of interest with a first coder, the first coder having a first coding error; and coding at least a portion of the image of the given frame, which portion excludes the determined region of interest, with a second coder, the second coder having a second coding error, wherein said first coding error is smaller than said second coding error.
Claim 2
Independent Claim
179 words
A method of coding a video signal, the video signal comprising a succession of frames, a given one of the frames comprising an image including a foreground portion and a background portion, the foreground portion of the image of the given frame including a predetermined object, the method comprising the steps of: automatically determining the foreground portion of the image of the given frame; automatically determining a region of interest within the foreground portion of the image of the given frame, said region of interest including at least a portion of the predetermined object, by matching one or more predetermined shapes to the predetermined object in the foreground portion of the image of the given frame; coding the determined region of interest with a first coder, the first coder having a first coding error; and coding at least a portion of the image of the given frame, which portion excludes the determined region of interest, with a second coder, the second coder having a second coding error, wherein said first coding error is smaller than said second coding error.
Claim 3
Dependent Claim
45 words
The method of claim 1 wherein the step of determining the region of interest comprises performing edge extraction on at least a portion of the image of the given frame to determine a shape of the predetermined object in the foreground portion of the image.
Claim 4
Dependent Claim
67 words
The method of claim 1 wherein the step of determining the foreground portion of the image of the given frame comprises comparing the image of the given frame with an image comprised in a frame previous to the given frame, and determining a portion of the image of the given frame which has moved relative to a corresponding portion of the frame previous to the given frame.
Claim 5
Dependent Claim
112 words
The method of claim 1 wherein the step of determining the foreground portion of the image of the given frame comprises determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame previous to the given frame, determining a background portion of the image of the given frame, said background portion having exhibited motion in accordance with said one or more global motion estimation parameters as applied to a corresponding portion of the frame previous to the given frame, and determining the foreground portion of the image of the given frame based on the determined background portion.
Claim 6
Dependent Claim
203 words
The method of claim 1 further comprising the steps of determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame subsequent to the given frame, determining a search region of the image of the frame subsequent to the given frame based on the determined region of interest of the image of the given frame and on said one or more global motion estimation parameters, automatically determining a region of interest of the frame subsequent to the given frame, said region of interest of the frame subsequent to the given frame including at least a portion of the predetermined object, by matching at least one of said one or more predetermined shapes to the predetermined object within the search region of the image of the subsequent frame, coding the determined region of interest of the image of the frame subsequent to the given frame with the first coder, and coding at least a portion of the image of the frame subsequent to the given frame, which portion excludes the determined region of interest of the image of the frame subsequent to the given frame, with the second coder.
Claim 7
Dependent Claim
197 words
The method of claim 1 further comprising the steps of determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame subsequent to the given frame, selecting at least one of said one or more predetermined shapes based on the determined region of interest of the image of the given frame and on said one or more global motion estimation parameters, automatically determining a region of interest of the frame subsequent to the given frame, said region of interest of the frame subsequent to the given frame including at least a portion of the predetermined object, by matching the selected at least one of said one or more predetermined shapes to the predetermined object in the image of the subsequent frame, coding the determined region of interest of the image of the frame subsequent to the given frame with the first coder, and coding at least a portion of the image of the frame subsequent to the given frame, which portion excludes the determined region of interest of the image of the frame subsequent to the given frame, with the second coder.
Claim 8
Dependent Claim
142 words
The method of claim 1 further comprising the steps of determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame subsequent to the given frame, determining a region of interest of the image of the frame subsequent to the given frame based on the determined region of interest of the image of the given frame and on said one or more global motion estimation parameters, coding the determined region of interest of the image of the frame subsequent to the given frame with the first coder, and coding at least a portion of the image of the frame subsequent to the given frame, which portion excludes the determined region of interest of the image of the frame subsequent to the given frame, with the second coder.
Claim 9
Dependent Claim
26 words
The method of claim 1 wherein the predetermined object comprises a person's head and wherein each of the one or more predetermined shapes comprises an ellipse.
Claim 10
Dependent Claim
26 words
The method of claim 1 wherein the predetermined object comprises a person's body and wherein each of the one or more predetermined shapes comprises an ellipse.
Claim 11
Dependent Claim
40 words
The method of claim 1 wherein the first coder comprises a waveform coder having a first quantizer and the second coder comprises a waveform coder having a second quantizer, the first quantizer providing a finer quantization than the second quantizer.
Claim 12
Dependent Claim
181 words
A video coder for coding a video signal, the video signal comprising a succession of frames, a given one of the frames comprising an image including a foreground portion and a background portion, the foreground portion of the image of the given frame including a predetermined object, the apparatus comprising: means for automatically determining the foreground portion of the image of the given frame; means for automatically determining a region of interest within the foreground portion of the image of the given frame, said region of interest including at least a portion of the predetermined object, by matching one or more predetermined shapes to the predetermined object in the foreground portion of the image of the given frame; a first coder for coding the determined region of interest, the first coder having a first coding error; and a second coder for coding at least a portion of the image of the given frame, which portion excludes the determined region of interest, the second coder having a second coding error, wherein said first coding error is smaller than said second coding error.
Claim 13
Dependent Claim
181 words
A video coder for coding a video signal, the video signal comprising a succession of frames, a given one of the frames comprising an image including a foreground portion and a background portion, the foreground portion of the image of the given frame including a predetermined object, the apparatus comprising: means for automatically determining the foreground portion of the image of the given frame; means for automatically determining a region of interest within the foreground portion of the image of the given frame, said region of interest including at least a portion of the predetermined object, by matching one or more predetermined shapes to the predetermined object in the foreground portion of the image of the given frame; a first coder for coding the determined region of interest, the first coder having a first coding error; and a second coder for coding at least a portion of the image of the given frame, which portion excludes the determined region of interest, the second coder having a second coding error, wherein said first coding error is smaller than said second coding error.
Claim 14
Dependent Claim
48 words
The video coder of claim 11 wherein the means for determining the region of interest comprises means for performing edge extraction on at least a portion of the image of the given frame to determine a shape of the predetermined object in the foreground portion of the image.
Claim 15
Dependent Claim
72 words
The video coder of claim 11 wherein the means for determining the foreground portion of the image of the given frame comprises means for comparing the image of the given frame with an image comprised in a frame previous to the given frame, and means for determining a portion of the image of the given frame which has moved relative to a corresponding portion of the frame previous to the given frame.
Claim 16
Dependent Claim
119 words
The video coder of claim 11 wherein the means for determining the foreground portion of the image of the given frame comprises means for determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame previous to the given frame, means for determining a background portion of the image of the given frame, said background portion having exhibited motion in accordance with said one or more global motion estimation parameters as applied to a corresponding portion of the frame previous to the given frame, and means for determining the foreground portion of the image of the given frame based on the determined background portion.
Claim 17
Dependent Claim
211 words
The video coder of claim 11 further comprising means for determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame subsequent to the given frame, means for determining a search region of the image of the frame subsequent to the given frame based on the determined region of interest of the image of the given frame and on said one or more global motion estimation parameters, means for automatically determining a region of interest of the frame subsequent to the given frame, said region of interest of the frame subsequent to the given frame including at least a portion of the predetermined object, by matching at least one of said one or more predetermined shapes to the predetermined object within the search region of the image of the subsequent frame, means for coding the determined region of interest of the image of the frame subsequent to the given frame with the first coder, and means for coding at least a portion of the image of the frame subsequent to the given frame, which portion excludes the determined region of interest of the image of the frame subsequent to the given frame, with the second coder.
Claim 18
Dependent Claim
205 words
The video coder of claim 11 further comprising means for determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame subsequent to the given frame, means for selecting at least one of said one or more predetermined shapes based on the determined region of interest of the image of the given frame and on said one or more global motion estimation parameters, means for automatically determining a region of interest of the frame subsequent to the given frame, said region of interest of the frame subsequent to the given frame including at least a portion of the predetermined object, by matching the selected at least one of said one or more predetermined shapes to the predetermined object in the image of the subsequent frame, means for coding the determined region of interest of the image of the frame subsequent to the given frame with the first coder, and means for coding at least a portion of the image of the frame subsequent to the given frame, which portion excludes the determined region of interest of the image of the frame subsequent to the given frame, with the second coder.
Claim 19
Dependent Claim
148 words
The video coder of claim 11 further comprising means for determining one or more global motion estimation parameters based on a comparison of the image of the given frame with an image comprised in a frame subsequent to the given frame, means for determining a region of interest of the image of the frame subsequent to the given frame based on the determined region of interest of the image of the given frame and on said one or more global motion estimation parameters, means for coding the determined region of interest of the image of the frame subsequent to the given frame with the first coder, and means for coding at least a portion of the image of the frame subsequent to the given frame, which portion excludes the determined region of interest of the image of the frame subsequent to the given frame, with the second coder.
Claim 20
Dependent Claim
27 words
The video coder of claim 11 wherein the predetermined object comprises a person's head and wherein each of the one or more predetermined shapes comprises an ellipse.
Claim 21
Dependent Claim
27 words
The video coder of claim 11 wherein the predetermined object comprises a person's body and wherein each of the one or more predetermined shapes comprises an ellipse.
Claim 22
Dependent Claim
41 words
The video coder of claim 11 wherein the first coder comprises a waveform coder having a first quantizer and the second coder comprises a waveform coder having a second quantizer, the first quantizer providing a finer quantization than the second quantizer.
References Cited (Backward Citations)
US4218704A
Cited
US4245248A
Cited
US4232338A
Cited
US5086480A
Cited
US5103488A
Cited
US5117287A
Cited
US5136374A
Cited
US5309232A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-06-08 | 2001-12-08 | 2002-06-09 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-06-08 | 2005-12-08 | 2006-06-09 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-06-08 | 2009-12-08 | 2010-06-09 | $7700.00 | Window Closed / Lapsed |