US6870945B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Video object tracking by estimating and subtracting background
Assignee: WASHINGTON, University of
•
Filing: 2001-06-04
•
Granted: 2005-03-22
•
Expires: 2021-06-04
Abstract
Abstract An object is tracked among a plurality of image frames. In an initial frame an operator selects an object. The object is distinguished from the remaining background portion of the image to yield a background and a foreground. A model of the background is used and updated in subsequent frames. A model of the foreground is used and updated in the subsequent frames. Pixels in subsequent frames are classified as belonging to the background or the foreground. In subsequent frames, decisions are made, including: which pixels do not belong to the background; which pixels in the foreground are to be updated; which pixels in the background were observed incorrectly in the current frame; and which background pixels are being observed for the first time. In addition, mask filtering is performed to correct errors, eliminate small islands and maintain spatial and temporal coherency of a foreground mask.
Inventors
- 👤 Todd Schoepflin
- 👤 David R. Haynor
- 👤 John D. Sahr
- 👤 Yongmin Kim
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-07 07:49
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06T7/254; G06T7/10; G06T7/12; G06T7/155; G06T2207/00; G06T2207/10; G06T2207/10016
Claims Analysis
Total Claims: 35
Claim 1
Independent Claim
220 words
A method for tracking an object among a plurality of image frames, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being within observable portions of the background for the given image frame; deriving a background model from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame.
Claim 2
Independent Claim
220 words
A method for tracking an object among a plurality of image frames, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being within observable portions of the background for the given image frame; deriving a background model from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame.
Claim 3
Dependent Claim
56 words
The method of claim 1 , further comprising the step of performing a mask filtering operation on the background mask of the current image frame to decide whether to change a pixel classification from being a background pixel in the background mask to being a foreground pixel in the foreground mask of the current image frame.
Claim 4
Dependent Claim
56 words
The method of claim 1 , further comprising the step of performing a mask filtering operation on the foreground mask of the current image frame to decide whether to change a pixel classification from being a foreground pixel in the foreground mask to being a background pixel in the background mask of the current image frame.
Claim 5
Dependent Claim
59 words
The method of claim 1 , further comprising, for each one pixel of a current image frame, the step of predicting a background value for said one pixel based upon a predicted background value of said one pixel from the prior image frame, a pixel value of said one pixel from the current image frame, and a mixing factor.
Claim 6
Dependent Claim
35 words
The method of claim 4 , in which the step of classifying said each one pixel of the current image frame is based in part on the predicted background value for said each one pixel.
Claim 7
Dependent Claim
85 words
The method of claim 1 , in which the step of classifying comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; motion is detected in the immediate past for said one pixel; motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 8
Dependent Claim
123 words
The method of claim 1 , in which the step of classifying comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; a first degree of motion is detected in the immediate past for said one pixel; the first degree of motion is detected in the immediate future for said one pixel; a second degree of motion is detected in the immediate past for said one pixel; the second degree of motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 9
Independent Claim
204 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the apparatus comprising: a background model derived from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames; a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on a current state of the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame.
Claim 10
Independent Claim
204 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the object moving relative to a background, wherein portions of the background that are initially hidden, become observable during tracking, the apparatus comprising: a background model derived from the plurality of image frames, wherein for each image frame, background pixels within an observable portion of the background are identified, and wherein an identified background pixel is included in the background model only after being observable within the background for at least three consecutive image frames; a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on a current state of the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame.
Claim 11
Dependent Claim
51 words
The apparatus of claim 8 , further comprising a filter which filters the background mask of the current image frame to decide whether to change a pixel classification from being a background pixel in the background mask to being a foreground pixel in the foreground mask of the current image frame.
Claim 12
Dependent Claim
51 words
The apparatus of claim 8 , further comprising a filter which filters the foreground mask of the current image frame to decide whether to change a pixel classification from being a foreground pixel in the foreground mask to being a background pixel in the background mask of the current image frame.
Claim 13
Dependent Claim
52 words
The apparatus of claim 8 , in which the processor predicts a background value for said each one pixel based upon a predicted background value of said each one pixel from the prior image frame, a pixel value of said each one pixel from the current image frame, and a mixing factor.
Claim 14
Dependent Claim
33 words
The apparatus of claim 11 , in which the processor classifies said each one pixel of the current image frame based in part on the predicted background value for said each one pixel.
Claim 15
Dependent Claim
86 words
The apparatus of claim 8 , in which the classifying by the processor comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; motion is detected in the immediate past for said one pixel; motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 16
Dependent Claim
124 words
The apparatus of claim 8 , in which the classifying by the processor comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; a first degree of motion is detected in the immediate past for said one pixel; the first degree of motion is detected in the immediate future for said one pixel; a second degree of motion is detected in the immediate past for said one pixel; the second degree of motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 17
Independent Claim
224 words
A method for tracking an object among a plurality of image frames, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame; maintaining a background model of background pixels which have been observable in at least three consecutive image frames by predicting, for each one pixel of a current image frame, a background value for said one pixel based upon a predicted background value of said one pixel from the prior image frame, a pixel value of said one pixel from the current image frame, and a mixing factor for weighting the background value of said one pixel from the prior image frame; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame.
Claim 18
Independent Claim
224 words
A method for tracking an object among a plurality of image frames, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame; maintaining a background model of background pixels which have been observable in at least three consecutive image frames by predicting, for each one pixel of a current image frame, a background value for said one pixel based upon a predicted background value of said one pixel from the prior image frame, a pixel value of said one pixel from the current image frame, and a mixing factor for weighting the background value of said one pixel from the prior image frame; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame.
Claim 19
Dependent Claim
35 words
The method of claim 15 , in which the step of classifying said each one pixel of the current image frame is based in part on the predicted background value for said each one pixel.
Claim 20
Dependent Claim
56 words
The method of claim 15 , further comprising the step of performing a mask filtering operation on the background mask of the current image frame to decide whether to change a pixel classification from being a background pixel in the background mask to being a foreground pixel in the foreground mask of the current image frame.
Claim 21
Dependent Claim
56 words
The method of claim 15 , further comprising the step of performing a mask filtering operation on the foreground mask of the current image frame to decide whether to change a pixel classification from being a foreground pixel in the foreground mask to being a background pixel in the background mask of the current image frame.
Claim 22
Independent Claim
239 words
A method for tracking an object among a plurality of image frames, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame; maintaining a background model of background pixels which have been observable in at least three consecutive image frames; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame; wherein the step of classifying comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; motion is detected in the immediate past for said one pixel; motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 23
Independent Claim
239 words
A method for tracking an object among a plurality of image frames, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame; maintaining a background model of background pixels which have been observable in at least three consecutive image frames; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame; wherein the step of classifying comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; motion is detected in the immediate past for said one pixel; motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 24
Dependent Claim
56 words
The method of claim 19 , further comprising the step of performing a mask filtering operation on the background mask of the current image frame to decide whether to change a pixel classification from being a background pixel in the background mask to being a foreground pixel in the foreground mask of the current image frame.
Claim 25
Dependent Claim
56 words
The method of claim 19 , further comprising the step of performing a mask filtering operation on the foreground mask of the current image frame to decide whether to change a pixel classification from being a foreground pixel in the foreground mask to being a background pixel in the background mask of the current image frame.
Claim 26
Independent Claim
278 words
A method for tracking an object among a plurality of image frames, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame; maintaining a background model of background pixels which have been observable in at least three consecutive image frames; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame; and wherein the step of classifying comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; a first degree of motion is detected in the immediate past for said one pixel; the first degree of motion is detected in the immediate future for said one pixel; a second degree of motion is detected in the immediate past for said one pixel; the second degree of motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 27
Independent Claim
278 words
A method for tracking an object among a plurality of image frames, the method comprising the steps of: defining a background mask for each one frame among a plurality of image frames, including a current image frame, the background mask of a given image frame comprising background pixels, the background pixels of the given image frame being observable for the given image frame; maintaining a background model of background pixels which have been observable in at least three consecutive image frames; classifying each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame and a subsequent image frame; defining a foreground mask for the current image frame as being pixels not in the background mask for said current image frame; and identifying the object as being the pixels within the foreground mask for the current image frame; and wherein the step of classifying comprises identifying an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; a first degree of motion is detected in the immediate past for said one pixel; the first degree of motion is detected in the immediate future for said one pixel; a second degree of motion is detected in the immediate past for said one pixel; the second degree of motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 28
Dependent Claim
56 words
The method of claim 22 , further comprising the step of performing a mask filtering operation on the background mask of the current image frame to decide whether to change a pixel classification from being a background pixel in the background mask to being a foreground pixel in the foreground mask of the current image frame.
Claim 29
Dependent Claim
56 words
The method of claim 22 , further comprising the step of performing a mask filtering operation on the foreground mask of the current image frame to decide whether to change a pixel classification from being a foreground pixel in the foreground mask to being a background pixel in the background mask of the current image frame.
Claim 30
Independent Claim
211 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising: a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames; a first processor which predicts a background value for said each one pixel based upon a predicted background value of said each one pixel from the prior image frame, a pixel value of said each one pixel from the current image frame, and a mixing factor for weighting the background value of said one pixel from the prior image frame; a second processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame.
Claim 31
Independent Claim
211 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising: a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames; a first processor which predicts a background value for said each one pixel based upon a predicted background value of said each one pixel from the prior image frame, a pixel value of said each one pixel from the current image frame, and a mixing factor for weighting the background value of said one pixel from the prior image frame; a second processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame.
Claim 32
Independent Claim
225 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising: a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames; a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame; wherein the processor identifies an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; motion is detected in the immediate past for said one pixel; motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 33
Independent Claim
225 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising: a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames; a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame; wherein the processor identifies an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; motion is detected in the immediate past for said one pixel; motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 34
Independent Claim
263 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising: a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames; a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame; wherein the processor identifies an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; a first degree of motion is detected in the immediate past for said one pixel; the first degree of motion is detected in the immediate future for said one pixel; a second degree of motion is detected in the immediate past for said one pixel; the second degree of motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
Claim 35
Independent Claim
263 words
An apparatus for tracking an object among a plurality of image frames, the apparatus receiving an initial estimate of the object for an initial image frame, the apparatus comprising: a background model of values for a plurality of background pixels which have been observable in at least three consecutive image frames; a processor which classifies each one pixel of the current image frame as being a background pixel or a foreground pixel based on the background model, the current image frame, and at least one of a prior image frame or a subsequent image frame, the processor identifying a background mask for the current image frame; and a foreground mask for the current image frame formed as being pixels not in the background mask for said current image frame, wherein the object being tracked is identified as corresponding to the pixels within the foreground mask of the current image frame; wherein the processor identifies an event from the group of events comprising: prospectively include said one pixel in the background mask for the current image frame; a first degree of motion is detected in the immediate past for said one pixel; the first degree of motion is detected in the immediate future for said one pixel; a second degree of motion is detected in the immediate past for said one pixel; the second degree of motion is detected in the immediate future for said one pixel; prospectively include said one pixel in the foreground pixel for the current image frame; and said one pixel is not to be updated in the background model.
References Cited (Backward Citations)
US5526053A
Cited
US5491517A
Cited
US5852475A
Cited
US5710602A
Cited
US6400831B2
Cited
US6545706B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2008-03-21 | 2008-09-20 | 2009-03-22 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2012-03-21 | 2012-09-20 | 2013-03-22 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2016-03-21 | 2016-09-20 | 2017-03-22 | $7700.00 | Window Closed / Lapsed |