US6661918B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Background estimation and segmentation based on range and color
Assignee: Intel Corp
•
Filing: 1999-12-03
•
Granted: 2003-12-09
•
Expires: 2019-12-03
Abstract
Abstract Segmentation of background and foreground objects in an image is based upon the joint use of both range and color data. Range-based data is largely independent of color image data, and hence not adversely affected by the limitations associated with color-based segmentation, such as shadows and similarly colored objects. Furthermore, color segmentation is complementary to range measurement in those cases where reliable range data cannot be obtained. These complementary sets of data are used to provide a multidimensional background estimation. The segmentation of a foreground object in a given frame of an image sequence is carried out by comparing the image frames with background statistics relating to range and normalized color, using the sets of statistics in a complementary manner.
Inventors
- 👤 Gaile Gordon
- 👤 Michael Harville
- 👤 John Woodfill
- 👤 Trevor Darrell
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 12:56
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/10; G06T7/11; G06T7/143; G06T7/174; G06T7/194; G06V; G06V10/00; G06V10/20; G06V10/28; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/10021; G06T2207/10028; G06T2207/20; G06T2207/20212; G06T2207/20224
Claims Analysis
Total Claims: 31
Claim 1
Independent Claim
169 words
A method for distinguishing between background and foreground objects in an image, comprising the steps of: i) determining respective range values for pixels in each of a plurality of related images; ii) determining respective intensity values for pixels in each of said plurality of related images; iii) establishing a multi-dimensional background model for at least some of said pixels on the basis of said range values and said intensity values; iv) comparing the range value for a particular pixel in a given image with the background model for that pixel, and labeling the pixel as a foreground pixel if the range value differs from the background model by a range threshold; v) comparing the intensity value for said particular pixel in a given image with the background model for that pixel, and labeling the pixel as a foreground pixel if the intensity value differs from the background model by an intensity threshold; and vi) adjusting said intensity threshold in accordance with a determination made during said comparing step (iv).
Claim 2
Independent Claim
169 words
A method for distinguishing between background and foreground objects in an image, comprising the steps of: i) determining respective range values for pixels in each of a plurality of related images; ii) determining respective intensity values for pixels in each of said plurality of related images; iii) establishing a multi-dimensional background model for at least some of said pixels on the basis of said range values and said intensity values; iv) comparing the range value for a particular pixel in a given image with the background model for that pixel, and labeling the pixel as a foreground pixel if the range value differs from the background model by a range threshold; v) comparing the intensity value for said particular pixel in a given image with the background model for that pixel, and labeling the pixel as a foreground pixel if the intensity value differs from the background model by an intensity threshold; and vi) adjusting said intensity threshold in accordance with a determination made during said comparing step (iv).
Claim 3
Dependent Claim
34 words
The method of claim 1 further including the step of determining whether the range value for said particular pixel is valid, and performing said comparing step (iv) only if said range value is valid.
Claim 4
Dependent Claim
63 words
The method of claim 2 further including the step of determining whether background range data for said particular pixel is valid, and if said background range data is not valid, labeling the particular pixel as a foreground pixel in response to the comparing step (iv) for neighboring pixels if a range gradient associated with said particular pixel is less than a gradient threshold.
Claim 5
Dependent Claim
41 words
The method of claim 1 further including the step of determining whether the luminance of said particular pixel is greater than a designated minimum value, and performing said comparing step (v) only if the luminance is greater than said minimum value.
Claim 6
Dependent Claim
47 words
The method of claim 1 further including the step of determining the ratio of the intensity value for said particular pixel to the background intensity value for that pixel, and labeling the pixel as a foreground pixel if said ratio is less than a first predetermined value.
Claim 7
Dependent Claim
23 words
The method of claim 5 wherein said first predetermined value is based upon a decrease in luminance due to shadows in an image.
Claim 8
Dependent Claim
27 words
The method of claim 5 further including the step of labeling the pixel as a foreground pixel if said ratio is greater than a second predetermined value.
Claim 9
Dependent Claim
23 words
The method of claim 7 wherein said second predetermined value is based upon an increase in luminance due to interreflections in an image.
Claim 10
Dependent Claim
35 words
The method of claim 1 further including the step of alternatively labeling the pixel as a foreground pixel if the luminance of the pixel is greater than a designated minimum value by a predetermined factor.
Claim 11
Dependent Claim
12 words
The method of claim 9 wherein said factor is at least 2.
Claim 12
Dependent Claim
31 words
The method of claim 9 wherein said alternative labeling step is performed only if the luminance of the background texture model for the pixel is less than said designated minimum value.
Claim 13
Dependent Claim
33 words
The method of claim 1 wherein said intensity threshold is increased if the range value for the pixel does not differ from the background range data for the pixel by said range threshold.
Claim 14
Dependent Claim
30 words
The method of claim 1 further including the step of updating said background model on the basis of range values and intensity values obtained from a sequence of N images.
Claim 15
Dependent Claim
33 words
The method of claim 13 further including the steps of detecting image-to-image changes in said range values and intensity values, and effectively adjusting the value of N in accordance with the detected changes.
Claim 16
Dependent Claim
20 words
The method of claim 14 wherein the value of N is effectively increased when the magnitude of detected changes increases.
Claim 17
Dependent Claim
18 words
The method of claim 1 wherein said intensity values are one-dimensional values corresponding to the luminance of pixels.
Claim 18
Dependent Claim
17 words
The method of claim 1 wherein said intensity values are two-dimensional values which are invariant to luminance.
Claim 19
Dependent Claim
14 words
The method of claim 1 wherein said intensity values represent a three-dimensional color space.
Claim 20
Dependent Claim
17 words
The method of claim 18 further including the step of normalizing said intensity values according to luminance.
Claim 21
Independent Claim
127 words
A system for identifying foreground objects in an image comprising: a sensor which determines intensity values for pixels within an image; a range processor which determines distance values for pixels within said image; a background estimator which computes a multi-dimensional background model for said pixels on the basis of said intensity values and said distance values, updates said background model in accordance with successive images in a sequence of plural images, determines a level of activity within the sequence of images, and adjusts the rate at which said background model is updated in accordance with the determined level of activity; and a discriminator which compares intensity and distance values for pixels in an image to said background model and determines whether said pixels represent a foreground object.
Claim 22
Independent Claim
127 words
A system for identifying foreground objects in an image comprising: a sensor which determines intensity values for pixels within an image; a range processor which determines distance values for pixels within said image; a background estimator which computes a multi-dimensional background model for said pixels on the basis of said intensity values and said distance values, updates said background model in accordance with successive images in a sequence of plural images, determines a level of activity within the sequence of images, and adjusts the rate at which said background model is updated in accordance with the determined level of activity; and a discriminator which compares intensity and distance values for pixels in an image to said background model and determines whether said pixels represent a foreground object.
Claim 23
Dependent Claim
41 words
The system of claim 20 wherein said background estimator generates a multi-dimensional histogram of intensity and distance values for a pixel within a plurality of related images, and computes a background model for said pixel by means of a clustering technique.
Claim 24
Dependent Claim
40 words
The system of claim 20 wherein said discriminator determines the differences between the intensity and distance values for a pixel in an image and the background model for that pixel, and compares said differences to respective intensity and distance thresholds.
Claim 25
Dependent Claim
26 words
The system of claim 22 wherein said discriminator selectively adjusts said intensity threshold in accordance with the result of a comparison based upon said distance threshold.
Claim 26
Dependent Claim
38 words
The system of claim 22 wherein said discriminator increases said intensity threshold if the difference between the distance value for a pixel in an image and the background model for that pixel is less than said distance threshold.
Claim 27
Dependent Claim
18 words
The system of claim 20 wherein said background estimator decreases the update rate as the activity level increases.
Claim 28
Dependent Claim
18 words
The system of claim 20 wherein said intensity values are one-dimensional values corresponding to the luminance of pixels.
Claim 29
Dependent Claim
17 words
The system of claim 20 wherein said intensity values are two-dimensional values which are invariant to luminance.
Claim 30
Dependent Claim
14 words
The system of claim 20 wherein said intensity values represent a three-dimensional color space.
Claim 31
Dependent Claim
18 words
The system of claim 28 further including an intensity processor which normalizes said intensity values according to luminance.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2006-12-08 | 2007-06-09 | 2007-12-09 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2010-12-08 | 2011-06-09 | 2011-12-09 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2014-12-08 | 2015-06-09 | 2015-12-09 | $7700.00 | Window Closed / Lapsed |