US6707940B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for image segmentation
Assignee: Beijing Xiaomi Mobile Software Co Ltd
•
Filing: 2000-03-31
•
Granted: 2004-03-16
•
Expires: 2020-03-31
Abstract
Abstract A method and apparatus is described that smoothes images within regions but not across boundaries. In one embodiment, a segmentation map is smoothed. Based on the color difference between pixels in an input frame and corresponding pixels in a background image, this segmentation map indicates the likelihood that each pixel in the input frame belongs to the foreground. The smoothed segmentation map may be processed further to reduce misclassification errors. Based on the segmentation map, the foreground portion of the image is superimposed over a new background image.
Inventors
- 👤 Richard J. Qian
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:22
CPC Classifications
G; G06; G06T; G06T5/00; G06T5/70; G06T5/20; G06T5/50; G06T7/00; G06T7/10; G06T7/194; G06T2207/00; G06T2207/10; G06T2207/10016; G06T2207/20; G06T2207/20172; G06T2207/20192
Claims Analysis
Total Claims: 36
Claim 1
Independent Claim
220 words
An apparatus comprising: a gradient calculator configured and arranged to calculate (a) a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, and to calculate (b) smoothing factors, each smoothing factor corresponding to one among the gradients, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; and a pixel value calculator configured and arranged to calculate a new value for the center pixel, the new value being expressed in terms of at least (c) a center pixel contribution and (d) a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (e) one among the smoothing factors and (f) a value of a pixel that lies in the direction corresponding to the smoothing factor.
Claim 2
Independent Claim
220 words
An apparatus comprising: a gradient calculator configured and arranged to calculate (a) a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, and to calculate (b) smoothing factors, each smoothing factor corresponding to one among the gradients, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; and a pixel value calculator configured and arranged to calculate a new value for the center pixel, the new value being expressed in terms of at least (c) a center pixel contribution and (d) a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (e) one among the smoothing factors and (f) a value of a pixel that lies in the direction corresponding to the smoothing factor.
Claim 3
Dependent Claim
37 words
The apparatus according to claim 1 , wherein the gradient calculator is further configured and arranged to calculate at least one among the smoothing factors at least in part by normalizing a magnitude of a corresponding gradient.
Claim 4
Dependent Claim
35 words
The apparatus according to claim 1 , wherein at least one among the center pixel contribution and the neighborhood contribution is weighted by a factor related to the number of the corresponding plurality of directions.
Claim 5
Dependent Claim
37 words
The apparatus according to claim 1 , wherein the new value for the center pixel is expressed in terms of at least the original value for the center pixel, the center pixel contribution, and the neighborhood contribution.
Claim 6
Dependent Claim
27 words
The apparatus according to claim 1 , wherein a value of at least one among the gradients is independent of the original value of the center pixel.
Claim 7
Dependent Claim
22 words
The apparatus according to claim 1 , wherein the directions include at least three directions that are not parallel to one another.
Claim 8
Dependent Claim
24 words
The apparatus according to claim 1 , wherein the directions include at least two directions that are neither parallel nor orthogonal to one another.
Claim 9
Independent Claim
282 words
An apparatus including a filter, said filter comprising: a gradient calculator configured and arranged to calculate (a) a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel having eight eight-neighbor pixels, the two diagonal directions being orthogonal to each other at the center pixel, and to calculate (b) smoothing factors, each smoothing factor corresponding to one among the gradients, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; and a pixel value calculator configured and arranged to calculate a filtered value for the center pixel, the filtered value being expressed in terms of at least a center pixel contribution and a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (e) one among the smoothing factors and (f) a value of a pixel that lies in the direction corresponding to the smoothing factor, wherein said filter is used to process each among substantially all of the pixels of an input image as a center pixel, and wherein a filtered image comprises the filtered values calculated for said substantially all of the pixels of the input image.
Claim 10
Independent Claim
282 words
An apparatus including a filter, said filter comprising: a gradient calculator configured and arranged to calculate (a) a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel having eight eight-neighbor pixels, the two diagonal directions being orthogonal to each other at the center pixel, and to calculate (b) smoothing factors, each smoothing factor corresponding to one among the gradients, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; and a pixel value calculator configured and arranged to calculate a filtered value for the center pixel, the filtered value being expressed in terms of at least a center pixel contribution and a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (e) one among the smoothing factors and (f) a value of a pixel that lies in the direction corresponding to the smoothing factor, wherein said filter is used to process each among substantially all of the pixels of an input image as a center pixel, and wherein a filtered image comprises the filtered values calculated for said substantially all of the pixels of the input image.
Claim 11
Dependent Claim
14 words
The apparatus according to claim 8 , said input image being a segmentation map.
Claim 12
Dependent Claim
30 words
The apparatus according to claim 9 , said segmentation map being obtained by comparing at least a portion of an original image to a portion of a corresponding background image.
Claim 13
Dependent Claim
64 words
The apparatus according to claim 9 , a value of each among substantially all of said pixels of said segmentation map being obtained at least in part by comparing a value of a pixel of an original image with a value of a corresponding pixel of an image of a background, wherein said background appears in at least a portion of the original image.
Claim 14
Dependent Claim
64 words
The apparatus according to claim 11 , wherein said comparing includes applying one among a first function and a second function, wherein the first function is applied when an intensity of the pixel of the original image is below a predetermined threshold, and wherein the second function is applied when the intensity of the pixel of the original image is above the predetermined threshold.
Claim 15
Dependent Claim
52 words
The apparatus according to claim 12 , at least one among said first function and said second function being based at least in part upon a distance in color space between a value of the pixel in the original image and a value,of the corresponding pixel in the image of the background.
Claim 16
Dependent Claim
87 words
The apparatus according to claim 11 , said apparatus further including a spurious blob remover to receive the filtered image and to produce a refined image, wherein said spurious blob remover identifies connected components in a binary image, said binary image being derived from the filtered image, and wherein for each among substantially all of the pixels in a connected component in the binary image, the value of a corresponding pixel in the refined image depends at least in part on the size of the connected component.
Claim 17
Dependent Claim
57 words
The apparatus according to claim 14 , said apparatus further including a pixel stabilizer to receive the refined image and to produce a stabilized image, wherein said pixel stabilizer determines a value for a pixel in the stabilized image according to a classification history of a corresponding pixel in the refined image over a window of time.
Claim 18
Dependent Claim
86 words
The apparatus according to claim 15 , wherein a first candidate for a value of a pixel in an output image is a value of a corresponding pixel in the original image, and wherein a second candidate for the value of the pixel in the output image is a value of a corresponding pixel in a background image, and wherein the value of the pixel in the output image is determined at least in part by a value of a corresponding pixel in the stabilized image.
Claim 19
Independent Claim
198 words
A method comprising: calculating a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; for each among the gradients, calculating a corresponding smoothing factor; and calculating a new value for the center pixel, the new value being expressed in terms of at least a center pixel contribution and a neighborhood pixel contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (a) one among the smoothing factors and (b) a value of a pixel that lies in the direction corresponding to the smoothing factor.
Claim 20
Independent Claim
198 words
A method comprising: calculating a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; for each among the gradients, calculating a corresponding smoothing factor; and calculating a new value for the center pixel, the new value being expressed in terms of at least a center pixel contribution and a neighborhood pixel contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (a) one among the smoothing factors and (b) a value of a pixel that lies in the direction corresponding to the smoothing factor.
Claim 21
Dependent Claim
24 words
The method according to claim 17 , wherein calculating at least one among said smoothing factors comprises normalizing a magnitude of a corresponding gradient.
Claim 22
Dependent Claim
33 words
The method according to claim 17 , wherein at least one among the center pixel contribution and the neighborhood contribution is weighted by a factor related to the number of the corresponding plurality.
Claim 23
Dependent Claim
37 words
The method according to claim 17 , wherein the new value for the center pixel is expressed in terms of at least the original value for the center pixel, the center pixel contribution, and the neighborhood contribution.
Claim 24
Independent Claim
249 words
A method comprising filtering an input image, said filtering comprising: calculating a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; calculating smoothing factors, each smoothing factor corresponding to one among the gradients; and calculating a filtered value for the center pixel, the filtered value being expressed in terms of at least a center pixel contribution and a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (a) one among the smoothing factors and (b) a value of a pixel that lies in the direction corresponding to the smoothing factor, wherein said filtering is performed to process each among substantially all of the pixels of an input image as a center pixel, and wherein a filtered image comprises the filtered values calculated for said substantially all of the pixels of the input image.
Claim 25
Independent Claim
249 words
A method comprising filtering an input image, said filtering comprising: calculating a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, wherein the center pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; calculating smoothing factors, each smoothing factor corresponding to one among the gradients; and calculating a filtered value for the center pixel, the filtered value being expressed in terms of at least a center pixel contribution and a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (a) one among the smoothing factors and (b) a value of a pixel that lies in the direction corresponding to the smoothing factor, wherein said filtering is performed to process each among substantially all of the pixels of an input image as a center pixel, and wherein a filtered image comprises the filtered values calculated for said substantially all of the pixels of the input image.
Claim 26
Dependent Claim
14 words
The method according to claim 21 , said input image being a segmentation map.
Claim 27
Dependent Claim
30 words
The method according to claim 21 , said segmentation map being obtained by comparing at least a portion of an original image to a portion of a corresponding background image.
Claim 28
Dependent Claim
64 words
The method according to claim 22 , a value of each among substantially all of said pixels of said segmentation map being obtained at least in part by comparing a value of a pixel of an original image with a value of a corresponding pixel of an image of a background, wherein said background appears in at least a portion of the original image.
Claim 29
Dependent Claim
64 words
The method according to claim 24 , wherein said comparing includes applying one among a first function and a second function, wherein the first function is applied when an intensity of the pixel of the original image is below a predetermined threshold, and wherein the second function is applied when the intensity of the pixel of the original image is above the predetermined threshold.
Claim 30
Dependent Claim
53 words
The method according to claim 25 , at least one among said first function and said second function being based at least in part upon a distance in color space between a value of the pixel in the original image and a value of the corresponding pixel in the image of the background.
Claim 31
Dependent Claim
85 words
The method according to claim 24 , said method further including removing spurious blobs from the filtered image to produce a refined image, wherein said removing spurious blobs comprises identifying connected components in a binary image, said binary image being derived from the filtered image, and wherein for each among substantially all of the pixels in a connected component in the binary image, the value of a corresponding pixel in the refined image depends at least in part on the size of the connected component.
Claim 32
Dependent Claim
55 words
The method according to claim 27 , said method further including stabilizing pixels of the refined image to produce a stabilized image, wherein said stabilizing pixels comprises determining a value for a pixel in the stabilized image according to a classification history of a corresponding pixel in the refined image over a window of time.
Claim 33
Dependent Claim
86 words
The method according to claim 28 , wherein a first candidate for a value of a pixel in an output image is a value of a corresponding pixel in the original image, and wherein a second candidate for the value of the pixel in the output image is a value of a corresponding pixel in a background image, and wherein the value of the pixel in the output image is determined at least in part by a value of a corresponding pixel in the stabilized image.
Claim 34
Dependent Claim
273 words
A data storage medium having machine-readable code stored thereon, the machine-readable code comprising instructions executable by an array of logic elements, the instructions defining a method comprising filtering an input image, said filtering comprising: calculating a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, wherein the center Pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; calculating smoothing factors, each smoothing factor corresponding to one among the gradients; and calculating a filtered value for the center pixel, the filtered value being expressed in terms of at least a center pixel contribution and a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (a) one among the smoothing factors and (b) a value of a pixel that lies in the direction corresponding to the smoothing factor, wherein said filtering is performed to process each among substantially all of the pixels of an input image as a center pixel, and wherein a filtered image comprises the filtered values calculated for said substantially all of the pixels of the input image.
Claim 35
Dependent Claim
273 words
A data storage medium having machine-readable code stored thereon, the machine-readable code comprising instructions executable by an array of logic elements, the instructions defining a method comprising filtering an input image, said filtering comprising: calculating a gradient in each among a plurality of directions, including at least a vertical, a horizontal, and two diagonal directions, the directions being defined relative to a center pixel, wherein the center Pixel has eight eight-neighbor pixels, and wherein, for each of the directions, the gradient calculator calculates a corresponding gradient as a difference of values of only two of the eight-neighbor pixels for a corresponding direction; calculating smoothing factors, each smoothing factor corresponding to one among the gradients; and calculating a filtered value for the center pixel, the filtered value being expressed in terms of at least a center pixel contribution and a neighborhood contribution, wherein the center pixel contribution is expressed at least in terms of an original value for the center pixel and at least one smoothing weight, the smoothing weight being expressed at least in terms of one of the smoothing factors, and wherein the neighborhood contribution is expressed at least in terms of a smoothing product, wherein the smoothing product is expressed at least in terms of (a) one among the smoothing factors and (b) a value of a pixel that lies in the direction corresponding to the smoothing factor, wherein said filtering is performed to process each among substantially all of the pixels of an input image as a center pixel, and wherein a filtered image comprises the filtered values calculated for said substantially all of the pixels of the input image.
Claim 36
Dependent Claim
107 words
A method according to claim 30 , said method further comprising producing an output image, wherein the input image is a segmentation map of an original image, and wherein a first candidate for a value of a pixel in the output image is a value of a corresponding pixel in the original image, and wherein a second candidate for the value of the pixel in the output image is a value of a corresponding pixel in a background image, and wherein the value of the pixel in the output image is determined at least in part by a value of a corresponding pixel in the filtered image.
References Cited (Backward Citations)
US4809349A
Cited
US5034986A
Cited
US5390264A
Cited
US6005582A
Cited
US5819035A
Cited
US5914748A
Cited
US6011595A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-03-16 | 2007-09-15 | 2008-03-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-03-16 | 2011-09-15 | 2012-03-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-03-16 | 2015-09-15 | 2016-03-16 | $7700.00 | Window Closed / Lapsed |