US7171023B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Illumination-invariant object tracking method and image editing system using the same
Assignee: Samsung Electronics Co Ltd
•
Filing: 2002-11-05
•
Granted: 2007-01-30
•
Expires: 2022-11-05
Abstract
Abstract An illumination-invariant object tracking method and an image editing system using the same are provided. The illumination-invariant object tracking method involves: designating an initial position of an object of interest to be tracked in an image; constructing a probability map for object tracking in consideration of a color ratio of adjacent pixels in the image; and performing Kalman filtering based on the initial position of the object and the probability map. The image editing system based on the illumination-invariant object tracking method includes: a data input unit which receives an image to be edited; an object tracking unit which tracks a target object in consideration of a color ratio of adjacent pixels in the image; an image editing unit which edits and outputs an edited image in a predetermined region of the image, in response to information on the position, size, and boundary of the tracked target object.
Inventors
- 👤 Tae-Kyun Kim
- 👤 In-myung Cho
- 👤 Jong-ha Lee
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 13:41
CPC Classifications
G; G06; G06T; G06T7/00; G06T7/20; G06T7/246; G06T7/277; G06V; G06V10/00; G06V10/20; G06V10/24
Claims Analysis
Total Claims: 24
Claim 1
Independent Claim
74 words
An illumination-invariant object tracking method comprising: (a) designating an initial position of an object of interest to be tracked in an image; (b) modeling color information on the image in consideration of a color ratio of adjacent pixels in the image; (c) constructing a probability map based on the result of the color modeling; and (d) tracking a current position of the target object in response to the initial position and the probability map.
Claim 2
Dependent Claim
74 words
An illumination-invariant object tracking method comprising: (a) designating an initial position of an object of interest to be tracked in an image; (b) modeling color information on the image in consideration of a color ratio of adjacent pixels in the image; (c) constructing a probability map based on the result of the color modeling; and (d) tracking a current position of the target object in response to the initial position and the probability map.
Claim 3
Dependent Claim
115 words
The illumination-invariant object tracking method of claim 1 ,wherein (a) designating the initial position of the object of interest comprises: (a-1) determining whether to manually or automatically designate the initial position of the object; (a-2) if it is determined in (a-1) to manually designate the initial position of the object, a user directly inputting the initial position of the object; and (a-3) if it is determined in (a-1) to automatically designate the initial position of the object, reading a color histogram of the object from a data storing unit where color histrograms for a plurality of objects have been stored and automatically designating the initial position of the object by back-projecting the read color histogram.
Claim 4
Dependent Claim
140 words
The illumination-invariant object tracking method of claim 2 , wherein denoting a disk having a radius r as Dr, (a-3) automatically designating the initial position of the object by back-projecting the read color histogram comprises: (a-3-1) outputting a smaller value between the quotient of dividing the color histogram of a previous model for each histogram bin by the color histogram of a current image for each histogram bin and 1; (a-3-2) performing the operation of (a-3-1) on all pixels in the image; (a-3-3) convolving the result of the operation of (a-3-2) with Dr; and (a-3-4) tracking a coordinate having the largest convolution result and designating the coordinate as the initial position of the object; D r ( x,y )=1 if √{square root over ( x 2 +y 2 )}0 otherwise wherein where (x, y) denotes a coordinate in the image.
Claim 5
Dependent Claim
50 words
The illumination-invariant image tracking method of claim 1 , wherein (b) constructing the probability map comprises: (b-1) modeling color information on the object in consideration of the color ratio of adjacent pixels; and (b-2) modeling color information on the entire image in consideration of the color ratio of adjacent pixels.
Claim 6
Dependent Claim
117 words
The illumination-invariant image tracking method of claim 4 ,wherein (b-1) modeling the color information on the object comprises: (b-1-1) analyzing color components of the object; (b-1-2) if the object is determined to be polychromatic as the result of the analysis (b-1-1), constructing a 3-dimensional histogram of the object with an axis of the brightness ratio for each of R, G, and B channels; (b-1-3) if the object is determined to be monochromatic as the result of the analysis (b-1-1), defining a color probability for the object by 2-dimensional Gaussian modeling for the hue and saturation of the object; and (b-1-4) constructing a color histogram of the object with an axis of the color probability defined in (b-1-3).
Claim 7
Dependent Claim
165 words
The illumination-invariant image tracking method of claim 5 , wherein in (b-1-1), a value of 1 S ∑ i , j { ( R ( i , j ) - u R ) 2 + ( G ( i , j ) - u G ) 2 + ( B ( i , j ) - u B ) 2 } , where S indicates the number of (i,j) pairs, U R indicates an average value of red (R) image data of the (i,j) pairs, U G indicates an average value of green (G) image data of the (i,j) pairs, and U B indicates an average value of blue (B) image data of the (i,j) pairs, is calculated, the object is determined to be monochromic if the result of the calculation is smaller than a predetermined threshold, and the object is determined to be polychromatic if the result of the calculation is greater than or equal to the predetermined threshold.
Claim 8
Dependent Claim
142 words
The illumination-invariant image tracking method of claim 5 , wherein in (b-1-3), the color probability is defined using the following equation: P skin ( x , y ) = ∑ i = x - n / 2 i = x + n / 2 ∑ j = x - n / 2 j = x + n / 2 g ( Hue ( i , j ) , Sat ( i , k ) : u → , Σ ) n 2 where u _ = [ 175 - 0.6 150 I + 1.0 ] , Σ = [ 30 2 0 0 σ sat 2 ] , g(i,j;{right arrow over (u)},Σ)denotes a 2-dimensional Gaussian function, n denotes a size of the object, and I denotes an average brightness intensity of the image.
Claim 9
Dependent Claim
105 words
The illumination-invariant object tracking method of claim 4 , wherein (b-2) modelling the color information on the entire image comprises: (b-2-1) measuring color variations of the entire image; (b-2-2) if the object is determined to be illuminated by white light as the result of (b-2-1), constructing a color distribution histogram of the entire image irrespective of the result of the object color information modelling in (b-1); and (b-2-3) if the object is determined to be illuminated by colored light as the result of (b-2-1), re-performing (b-1) on an object tracked in a previous image frame and constructing a color distribution histogram of the entire image.
Claim 10
Dependent Claim
162 words
The illumination-invariant object tracking method of claim 8 , wherein in (b-2-1), a value of 1 S ∑ i , j { ( R n R n + G n + B n - R m R m + G m + B m ) 2 ( G n R n + G n + B n - G m R m + G m + B m ) 2 } , where S indicates the number of (i,j) pairs, n indicates a current frame image, m indicates a previous frame image, and R, G, and B indicate red, green, and blue data of the image, respectively, is calculated, the object is determined to be illuminated by the white light if the result of the calculation is smaller than a predetermined threshold, and the object is determined to be illuminated by the colored light if the result of the calculation is greater than or equal to the predetermined threshold.
Claim 11
Dependent Claim
74 words
The illumination-invariant object tracking method of claim 4 , wherein the probability map is expressed as P ( object color ) = h ratio ( color ) = h object ( color ) h total ( color ) where h object (color) indicates the result of the color information modelling of the object and h total (color) indicates the result of the color information modelling of the entire image.
Claim 12
Dependent Claim
78 words
The illumination-invariant image tracking method of claim 1 , wherein the step (d) comprises: (d-1) storing a previous position of the object; (d-2) performing Kalman filtering based on the probability map and the initial position of the object for tracking the current position and size of the object of interest in a series of pictures; and (d-3) updating a covariance matrix of a Kalman filter in response to the previous position and the current position of the object.
Claim 13
Dependent Claim
18 words
A computer readable physical media having embodied thereon a computer program for the method of claim 1 .
Claim 14
Dependent Claim
99 words
An object tracking system comprising: an initial position designating portion which designates an initial position of a target object to be tracked in an image; a color modelling portion which performs a color modelling on the target object and the entire image in consideration of a color ratio of adjacent pixels in the image; a probability map constructing portion which constructs a probability map for object tracking based on the result of the color modelling; and an object tracking portion which tracks a current position of the target object in response to the initial position and the probability map.
Claim 15
Dependent Claim
99 words
An object tracking system comprising: an initial position designating portion which designates an initial position of a target object to be tracked in an image; a color modelling portion which performs a color modelling on the target object and the entire image in consideration of a color ratio of adjacent pixels in the image; a probability map constructing portion which constructs a probability map for object tracking based on the result of the color modelling; and an object tracking portion which tracks a current position of the target object in response to the initial position and the probability map.
Claim 16
Dependent Claim
94 words
The object tracking system of claim 13 , wherein the initial position designating portion comprises: a first initial position designator which receives a predetermined position designated by a user as the initial position of the target object; an object database which stores a list of objects and color modelling data for the objects; and a second initial position designator which reads the color modelling data for the target object designated by the user from the object database and automatically designates the initial position of the target object by back-projecting the read color modelling data.
Claim 17
Dependent Claim
60 words
The object tracking system of claim 13 , wherein the color modelling portion comprises: a first color modeler which models color information on the target object; a second color modeler which models color information on the entire image; and a third color modeler which adaptively models the color information on the target object based on illumination variations of the image.
Claim 18
Dependent Claim
111 words
The object tracking system of claim 15 , wherein the first color modeler analyzes color components of the target object, constructs a 3-dimensional histogram of the target object with an axis of the brightness ratio for each of R, 0 ,and B channels if the target object is determined to be polychromatic as the result of the analysis, and defines a color probability for the target object by 2-dimensional Gaussian modeling for the hue and saturation of the target object and constructs a color histogram of the target object with an axis of the color probability if the target object is determined to be monochromatic as the result of the analysis.
Claim 19
Dependent Claim
165 words
The object tracking system of claim 16 , wherein the first color modeler calculates a value of 1 S ∑ i , j { ( R ( i , j ) - u R ) 2 + ( G ( i , j ) - u G ) 2 + ( B ( i , j ) - u B ) 2 } , where S indicates the number of (i,j) pairs, U R indicates an average value of red (R) image data of the (i,j) pairs, U G indicates an average value of green (G) image data of the (i,j) pairs, and U B indicates an average value of blue (B) image data of the (i,j) pairs, determines the target object to be monochromic if the result of the calculation is smaller than a predetermined threshold, and determines the target object to be polychromatic if the result of the calculation is greater than or equal to the predetermined threshold.
Claim 20
Dependent Claim
146 words
The object tracking system of claim 16 , wherein the first color modeler defines the color probability of the target object using the following equation: P skin ( x , y ) = ∑ i = x - n / 2 i = x + n / 2 ∑ j = x - n / 2 j = x + n / 2 g ( Hue ( i , j ) , Sat ( i , k ) : u → , Σ ) n 2 where u _ = [ 175 - 0.6 150 I + 1.0 ] , Σ = [ 30 2 0 0 σ sat 2 ] , g(i,j;{right arrow over (u)},Σ)denotes a 2-dimensional Gaussian function, n denotes a size of the object, and I denotes an average brightness intensity of the image.
Claim 21
Dependent Claim
63 words
The object tracking system of claim 15 , wherein the third color modeler measures color variations of the entire image and re-performs the color modelling on an object tracked in a previous image and constructs a color distribution histogram of the entire image if the target object is determined to be illuminated by colored light as the result of the color variation measurement.
Claim 22
Dependent Claim
161 words
The object tracking system of claim 19 , wherein the third color modeler calculates a value of 1 S ∑ i , j { ( R n R n + G n + B n - R m R m + G m + B m ) 2 ( G n R n + G n + B n - G m R m + G m + B m ) 2 } , where S indicates the number of (i,j) pairs, n indicates a current frame image, m indicates a previous frame image, and R, G, and B indicate red, green, and blue data of the image, respectively, determines the target object to be illuminated by white light if the result of the calculation is smaller than a predetermined threshold, and determines the target object to be illuminated by the colored light if the result of the calculation is greater than or equal to the predetermined threshold.
Claim 23
Dependent Claim
81 words
The object tracking system of claim 13 , wherein the probability map constructing portion calculates the probability map as the following equation P ( object color ) = h ratio ( color ) = h object ( color ) h total ( color ) where h object (color) indicates the result of the color information modelling of the target object and h total (color) indicates the result of the color information modelling of the entire image.
Claim 24
Dependent Claim
82 words
The object tracking system of claim 13 , wherein the object tracking portion comprises: a memory for storing a previous position of the object; a Kalman filter for tracking the current position and size of the object of interest in a series of pictures, based on the probability map and the Initial position of the object; and a velocity predictor for updating a covariance matrix of the Kalman filter in response to the previous position and the current position of the object.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2010-01-29 | 2010-07-31 | 2011-01-30 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2014-01-29 | 2014-07-31 | 2015-01-30 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2018-01-29 | 2018-07-31 | 2019-01-30 | $7700.00 | Window Closed / Lapsed |