Patent US7062083B2 successfully refreshed from Google Patents.
US7062083B2
Expired / Lapsed
US
⏱️ 0.0 years remaining
Image retrieval method based on combination of color and texture features
Assignee: Samsung Electronics Co Ltd
•
Filing: 2002-01-09
•
Granted: 2006-06-13
•
Expires: 2022-01-09
Abstract
Abstract An image retrieval method with improved performance in which a data image similar to a query image is retrieved by appropriately combining color and texture features. The method for retrieving a data image similar to a query image in an image database containing a plurality of data images involves: calculating color and texture distances between a query image and each data image in the image database; weighting the calculated color and texture distances with predetermined weighting factors; calculating a feature distance between the query image and each data image by combining the weighted color and texture distances by considering human visual perception attributes; and determining the data image similar to the query image using the feature distance. According to the image retrieval method, data images similar to a query image can be retrieved based on the human visual perception mechanism by combining both color and texture features extracted from image regions. In particular, the image region based retrieval enables accurate retrieval of more objects and many kinds of information from a single image.
Inventors
- 👤 Choi Yang Lim
- 👤 Cho Nam Ik
- 👤 Kwak Jung Won
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 11:39
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/583; G06F16/5862; G06F16/5838; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933; Y10S707/99936
Claims Analysis
Total Claims: 36
Claim 1
Independent Claim
110 words
A method for retrieving a data image similar to a query image using a feature distance calculated by combining one or more weighted color distances and one or more weighted texture distances by considering human visual perception attributes, wherein the one or more weighted color distances and the one or more weighted texture distances are obtained by applying predetermined weighting factor to each texture distance and to each color distance, and wherein the predetermined weighting factor applied to said each texture distance is determined based on an area of an initial sample region extracted from the iuery image and an area of a sample region extracted from each data image.
Claim 2
Independent Claim
110 words
A method for retrieving a data image similar to a query image using a feature distance calculated by combining one or more weighted color distances and one or more weighted texture distances by considering human visual perception attributes, wherein the one or more weighted color distances and the one or more weighted texture distances are obtained by applying predetermined weighting factor to each texture distance and to each color distance, and wherein the predetermined weighting factor applied to said each texture distance is determined based on an area of an initial sample region extracted from the iuery image and an area of a sample region extracted from each data image.
Claim 3
Dependent Claim
59 words
The method according to claim 1 , wherein said combining of the one or more weighted color distances and the one or more weighted texture distances comprises weighing the one or more weighted color distances and the one or more weighted texture distances with differing weighing factors, said variation of the weighting factors reflect the human visual perception attributes.
Claim 4
Independent Claim
162 words
A method for retrieving a data image similar to a query image in an image database containing a plurality of data images, the method comprising: (a) calculating a plurality of color distances and a plurality of texture distances between a query image and each data image in the image database; (b) weighting each of the calculated color distances and texture distances with a respective predetermined first weighting factor; (c) calculating a feature distance between the query image and each data image by combining the weighted color distances and the weighted texture distances by applying a second set of differing weighing factors that reflect human visual perception attributes; and (d) determining the data image similar to the query image using the feature distances, wherein the first predetermined weighting factor applied to said each texture distance is determined based on an area of an initial sample region extracted from the query image and an area of a sample region extracted from each data image.
Claim 5
Independent Claim
162 words
A method for retrieving a data image similar to a query image in an image database containing a plurality of data images, the method comprising: (a) calculating a plurality of color distances and a plurality of texture distances between a query image and each data image in the image database; (b) weighting each of the calculated color distances and texture distances with a respective predetermined first weighting factor; (c) calculating a feature distance between the query image and each data image by combining the weighted color distances and the weighted texture distances by applying a second set of differing weighing factors that reflect human visual perception attributes; and (d) determining the data image similar to the query image using the feature distances, wherein the first predetermined weighting factor applied to said each texture distance is determined based on an area of an initial sample region extracted from the query image and an area of a sample region extracted from each data image.
Claim 6
Dependent Claim
56 words
The method of claim 3 , before step (a), further comprising: (pa-1) segmenting the query image and each data image into a plurality of first regions using a plurality of color features; and (pa-2) determining a plurality of sample regions in the query image and each data image for extraction of a plurality of texture features.
Claim 7
Dependent Claim
55 words
The method of claim 4 , wherein step (a) comprises: (a1) generating a plurality of color vectors of the first regions using the color features and calculating a plurality of color distances; and (a2) generating a plurality of texture vectors of the sample regions using the texture features and calculating a plurality of texture distances.
Claim 8
Dependent Claim
39 words
The method of claim 5 , wherein the color features are expressed by a color feature descriptor f c (I k ) with a representative color vector and a percentage of the representative color vector for each first region.
Claim 9
Dependent Claim
119 words
The method of claim 6 , wherein the color feature descriptor f c (I k ) is expressed by: f c ( I k )={ {overscore (c)} k1 , p ki ), ( {overscore (c)} k2 , p k2 ), . . . , ( {overscore (c)} KN k , pKN K )} wherein k is a positive integer indicating a serial number of each region, {overscore (c)} ki is an i-th representative color vector of a k-th region i=(1, 2, . . . , N K ), P ki is a percentage of the ith color representative color vector in the k-th region, and N k is the number of the representative color vectors in the k-th region.
Claim 10
Dependent Claim
190 words
The method of claim 7 , wherein the color distance between each color vector of each region of the query image and each color vector of each region of each data image is calculated using: d c ( I 1 , I 2 ) = ∑ i = 1 N 1 p 1 i 2 + ∑ i = 1 N 2 p 2 i 2 - ∑ i = 1 N 1 ∑ j = 1 N 2 2 a 1 i , 2 j p 1 i p 2 j wherein when T d is a maximum distance by which a similarity of two colors is determined, α is a predetermined scaling coefficient, d max =α α T d , and d ij is the Euclidean distance ∥c i –c j ∥ between two color vectors c i , and c j , such that a ij = 1 - d ij d max if d ij ≦T d , and α lj =0 if d lj >T d .
Claim 11
Dependent Claim
13 words
The method of claim 5 , wherein step (a2) uses a Gabor function.
Claim 12
Dependent Claim
42 words
The method of claim 9 , wherein, in step (a2), the texture vectors of the plurality of sample regions are generated using the Gabor function having N frequency channels and M orientation channels, where N and M are both predetermined positive integers.
Claim 13
Dependent Claim
159 words
The method of claim 10 , wherein the texture features are expressed by a texture feature descriptor f t (I k ): f t ( I k )={( m k11 , σ k11 ), ( m k12 , σ k12 ), . . . , ( m k1M , σ k1M ), ( m k21 , σ k21 ), . . . , ( m kij , σ kij ), . . . , ( m kNM , σ kNM )} wherein m kij is a mean value of a plurality of pixel values of the i-th frequency channel and the j-th wherein m kiJ is a mean value of a plurality of pixel values of the i-th frequency channel and the j-th orientation channel for a sample region I k , and σ kij is a deviation of the pixel values of the i-th frequency channel and the j-th orientation channel for the sample region I k .
Claim 14
Dependent Claim
134 words
The method of claim 11 , wherein, in step (a2), the texture distance between each color vector of each region of the query image and each color vector of each sample region of each data image is calculated using: d t ( I 1 , I 2 ) = ∑ i = 1 N ∑ j = 1 M m 1 ij - m 2 ij σ ( m ij ) + ∑ i = 1 N ∑ j = 1 M σ 1 ij - σ 2 ij σ ( σ ij ) where σ(m ij ) and σ(σ ij ) are the standard deviations for all m ij and σ ij , respectively.
Claim 15
Dependent Claim
24 words
The method of claim 4 , wherein step (pa-1) comprises quantizing a plurality of color vectors of the query image and each data image.
Claim 16
Dependent Claim
54 words
The method of claim 13 , wherein quantizing the color vectors comprises: (pa-1-1) performing a peer group filtering on the query image and each data image for noise removal and smoothing effects; and (pa-1-2) clustering a plurality of filtered pixel values of the query image and each data image using a generalized Lloyd algorithm.
Claim 17
Dependent Claim
27 words
The method of claim 14 , wherein step (pa-1-2) comprises applying a predetermined algorithm to increase the number of resulting clusters or to merge the resulting clusters.
Claim 18
Dependent Claim
107 words
The method of claim 13 , further comprising: defining a J-value indicating a color uniformity in each pixel of a plurality of pixels of the query image and each data image, which have undergone quantization; storing the J-value in each pixel of the query image and each data image to obtain a plurality of J-images; segmenting each J-image into a plurality of second regions by a predetermined segmentation method; repeating the segmentation of each J-image to obtain a map of one or more over-segmented regions for each J-image; and obtaining a final map for each J-image by merging the over-segmented regions based on a correlation of color.
Claim 19
Dependent Claim
28 words
The method of claim 16 , further comprising indexing a feature vector space by a representative color and a percentage of the representative color in each second region.
Claim 20
Dependent Claim
46 words
The method of claim 17 , wherein indexing the feature vector space comprises: assigning a plurality of representative colors to a plurality of grid points in a color space having a grid structure; and storing the result of the assignment as a table in a database.
Claim 21
Dependent Claim
42 words
The method of claim 4 , wherein step (pa-2) comprises: (pb-1-) obtaining a grid map of the query image and each data image; and (pb-2-) obtaining a texture sample of a desired size for each sample region based on the grid map.
Claim 22
Dependent Claim
176 words
The method of claim 19 , wherein step (pb-1) comprises: (pb-1-1) taking a rectangle M(i, j) having largest side lengths for a sample region, wherein 0≦i≦n and 0≦j≦m; (pb-1-2) dividing the rectangle M(i, j) into a plurality of sub-rectangles each having a l×l size; and (pb-1-3) obtaining a grid map for the sample region using a predetermined function which outputs 1 if all the divided grids belong to the sample region, and otherwise, outputs 0, the predetermined function expressed by: M grid ( i , j ) = { 1 if p ij ( x , y ) ∈ I k for ∀ ( x , y ) 0 otherwise wherein 0 ≤ i ≤ [ n l ] - 1 , being a result of applying the floor function, 0 ≤ j ≤ [ m l ] - 1 where [m/l] is a result of applying the ceiling function, 0≦x≦l −1, 0≦y≦l −1, and p ij (x, y) is a point that belongs to rectangle M(il+x, jl+y).
Claim 23
Dependent Claim
200 words
The method of claim 20 , after step (pb-1-3), further comprising: (ph-1-4) transforming the grid map obtained in step (pb-1-3) to a grid distance map by repeating computations with: { M grid d ( i, j )} 0 =M grid ( i, j ),and { M grid d ( i, j )} n =min({ M grid d ( i, j −1)} n-1 , {M grid d ( i −1 , j )} n-1 , {M grid d ( i, j +1)} n-1 , {M grid d ( i +1 , j )} n-1 )+1, if { M grid d ( i, j )} n-1 ={M grid d ( i, j )} n , M grind d ( i, j )= {M grid d ( i, j )} n , and (pb-1-5) performing region growing in every direction from a seed of the grid map, which has the largest distance in the grid and is expressed by ( a , b ) = arg max ( i , j ) { M grid d ( i , j ) } , to satisfy the relation M grid d (i, j)<0 until a maximum area rectangle is extracted.
Claim 24
Dependent Claim
27 words
The method of claim 21 , further comprising fitting the maximum area rectangle extracted in step (pb-1-5) to the desired size of the texture sample by wrapping.
Claim 25
Dependent Claim
27 words
The method of claim 21 , further comprising fitting the maximum area rectangle extracted in step (pb-1-5) to the desired size of the texture sample by mirroring.
Claim 26
Dependent Claim
64 words
The method of claim 4 , wherein step (b) comprises: (b-1) placing each of the color distances and the texture distances in a 2-dimensional vector space, each vector space defined by the respective distances and associated predetermined weighting factors; and (b-2) projecting the result of the placement onto the 2-dimensional vector spaces onto a 1-dimensional distance space based on the human visual perception mechanism.
Claim 27
Dependent Claim
62 words
The method of claim 24 , wherein the predetermined weighting factor to the color distance is determined by: ω c = 1 - ∑ i = 1 N p qi log 10 ( 1 p qi ) wherein p qi is a percentage of an i-th representative color of one of the first regions of the query image.
Claim 28
Dependent Claim
140 words
The method of claim 24 , wherein each texture distance is defined by: d t ′ ( I q , I l ) = a ( A ( s ) A ( I l ) + ( count ( I l ) - 1 ) d t ( I q , I l ) wherein I q denotes the query image or each first region of the query image, s denotes a sample region of a desired size, A(·) denotes the area of the sample region of the desired size, count(·) is the number of wrappings done to obtain the sample region of the desired size, and α is a constant, and where d t (I q ,I l ) is a distance function for two images based on their texture vectors.
Claim 29
Dependent Claim
44 words
The method of claim 24 , wherein the predetermined weighting factor applied to each texture distance is determined based on an area of an initial sample region extracted from the query image and the area of a sample region extracted from each data image.
Claim 30
Dependent Claim
105 words
The method of claim 27 , wherein the predetermined weighting factor applied to each texture distance is determined by: ω t = 1 a ( A ( s ) A ( I q ) + ( count ( I q ) - 1 ) ) wherein I q denotes the query image or each first region of the query image, s denotes a sample region of a desired size, A(·) denotes the area of the sample region of the desired size, count(·) is the number of wrappings done to obtain the sample region of the desired size, and α is a constant.
Claim 31
Dependent Claim
21 words
The method of claim 24 , before step (b-2), further comprising normalizing each of the color distances and the texture distances.
Claim 32
Dependent Claim
23 words
The method of claim 29 , wherein normalizing each of the color distances and the texture distances is performed using a Gaussian normalization.
Claim 33
Dependent Claim
97 words
The method of claim 30 , wherein normalizing each of the color distances and the texture distances comprises: wherein V k is a Gaussian sequence, performing a normalization within a range of [−1, 1] using v m , k = v m , k - μ k 3 σ k based on a mean value μ k and a deviation σ k of the sequence V k ; and mapping the result of the normalization into a range of [0, 1] using v m , k ′ = v m , k + 1 2 .
Claim 34
Dependent Claim
68 words
The method of claim 31 , wherein normalizing each of the color distances and the texture distances comprises normalizing the texture distances by updating the mean value μ t , and the deviation σ t of the texture distances by excluding a largest texture distance until the condition of k × μ c σ c ≤ μ t σ t , wherein k is a constant, is satisfied.
Claim 35
Dependent Claim
212 words
The method of claim 24 , wherein step (b-2) comprises projecting the color distances and the texture distances onto a 1-dimensional distance space using: d ( I q , I l ) = W c d c ( I q , I l ) ( 1 + 2 π tan - 1 d t ′ ( I q , I l ) d c ( I q , I l ) ) + W t d t ′ ( I q , I l ) ( 1 + 2 π tan - 1 d c ( I q , I l ) d t ′ ( I q , I l ) ) wherein W c = ω c ω c + ω t and W t = ω t ω c + ω t . wherein ω c and ω t are weighing factors of the color distances and texture distances respectively, where d c (I q , I l ) and where d t ′ (I q , I l ) represent normalized distance functions for two image regions based on their color and texture vectors, respectively.
Claim 36
Dependent Claim
24 words
The method of claim 3 , wherein the predetermined weighting factor to the color distance is determined based on a distribution of representative colors.
References Cited (Backward Citations)
US5083571A
Cited
US5999647A
Cited
US6131082A
Cited
JPH1196364A
Cited
US6519360B1
Cited
US6345274B1
Cited
JP2000048181A
Cited
US6463432B1
Cited
JP2000216979A
Cited
WO2000045337A1
Cited
US6693962B1
Cited
WO2000046748A1
Cited
WO2000065839A1
Cited
JP2000348179A
Cited
US6801657B1
Cited
WO2000070881A1
Cited
US6563959B1
Cited
US6865302B2
Cited
US20020006221A1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2009-06-12 | 2009-12-12 | 2010-06-13 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2013-06-12 | 2013-12-12 | 2014-06-13 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2017-06-12 | 2017-12-12 | 2018-06-13 | $7700.00 | Window Closed / Lapsed |