US6744935B2 — Content-based image retrieval apparatus and metho…

Claim 1

Independent Claim 373 words

A content-based image retrieval apparatus via relevance feedback by using fuzzy integral, the apparatus comprising: image input means for receiving images from a user; feature extraction means for extracting image features necessary for image similarity operation from the images inputted via said image input means; first storage means for storing the image features extracted by said feature extraction means; thumb nail generating means for dividing the images inputted via said image input means to generate thumb nails; second storage means for storing the thumb nails generated by said thumb nail generating means; similarity calculation means for calculating similarities between a query image and the images based on the image features stored on said first storage means, to thereby generate individual similarities for each image feature; similarity measuring means for performing fuzzy integral of the individual similarities calculated via said similarity calculation means, to thereby measure final image similarities; query processing means for interpreting the query image from the user to measure similarities, and for sequentially calling and transmitting the thumb nails stored on said second storage means according to the calculated similarities; user interface means for performing relevance feedback about the image data transmitted from said query processing means, and for transmitting selective information according to relevance feedback to said query processing means wherein the final image similarities are measured based on an equation as: C g ( S, i, j )= k=1, . . . , n [S (k) ( x i(k) , x j(k) )− S (k−1) ( x i(k−1) , x j(k−1) ) g ( A (k) ),] where S={s 1 , . . . , s n } is a set of similarity measurement functions between two images, X={x 1 , . . . , x n } is a set of image features, x i(k) and x j(k) are k th image features of query images i and j, n is the total number of used image features, g(A (k) ), which is also expressed as g k , denotes relevance of a feature subset A (k) ={x 1 , . . . , x (k) }, and 0≦s (1) (x i(1) , x j(1) )≦ . . . ≦s (n) (x i(n) , x j(n) ) ≦1.

Claim 2

Dependent Claim 373 words

A content-based image retrieval apparatus via relevance feedback by using fuzzy integral, the apparatus comprising: image input means for receiving images from a user; feature extraction means for extracting image features necessary for image similarity operation from the images inputted via said image input means; first storage means for storing the image features extracted by said feature extraction means; thumb nail generating means for dividing the images inputted via said image input means to generate thumb nails; second storage means for storing the thumb nails generated by said thumb nail generating means; similarity calculation means for calculating similarities between a query image and the images based on the image features stored on said first storage means, to thereby generate individual similarities for each image feature; similarity measuring means for performing fuzzy integral of the individual similarities calculated via said similarity calculation means, to thereby measure final image similarities; query processing means for interpreting the query image from the user to measure similarities, and for sequentially calling and transmitting the thumb nails stored on said second storage means according to the calculated similarities; user interface means for performing relevance feedback about the image data transmitted from said query processing means, and for transmitting selective information according to relevance feedback to said query processing means wherein the final image similarities are measured based on an equation as: C g ( S, i, j )= k=1, . . . , n [S (k) ( x i(k) , x j(k) )− S (k−1) ( x i(k−1) , x j(k−1) ) g ( A (k) ),] where S={s 1 , . . . , s n } is a set of similarity measurement functions between two images, X={x 1 , . . . , x n } is a set of image features, x i(k) and x j(k) are k th image features of query images i and j, n is the total number of used image features, g(A (k) ), which is also expressed as g k , denotes relevance of a feature subset A (k) ={x 1 , . . . , x (k) }, and 0≦s (1) (x i(1) , x j(1) )≦ . . . ≦s (n) (x i(n) , x j(n) ) ≦1.

Claim 3

Dependent Claim 41 words

The apparatus as recited in claim 1 , wherein the fuzzy integral is operated by studying and initializing the fuzzy measurement about individual image features and then studying special interactions to perform the fuzzy measurement necessary for fuzzy via relevance feedback.

Claim 4

Dependent Claim 281 words

The apparatus as recited in claim 2 , wherein the fuzzy studying of the individual features is performed by establishing fuzzy rules based on representative similarity differences and by measuring the fuzzy integral, wherein the fuzzy rules are established based on equations as: If Δ s k is positive, then g k =1 Else if Δ s k is negative, then g k =0, and Else if Δ s k is zero, then g k =g k ; Δs k is a difference between ps k and ns k for the kth image feature x k , ps k is an average similarity of the query image q and the image i belonging to a related image set R + , ns k is an average similarity of the query image q and the image j belonging to an unrelated image set R − , wherein the fuzzy integral is measured based on an equation as: g k =( p (Δ s k )*1.0+ n (Δ s k )*0.0+ g k *z (Δ s k ))/( p (Δ s k )+ n (Δ s k )+ z (Δ s k ))= ( p (Δ s k )+ g k *z (Δ s k ))/( p (Δ s k )+ n (Δ s k )+ z (Δ s k )) where (p(Δ s k ) is a membership representing the degree that Δ s k belongs to the fuzzy set positive, n(Δ s k ) is a membership representing the degree that Δ s k belongs to the fuzzy set negative, and z(Δ s k ) is a membership representing the degree that Δ s k belongs to the fuzzy set zero.

Claim 5

Dependent Claim 153 words

The apparatus as recited in claim 2 , wherein the studying of the special interactions is performed by obtaining the fuzzy measurement whereby the following equation is minimized, and studying while satisfying monotonicity which is a fuzzy measurement condition, wherein the equation is: J ( g )=Σ iεR+ [Cg ( S, q, i )− s (n) ( x q(n) , x i(n) )] 2 +Σ iεR− [Cg ( S, q, j )− s (1) ( x q(1) , x j(1) )] 2 where s (n) (x q(n) , x i(n) ) is the maximum of the similarities s 1 (.), . . . , s n (.) according to feature between the image i which belongs to R + and the query image q, and s (1) (x q(1) , x j(1) ) is the minimum of the similarities between the image j which belongs to R − and the query image q.

Claim 6

Dependent Claim 245 words

A content-based image retrieval method via relevance feedback by using fuzzy integral, comprising: measuring similarities according to feature between images stored in a database and a new query image; associating the similarities measured according to feature by using the fuzzy integral to measure similarities according to the fuzzy integral; bringing images sequentially in the order of higher similarity according to the finally obtained similarities and transmitting a search result; and recalculating similarities according to feature via relevance feedback about the query result to output converted images wherein the final image similarities are measured based on an equation as: C g ( S, i, j )= k=1 , . . . , n [S (k) ( x i(k) , x j(k) )− S (k−1) ( x i(k−1) , x j(k−1) ) g ( A (k) )] where S={s 1 , . . . , s n } is a set of similarity measurement functions between two images, X={x 1 , . . . x n } is a set of image features, x i(k) and x j(k) are k th image features of query images i and j, n is a total number of used image features, g(A (k) ), which is also expressed as g k , denotes relevance of a feature subset A (k) ={x (1) , . . . , x (k) }, and 0≦s (1) (x i(1) , x j(1) )≦ . . . ≦s (n) (x i(n) , x j(n) )≦1.

Claim 7

Dependent Claim 245 words

A content-based image retrieval method via relevance feedback by using fuzzy integral, comprising: measuring similarities according to feature between images stored in a database and a new query image; associating the similarities measured according to feature by using the fuzzy integral to measure similarities according to the fuzzy integral; bringing images sequentially in the order of higher similarity according to the finally obtained similarities and transmitting a search result; and recalculating similarities according to feature via relevance feedback about the query result to output converted images wherein the final image similarities are measured based on an equation as: C g ( S, i, j )= k=1 , . . . , n [S (k) ( x i(k) , x j(k) )− S (k−1) ( x i(k−1) , x j(k−1) ) g ( A (k) )] where S={s 1 , . . . , s n } is a set of similarity measurement functions between two images, X={x 1 , . . . x n } is a set of image features, x i(k) and x j(k) are k th image features of query images i and j, n is a total number of used image features, g(A (k) ), which is also expressed as g k , denotes relevance of a feature subset A (k) ={x (1) , . . . , x (k) }, and 0≦s (1) (x i(1) , x j(1) )≦ . . . ≦s (n) (x i(n) , x j(n) )≦1.

Claim 8

Dependent Claim 40 words

The method as recited in claim 5 , wherein the fuzzy integral is operated by studying and initializing the fuzzy measurement about individual features and then studying special interactions to perform the fuzzy measurement necessary for fuzzy via relevance feedback.

Claim 9

Dependent Claim 281 words

The method as recited in claim 6 , wherein the fuzzy studying of the individual features is performed by establishing fuzzy rules based on representative similarity differences and by measuring the fuzzy integral, wherein the fuzzy rules are established based on equations as: If Δ s k is positive, then g k =1, Else if Δ s k is negative, then g k =0, and Else if Δ s k is zero, then g k =g k ; and Δs k is a difference between ps k and ns k for the kth image feature x k , ps k is an average similarity of the query image g and the image i belonging to a related image set R + , ns k is an average similarity of the query image q and the image j belonging to an unrelated image set R − , wherein the fuzzy integral is measured based on an equation as: g k =( p (Δ s k )*1.0+ n (Δ s k )*0.0+ g k *z(Δ s k ))/( p (Δ s k )+ n (Δ s k )+ z (Δ s k )= ( p (Δ s k )+ g k *z (Δ s k ))/( p (Δ s k )+ n (Δ s k )+ z (Δ s k )) where (p(Δ s k ) is a membership representing the degree that Δ s k belongs to the fuzzy set positive, n(Δ s k ) is a membership representing the degree that Δ s k belongs to the fuzzy set negative, and z(Δ s k ) is a membership representing the degree that Δ s k belongs to the fuzzy set zero.

Claim 10

Dependent Claim 245 words

A computer readable record medium storing instructions for executing a content-based image retrieval method, comprising: measuring similarities according to feature between images stored in a database and a new query image; associating the similarities measured according to feature by using the fuzzy integral to measure similarities according to the fuzzy integral; bringing images sequentially in the order of higher similarity according to the finally obtained similarities and transmitting a search result; and recalculating similarities according to feature via relevance feedback about the query result to output converted images wherein the final image similarities are measured based on an equation as: C g ( S, i, j )= k=1 , . . . , n [S (k) ( x i(k) , x j(k) )− S (k−1) ( x i(k−1) , x j(k−1) ) g ( A (k) )] where S={s 1 , . . . s n } is a set of similarity measurement functions between two images, X={x 1 , . . . x n } is a set of image features, x i(k) and x i(k) are kth image features of query images i and j, n is the total number of the used image features, g(A (k) ), which is also expressed as g k , denotes relevance of a feature subset A (k) ={x (1) , . . . x (k) } and 0≦s (1) (x i(1) , x j(1) )≦ . . . ≦s (n) (x i(n) , x j(n) )≦1.

Claim 11

Dependent Claim 245 words

A computer readable record medium storing instructions for executing a content-based image retrieval method, comprising: measuring similarities according to feature between images stored in a database and a new query image; associating the similarities measured according to feature by using the fuzzy integral to measure similarities according to the fuzzy integral; bringing images sequentially in the order of higher similarity according to the finally obtained similarities and transmitting a search result; and recalculating similarities according to feature via relevance feedback about the query result to output converted images wherein the final image similarities are measured based on an equation as: C g ( S, i, j )= k=1 , . . . , n [S (k) ( x i(k) , x j(k) )− S (k−1) ( x i(k−1) , x j(k−1) ) g ( A (k) )] where S={s 1 , . . . s n } is a set of similarity measurement functions between two images, X={x 1 , . . . x n } is a set of image features, x i(k) and x i(k) are kth image features of query images i and j, n is the total number of the used image features, g(A (k) ), which is also expressed as g k , denotes relevance of a feature subset A (k) ={x (1) , . . . x (k) } and 0≦s (1) (x i(1) , x j(1) )≦ . . . ≦s (n) (x i(n) , x j(n) )≦1.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2007-06-01	2007-12-01	2008-06-01	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2011-06-01	2011-12-01	2012-06-01	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2015-06-01	2015-12-01	2016-06-01	$7700.00	Window Closed / Lapsed

Content-based image retrieval apparatus and method via relevance feedback by using fuzzy integral

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag