US5852823A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Image classification and retrieval system using a query-by-example paradigm
Assignee: Microsoft Technology Licensing LLC
•
Filing: 1996-10-16
•
Granted: 1998-12-22
•
Expires: 2016-10-16
Abstract
Abstract Apparatus and an accompanying method for generating a semantically based, linguistically searchable, numeric descriptor of a pre-defined group of input images and which is particularly useful in a system for automatically classifying individual images, on a numerical basis, in, e.g., an image database, and, through a query-by-example paradigm, retrieving a desired image(s) therefrom. Specifically, a signature is computed for each image in a set using multi-level iterative convolution filtering, with pixel values supplied as input to each filtering level being separately convolved with each one of a set of predefined Gaussian kernels. Average and variance vectors, as collective numeric descriptors of all the images in the set, are separately computed across corresponding elements in all the image signatures for the set. A linguistic term, semantically descriptive of all the images in the set, is associated with the numeric descriptors of this set and, with this set, is stored in a database. For image retrieval, the descriptor for any set is accessed by a textual search through the database using the appropriate linguistic term. The descriptor is then compared against accessed signatures for other images in the database in order to retrieve a image, among those stored in the database, that is the most similar to those in the set associated with the descriptor.
Inventors
- 👤 Jeremy S. De Bonet
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-29 14:23
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/583; G06F16/5838; G06F16/5846; G06V; G06V10/00; G06V10/40; Y; Y10; Y10S; Y10S707/00; Y10S707/912; Y10S707/913; Y10S707/915; Y10S707/99931; Y10S707/99933; Y10S707/99936; Y10S707/99941; Y10S707/99944; Y10S707/99945
Claims Analysis
Total Claims: 102
Claim 1
Independent Claim
335 words
A system, responsive to a query image set, for retrieving, from a stored database, a desired image that is visually similar to the image set, said set containing at least one query image, said system comprising: an image database having stored images therein; a signature generator, responsive to an input image and having three successive convolution filters, for producing a signature for the input image, the signature containing a numeric measure of each of a plurality of pre-defined characteristics of the input image; wherein the signature generator, in response to the query image set, generates a corresponding signature for each image in the query image set, and, in response to a test image accessed from the stored database, generates a corresponding test signature; a statistics generator, responsive to the signature for each image in the query image set, for providing a separate pre-defined statistical measure, for each of the pre-defined characteristics, that represents variability of said each pre-defined characteristic across all images in the query image set; a image database manager, operative in conjunction with the image database, for retrieving successive ones of a plurality of stored images from the database and for routing each one of the plurality of successive images as the test image to the signature generator so as to generate a plurality of corresponding test signatures; a comparator for comparing, across all of the pre-defined characteristics, the statistical measure for each of the pre-defined characteristics against a corresponding numeric measure in each one of the test signatures so as to yield a similarity measure, the similarity measure for said one of the test images being reflective of a degree of visual similarity between the one test image and the query image set; and a manager, responsive to the similarity measure for each of the test images and operative in conjunction with the image database manager, for selecting at least one of the test images having a highest relative similarity measure as a retrieved image and providing the retrieved image as output.
Claim 2
Independent Claim
335 words
A system, responsive to a query image set, for retrieving, from a stored database, a desired image that is visually similar to the image set, said set containing at least one query image, said system comprising: an image database having stored images therein; a signature generator, responsive to an input image and having three successive convolution filters, for producing a signature for the input image, the signature containing a numeric measure of each of a plurality of pre-defined characteristics of the input image; wherein the signature generator, in response to the query image set, generates a corresponding signature for each image in the query image set, and, in response to a test image accessed from the stored database, generates a corresponding test signature; a statistics generator, responsive to the signature for each image in the query image set, for providing a separate pre-defined statistical measure, for each of the pre-defined characteristics, that represents variability of said each pre-defined characteristic across all images in the query image set; a image database manager, operative in conjunction with the image database, for retrieving successive ones of a plurality of stored images from the database and for routing each one of the plurality of successive images as the test image to the signature generator so as to generate a plurality of corresponding test signatures; a comparator for comparing, across all of the pre-defined characteristics, the statistical measure for each of the pre-defined characteristics against a corresponding numeric measure in each one of the test signatures so as to yield a similarity measure, the similarity measure for said one of the test images being reflective of a degree of visual similarity between the one test image and the query image set; and a manager, responsive to the similarity measure for each of the test images and operative in conjunction with the image database manager, for selecting at least one of the test images having a highest relative similarity measure as a retrieved image and providing the retrieved image as output.
Claim 3
Dependent Claim
44 words
The system in claim 1 further comprising a path for feeding back the test signature of the retrieved image to the statistics generator such that the statistical measure for each of the pre-defined characteristics is responsive to the query image and the retrieved image.
Claim 4
Dependent Claim
56 words
The system in claim 1 wherein each of said filters comprises a convolver for convolving pixel values, of an incoming image and for a given one of a plurality of color axes in the incoming image, with each one of a plurality of predefined kernels to yield a corresponding filter response image for said color axis.
Claim 5
Dependent Claim
91 words
The system in claim 3 wherein the statistics generator produces, for each one of said pre-defined characteristics and for each color axis in the incoming image, a value of an average of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield an average vector, and, as said pre-defined statistical measure, a value of variance of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield a variance vector.
Claim 6
Dependent Claim
44 words
The system in claim 4 further comprising a path for feeding back the test signature of the retrieved image to the statistics generator such that the statistical measure for each of the pre-defined characteristics is responsive to the query image and the retrieved image.
Claim 7
Dependent Claim
46 words
The system in claim 4 wherein the comparator comprises a comparison engine for determining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 8
Dependent Claim
112 words
The system in claim 6 wherein the comparator comprises: a subtractor for determining a difference between values in corresponding elements of said average vector and said text signature; a squarer for squaring a magnitude of the difference so as to yield a squared difference value; a divider for dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; a summer for summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and an inverter for determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 9
Dependent Claim
81 words
The system in claim 4 wherein the signature generator further comprises first, second and third serially connected convolution filters for iteratively convolving pixel values of said each color axis in the incoming image applied to said signature generator, wherein the first, second and third filters separately convolves the pixel values in the incoming image applied to the first, second and third filters with each one of a plurality of individual kernels in first, second and third sets of predefined kernels, respectively.
Claim 10
Dependent Claim
44 words
The system in claim 8 wherein each of the filters further comprises a squarer, responsive to the filter response image, for squaring a magnitude of each pixel value produced by the convolver in said each filter so as to yield a filter response image.
Claim 11
Dependent Claim
75 words
The system in claim 9 wherein the signature generator further comprises a sum and normalizer for summing all the pixel values in the filter response image produced by the third filter to yield a resultant sum and for normalizing the resultant sum by a total number of pixels in the filter response image produced by the third filter so as to yield a value of a corresponding one of the plurality of pre-defined image characteristics.
Claim 12
Dependent Claim
16 words
The system in claim 10 wherein the first, second and third sets of kernels are identical.
Claim 13
Dependent Claim
14 words
The system in claim 8 wherein each of said convolvers comprises a down-sizing convolver.
Claim 14
Dependent Claim
57 words
The system in claim 12 wherein each of the convolvers further comprises serially connected horizontal and vertical down-sizing convolvers for convolving said incoming image with first and second half-level kernels, respectively; each one of said individual kernels being a derivative of a Gaussian kernel and separable into a corresponding pair of said first and second half-level kernels.
Claim 15
Dependent Claim
46 words
The system in claim 8 wherein the comparator comprises a comparison engine for determining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 16
Dependent Claim
112 words
The system in claim 14 wherein the comparator comprises: a subtractor for determining a difference between values in corresponding elements of said average vector and said text signature; a squarer for squaring a magnitude of the difference so as to yield a squared difference value; a divider for dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; a summer for summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and an inverter for determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 17
Independent Claim
351 words
A system, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, said system comprising: a database having stored therein a plurality of images and a signature associated with each of said images, wherein each of the signatures contains a numeric measure of each of a plurality of pre-defined characteristics of an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; a manager for accessing, in response to a term associated with the pre-defined set of images, from the database an associated set of signatures for a query image set, and for accessing individual ones of the signatures for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor; a statistics generator, responsive to the set of signatures for the query image set, for providing separate statistical measures, for each of the pre-defined characteristics, that separately represent average and variability measures of said each pre-defined characteristic across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics; a comparison engine for determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; and wherein the manager, in response to the similarity measure for each of the test images, selects at least one of the test images having a highest relative similarity measure as a retrieved image and provides the retrieved image as output.
Claim 18
Independent Claim
351 words
A system, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, said system comprising: a database having stored therein a plurality of images and a signature associated with each of said images, wherein each of the signatures contains a numeric measure of each of a plurality of pre-defined characteristics of an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; a manager for accessing, in response to a term associated with the pre-defined set of images, from the database an associated set of signatures for a query image set, and for accessing individual ones of the signatures for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor; a statistics generator, responsive to the set of signatures for the query image set, for providing separate statistical measures, for each of the pre-defined characteristics, that separately represent average and variability measures of said each pre-defined characteristic across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics; a comparison engine for determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; and wherein the manager, in response to the similarity measure for each of the test images, selects at least one of the test images having a highest relative similarity measure as a retrieved image and provides the retrieved image as output.
Claim 19
Dependent Claim
45 words
The system in claim 16 further comprising a path for feeding back the test signature for the retrieved image to the statistics generator such that the statistical measure for each of the pre-defined characteristics is responsive to the query image set and the retrieved image.
Claim 20
Dependent Claim
69 words
The system in claim 16 further comprising a signature generator having a plurality of serially connected convolution filters wherein each of said filters comprises a convolver for convolving pixel values, of an incoming image and for a given one of a plurality of color axes in the incoming image, with each one of a plurality of predefined kernels to yield a corresponding filter response image for said color axis.
Claim 21
Dependent Claim
91 words
The system in claim 18 wherein the statistics generator produces, for each one of said pre-defined characteristics and for each color axis in the incoming image, a value of an average of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield an average vector, and, as said pre-defined statistical measure, a value of variance of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield a variance vector.
Claim 22
Dependent Claim
45 words
The system in claim 19 further comprising a path for feeding back the test signature for the retrieved image to the statistics generator such that the statistical measure for each of the pre-defined characteristics is responsive to the query image set and the retrieved image.
Claim 23
Dependent Claim
42 words
The system in claim 19 wherein the comparison engine determines the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 24
Dependent Claim
113 words
The system in claim 21 wherein the comparison engine comprises: a subtractor for determining a difference between values in corresponding elements of said average vector and said text signature; a squarer for squaring a magnitude of the difference so as to yield a squared difference value; a divider for dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; a summer for summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and an inverter for determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 25
Dependent Claim
81 words
The system in claim 19 wherein the signature generator further comprises first, second and third serially connected convolution filters for iteratively convolving pixel values of said each color axis in the incoming image applied to said signature generator, wherein the first, second and third filters separately convolves the pixel values in the incoming image applied to the first, second and third filters with each one of a plurality of individual kernels in first, second and third sets of predefined kernels, respectively.
Claim 26
Dependent Claim
44 words
The system in claim 23 wherein each of the filters further comprises a squarer, responsive to the filter response image, for squaring a magnitude of each pixel value produced by the convolver in said each filter so as to yield a filter response image.
Claim 27
Dependent Claim
75 words
The system in claim 24 wherein the signature generator further comprises a sum and normalizer for summing all the pixel values in the filter response image produced by the third filter to yield a resultant sum and for normalizing the resultant sum by a total number of pixels in the filter response image produced by the third filter so as to yield a value of a corresponding one of the plurality of pre-defined image characteristics.
Claim 28
Dependent Claim
16 words
The system in claim 25 wherein the first, second and third sets of kernels are identical.
Claim 29
Dependent Claim
14 words
The system in claim 23 wherein each of said convolvers comprises a down-sizing convolver.
Claim 30
Dependent Claim
57 words
The system in claim 27 wherein each of the convolvers further comprises serially connected horizontal and vertical down-sizing convolvers for convolving said incoming image with first and second half-level kernels, respectively; each one of said individual kernels being a derivative of a Gaussian kernel and separable into a corresponding pair of said first and second half-level kernels.
Claim 31
Dependent Claim
42 words
The system in claim 23 wherein the comparison engine determines the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 32
Dependent Claim
113 words
The system in claim 29 wherein the comparison engine comprises: a subtractor for determining a difference between values in corresponding elements of said average vector and said text signature; a squarer for squaring a magnitude of the difference so as to yield a squared difference value; a divider for dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; a summer for summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and an inverter for determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 33
Independent Claim
335 words
A system, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, said system comprising: a database having stored therein a plurality of images and a signature associated with each of said images, wherein each of the signatures contains a numeric measure of each of a plurality of pre-defined characteristics of an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; a manager for accessing, in response to a term associated with the pre-defined set of query images, from the database an associated set of statistical measures descriptive of the query image set, and for accessing individual ones of the signatures for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor, wherein the statistical measures separately represent average and variability measures of a corresponding one of the pre-defined characteristics across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics; a comparison engine for determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; and wherein the manager, in response to the similarity measure for each of the test images, selects at least one of the test images having a highest relative similarity measure as a retrieved image and provides the retrieved image as output.
Claim 34
Independent Claim
335 words
A system, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, said system comprising: a database having stored therein a plurality of images and a signature associated with each of said images, wherein each of the signatures contains a numeric measure of each of a plurality of pre-defined characteristics of an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; a manager for accessing, in response to a term associated with the pre-defined set of query images, from the database an associated set of statistical measures descriptive of the query image set, and for accessing individual ones of the signatures for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor, wherein the statistical measures separately represent average and variability measures of a corresponding one of the pre-defined characteristics across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics; a comparison engine for determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; and wherein the manager, in response to the similarity measure for each of the test images, selects at least one of the test images having a highest relative similarity measure as a retrieved image and provides the retrieved image as output.
Claim 35
Dependent Claim
69 words
The system in claim 31 further comprising a signature generator having a plurality of serially connected convolution filters wherein each of said filters comprises a convolver for convolving pixel values, of an incoming image and for a given one of a plurality of color axes in the incoming image, with each one of a plurality of predefined kernels to yield a corresponding filter response image for said color axis.
Claim 36
Dependent Claim
93 words
The system in claim 32 further comprising a statistics generator for producing, for each one of said pre-defined characteristics and for each color axis in the incoming image, a value of an average of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield an average vector, and, as said pre-defined statistical measure, a value of variance of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield a variance vector.
Claim 37
Dependent Claim
42 words
The system in claim 33 wherein the comparison engine determines the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 38
Dependent Claim
113 words
The system in claim 34 wherein the comparison engine comprises: a subtractor for determining a difference between values in corresponding elements of said average vector and said text signature; a squarer for squaring a magnitude of the difference so as to yield a squared difference value; a divider for dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; a summer for summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and an inverter for determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 39
Dependent Claim
81 words
The system in claim 33 wherein the signature generator further comprises first, second and third serially connected convolution filters for iteratively convolving pixel values of said each color axis in the incoming image applied to said signature generator, wherein the first, second and third filters separately convolves the pixel values in the incoming image applied to the first, second and third filters with each one of a plurality of individual kernels in first, second and third sets of predefined kernels, respectively.
Claim 40
Dependent Claim
44 words
The system in claim 36 wherein each of the filters further comprises a squarer, responsive to the filter response image, for squaring a magnitude of each pixel value produced by the convolver in said each filter so as to yield a filter response image.
Claim 41
Dependent Claim
75 words
The system in claim 37 wherein the signature generator further comprises a sum and normalizer for summing all the pixel values in the filter response image produced by the third filter to yield a resultant sum and for normalizing the resultant sum by a total number of pixels in the filter response image produced by the third filter so as to yield a value of a corresponding one of the plurality of pre-defined image characteristics.
Claim 42
Dependent Claim
16 words
The system in claim 38 wherein the first, second and third sets of kernels are identical.
Claim 43
Dependent Claim
14 words
The system in claim 36 wherein each of said convolvers comprises a down-sizing convolver.
Claim 44
Dependent Claim
57 words
The system in claim 40 wherein each of the convolvers further comprises serially connected horizontal and vertical down-sizing convolvers for convolving said incoming image with first and second half-level kernels, respectively; each one of said individual kernels being a derivative of a Gaussian kernel and separable into a corresponding pair of said first and second half-level kernels.
Claim 45
Dependent Claim
42 words
The system in claim 36 wherein the comparison engine determines the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 46
Dependent Claim
113 words
The system in claim 42 wherein the comparison engine comprises: a subtractor for determining a difference between values in corresponding elements of said average vector and said text signature; a squarer for squaring a magnitude of the difference so as to yield a squared difference value; a divider for dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; a summer for summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and an inverter for determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 47
Independent Claim
305 words
A method, responsive to a query image set, for retrieving, from a stored image repository, a desired image that is visually similar to the image set, said set containing at least one query image, the method comprising the steps of: generating, through three successive convolution filters responsive to an input image, a signature for the input image, the signature containing a numeric measure of each of a plurality of pre-defined characteristics of the input image; wherein the signature generating step comprises the steps of: generating, in response to the query image set, a corresponding signature for each image in the query image set, and generating, in response to a test image accessed from the stored image repository, a corresponding test signature; providing, in response to the signature for each image in the stored image repository, a separate pre-defined statistical measure, for each of the pre-defined characteristics, that represents variability of said each pre-defined characteristic across all images in the query image set; retrieving successive ones of a plurality of stored images from the stored image repository and applying each one of the plurality of successive images as the test image to the signature generating step so as to generate a plurality of corresponding test signatures; comparing, across all of the pre-defined characteristics, the statistical measure for each of the pre-defined characteristics against a corresponding numeric measure in each one of the test signatures so as to yield a similarity measure, the similarity measure for said one of the test images being reflective of a degree of visual similarity between the one test image and the query image set; selecting, in response to the similarity measure for each of the test images, at least one of the test images having a highest relative similarity measure as a retrieved image; and providing the retrieved image as output.
Claim 48
Independent Claim
305 words
A method, responsive to a query image set, for retrieving, from a stored image repository, a desired image that is visually similar to the image set, said set containing at least one query image, the method comprising the steps of: generating, through three successive convolution filters responsive to an input image, a signature for the input image, the signature containing a numeric measure of each of a plurality of pre-defined characteristics of the input image; wherein the signature generating step comprises the steps of: generating, in response to the query image set, a corresponding signature for each image in the query image set, and generating, in response to a test image accessed from the stored image repository, a corresponding test signature; providing, in response to the signature for each image in the stored image repository, a separate pre-defined statistical measure, for each of the pre-defined characteristics, that represents variability of said each pre-defined characteristic across all images in the query image set; retrieving successive ones of a plurality of stored images from the stored image repository and applying each one of the plurality of successive images as the test image to the signature generating step so as to generate a plurality of corresponding test signatures; comparing, across all of the pre-defined characteristics, the statistical measure for each of the pre-defined characteristics against a corresponding numeric measure in each one of the test signatures so as to yield a similarity measure, the similarity measure for said one of the test images being reflective of a degree of visual similarity between the one test image and the query image set; selecting, in response to the similarity measure for each of the test images, at least one of the test images having a highest relative similarity measure as a retrieved image; and providing the retrieved image as output.
Claim 49
Dependent Claim
23 words
The method in claim 44 wherein the stored image repository is a database stored on a server or across a plurality of servers.
Claim 50
Dependent Claim
19 words
The method in claim 45 wherein the stored images that collectively form a motion picture or a video program.
Claim 51
Dependent Claim
23 words
The method in claim 45 wherein said image providing step comprises the step of providing, as output, textual information accompanying the retrieved image.
Claim 52
Dependent Claim
47 words
The method in claim 45 further comprising the step of feeding back the test signature of the retrieved image to the statistical measure producing step such that the statistical measure for each one of the pre-defined characteristics is responsive to the query image and the retrieved image.
Claim 53
Dependent Claim
57 words
The method in claim 44 wherein each of said filtering steps comprises the step of convolving pixel values, of an incoming image and for a given one of a plurality of color axes in the incoming image, with each one of a plurality of predefined kernels to yield a corresponding filter response image for said color axis.
Claim 54
Dependent Claim
97 words
The method in claim 49 wherein the statistical measures providing step comprises the step of producing, for each one of said pre-defined characteristics and for each color axis in the incoming image, a value of an average of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield an average vector, and, as said pre-defined statistical measure, a value of variance of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield a variance vector.
Claim 55
Dependent Claim
47 words
The method of claim 50 further comprising the step of feeding back the test signature of the retrieved image to the statistical measure producing step such that the statistical measure for each one of the pre-defined characteristics is responsive to the query image and the retrieved image.
Claim 56
Dependent Claim
46 words
The method in claim 50 wherein the comparing step comprises the step of determining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 57
Dependent Claim
101 words
The method in claim 52 wherein the comparing step comprises the steps of: determining a difference between values in corresponding elements of said average vector and said text signature; squaring a magnitude of the difference so as to yield a squared difference value; dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 58
Dependent Claim
92 words
The method in claim 50 wherein the signature generating step further comprises first, second and third successively occurring convolution filtering steps for iteratively convolving pixel values of said each color axis in the incoming image applied to said signature generating step, wherein each of the first, second and third filtering steps comprises the step of separately convolving the pixel values in the incoming image applied to the first, second and third filtering steps with each one of a plurality of individual kernels in first, second and third sets of predefined kernels, respectively.
Claim 59
Dependent Claim
48 words
The method in claim 54 wherein each of the filtering steps further comprises the step of squaring, in response to the filter response image, a magnitude of each pixel value produced by the convolving step in said each filtering step so as to yield a filter response image.
Claim 60
Dependent Claim
73 words
The method in claim 55 wherein the signature generating step further comprises the steps of: summing all the pixel values in the filter response image produced by the third filter to yield a resultant sum; and normalizing the resultant sum by a total number of pixels in the filter response image produced by the third filter so as to yield a value of a corresponding one of the plurality of pre-defined image characteristics.
Claim 61
Dependent Claim
16 words
The method in claim 56 wherein the first, second and third sets of kernels are identical.
Claim 62
Dependent Claim
15 words
The method in claim 54 wherein each of said convolving steps comprises a down-sizing convolution.
Claim 63
Dependent Claim
62 words
The method in claim 58 wherein each of the convolving steps further comprises the step of separately and successively performing horizontal and vertical down-sizing convolutions of said incoming image with first and second half-level kernels, respectively; each one of said individual kernels a derivative of a Gaussian kernel and being separable into a corresponding pair of said first and second half-level kernels.
Claim 64
Dependent Claim
46 words
The method in claim 54 wherein the comparing step comprises the step of determining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 65
Dependent Claim
102 words
The method in claim 60 wherein the comparing step further comprises the steps of: determining a difference between values in corresponding elements of said average vector and said text signature; squaring a magnitude of the difference so as to yield a squared difference value; dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 66
Dependent Claim
15 words
A computer-readable medium having computer-executable instructions stored thereon for performing the steps of claim 44.
Claim 67
Independent Claim
337 words
A method, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, the method comprising the steps of: accessing, in response to a term associated with the pre-defined set of images, from a database an associated set of signatures for a query image set, and accessing individual signatures from the database for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor, the database storing a plurality of images and a signature associated with each of said images, wherein each of the signatures contains a numeric measure of each of a plurality of pre-defined characteristics of an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; providing, in response to the set of signatures for the query image set, separate statistical measures, for each of the pre-defined characteristics, that separately represent average and variability measures of said each pre-defined characteristic across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics; determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; selecting, in response to the similarity measure for each of the test images, at least one of the test images having a highest relative similarity measure as a retrieved image; and providing the retrieved image as output.
Claim 68
Independent Claim
337 words
A method, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, the method comprising the steps of: accessing, in response to a term associated with the pre-defined set of images, from a database an associated set of signatures for a query image set, and accessing individual signatures from the database for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor, the database storing a plurality of images and a signature associated with each of said images, wherein each of the signatures contains a numeric measure of each of a plurality of pre-defined characteristics of an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; providing, in response to the set of signatures for the query image set, separate statistical measures, for each of the pre-defined characteristics, that separately represent average and variability measures of said each pre-defined characteristic across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics; determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; selecting, in response to the similarity measure for each of the test images, at least one of the test images having a highest relative similarity measure as a retrieved image; and providing the retrieved image as output.
Claim 69
Dependent Claim
23 words
The method in claim 63 wherein the stored image repository is a database stored on a server or across a plurality of servers.
Claim 70
Dependent Claim
47 words
The method in claim 63 further comprising the step of feeding back the test signature of the retrieved image to the statistical measure producing step such that the statistical measure for each one of the pre-defined characteristics is responsive to the query image and the retrieved image.
Claim 71
Dependent Claim
23 words
The method in claim 63 wherein said image providing step comprises the step of providing, as output, textual information accompanying the retrieved image.
Claim 72
Dependent Claim
70 words
The method in claim 63 further comprising a signature generating step having a plurality of convolution filtering steps wherein each of said filtering steps comprises the step of convolving pixel values, of an incoming image and for a given one of a plurality of color axes in the incoming image, with each one of a plurality of predefined kernels to yield a corresponding filter response image for said color axis.
Claim 73
Dependent Claim
97 words
The method in claim 67 wherein the statistical measures providing step comprises the step of producing, for each one of said pre-defined characteristics and for each color axis in the incoming image, a value of an average of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield an average vector, and, as said pre-defined statistical measure, a value of variance of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield a variance vector.
Claim 74
Dependent Claim
47 words
The method in claim 68 further comprising the step of feeding back the test signature of the retrieval image to the statistical measure producing step such that the statistical measure for each one of the pre-defined characteristics is responsive to the query image and the retrieved image.
Claim 75
Dependent Claim
46 words
The method in claim 68 wherein the determining step comprises the step of ascertaining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 76
Dependent Claim
101 words
The method in claim 70 wherein the ascertaining step comprises the steps of: determining a difference between values in corresponding elements of said average vector and said text signature; squaring a magnitude of the difference so as to yield a squared difference value; dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 77
Dependent Claim
92 words
The method in claim 68 wherein the signature generating step further comprises first, second and third successively occurring convolution filtering steps for iteratively convolving pixel values of said each color axis in the incoming image applied to said signature generating step, wherein each of the first, second and third filtering steps comprises the step of separately convolving the pixel values in the incoming image applied to the first, second and third filtering steps with each one of a plurality of individual kernels in first, second and third sets of predefined kernels, respectively.
Claim 78
Dependent Claim
48 words
The method in claim 72 wherein each of the filtering steps further comprises the step of squaring, in response to the filter response image, a magnitude of each pixel value produced by the convolving step in said each filtering step so as to yield a filter response image.
Claim 79
Dependent Claim
73 words
The method in claim 73 wherein the signature generating step further comprises the steps of: summing all the pixel values in the filter response image produced by the third filter to yield a resultant sum; and normalizing the resultant sum by a total number of pixels in the filter response image produced by the third filter so as to yield a value of a corresponding one of the plurality of pre-defined image characteristics.
Claim 80
Dependent Claim
16 words
The method in claim 74 wherein the first, second and third sets of kernels are identical.
Claim 81
Dependent Claim
15 words
The method in claim 72 wherein each of said convolving steps comprises a down-sizing convolution.
Claim 82
Dependent Claim
62 words
The method in claim 76 wherein each of the convolving steps further comprises the step of separately and successively performing horizontal and vertical down-sizing convolutions of said incoming image with first and second half-level kernels, respectively; each one of said individual kernels being a derivative of a Gaussian kernel and separable into a corresponding pair of said first and second half-level kernels.
Claim 83
Dependent Claim
46 words
The method in claim 72 wherein the determining step comprises the step of ascertaining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 84
Dependent Claim
102 words
The method in claim 78 wherein the ascertaining step further comprises the steps of: determining a difference between values in corresponding elements of said average vector and said text signature; squaring a magnitude of the difference so as to yield a squared difference value; dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 85
Dependent Claim
15 words
A computer-readable medium having computer-executable instructions stored thereon for performing the steps of claim 63.
Claim 86
Independent Claim
331 words
A method, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, the method comprising the steps of: accessing, in response to a term associated with the pre-defined set of query images, from a database an associated set of statistical measures descriptive of a set of pre-defined characteristics of the query image set, and accessing individual ones of the signatures from the database for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor, the statistical measures separately representing average and variability measures of a corresponding one of said pre-defined characteristics across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics, wherein the database stores a plurality of images and a signature associated with each of said images, each of the signatures containing a numeric measure of each one of the pre-defined characteristics for an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; selecting, in response to the similarity measure for each of the test images, at least one of the test images having a highest relative similarity measure as a retrieved image; and providing the retrieved image as output.
Claim 87
Independent Claim
331 words
A method, responsive to a term chosen by a user, for retrieving, from a stored database, a desired image that is visually similar to images in a set associated with the term, said set containing at least one query image, the method comprising the steps of: accessing, in response to a term associated with the pre-defined set of query images, from a database an associated set of statistical measures descriptive of a set of pre-defined characteristics of the query image set, and accessing individual ones of the signatures from the database for a plurality of other images in the database so as to define a plurality of test images and a plurality of corresponding test signatures therefor, the statistical measures separately representing average and variability measures of a corresponding one of said pre-defined characteristics across all images in the query image set so as to form separate corresponding pluralities of average and variability measures for all of said characteristics, wherein the database stores a plurality of images and a signature associated with each of said images, each of the signatures containing a numeric measure of each one of the pre-defined characteristics for an associated image stored in the database so as to form a plurality of numeric measures in the signature for the associated image; determining, for each of the test signatures, a similarity measure reflective of a degree of visual similarity between a corresponding one of the test images and the query image set, wherein the similarity measure is a function, taken across all of the pre-defined characteristics, of separate corresponding ones of the pluralities of the average and variance measures for the query image set and a separate corresponding one of the plurality of said numeric measures for said each test signature; selecting, in response to the similarity measure for each of the test images, at least one of the test images having a highest relative similarity measure as a retrieved image; and providing the retrieved image as output.
Claim 88
Dependent Claim
23 words
The method in claim 81 wherein the stored image repository is a database stored on a server or across a plurality of servers.
Claim 89
Dependent Claim
22 words
The method in claim 81 said image providing step comprises the step of providing, as output, textual information accompanying the retrieved image.
Claim 90
Dependent Claim
70 words
The method in claim 81 further comprising a signature generating step having a plurality of convolution filtering steps wherein each of said filtering steps comprises the step of convolving pixel values, of an incoming image and for a given one of a plurality of color axes in the incoming image, with each one of a plurality of predefined kernels to yield a corresponding filter response image for said color axis.
Claim 91
Dependent Claim
92 words
The method in claim 84 further comprising the step of producing, for each one of said pre-defined characteristics and for each color axis in the incoming image, a value of an average of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield an average vector, and, as said pre-defined statistical measure, a value of variance of the numeric measure of said one pre-defined characteristic across all the signatures in the query image set so as to yield a variance vector.
Claim 92
Dependent Claim
46 words
The method in claim 85 wherein the determining step comprises the step of ascertaining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 93
Dependent Claim
101 words
The method in claim 86 wherein the ascertaining step comprises the steps of: determining a difference between values in corresponding elements of said average vector and said text signature; squaring a magnitude of the difference so as to yield a squared difference value; dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 94
Dependent Claim
92 words
The method in claim 85 wherein the signature generating step further comprises first, second and third successively occurring convolution filtering steps for iteratively convolving pixel values of said each color axis in the incoming image applied to said signature generating step, wherein each of the first, second and third filtering steps comprises the step of separately convolving the pixel values in the incoming image applied to the first, second and third filtering steps with each one of a plurality of individual kernels in first, second and third sets of predefined kernels, respectively.
Claim 95
Dependent Claim
48 words
The method in claim 88 wherein each of the filtering steps further comprises the step of squaring, in response to the filter response image, a magnitude of each pixel value produced by the convolving step in said each filtering step so as to yield a filter response image.
Claim 96
Dependent Claim
73 words
The method in claim 89 wherein the signature generating step further comprises the steps of: summing all the pixel values in the filter response image produced by the third filter to yield a resultant sum; and normalizing the resultant sum by a total number of pixels in the filter response image produced by the third filter so as to yield a value of a corresponding one of the plurality of pre-defined image characteristics.
Claim 97
Dependent Claim
16 words
The method in claim 90 wherein the first, second and third sets of kernels are identical.
Claim 98
Dependent Claim
15 words
The method in claim 88 wherein each of said convolving steps comprises a down-sizing convolution.
Claim 99
Dependent Claim
62 words
The method in claim 92 wherein each of the convolving steps further comprises the step of separately and successively performing horizontal and vertical down-sizing convolutions of said incoming image with first and second half-level kernels, respectively; each one of said individual kernels being a derivative of a Gaussian kernel and separable into a corresponding pair of said first and second half-level kernels.
Claim 100
Dependent Claim
46 words
The method in claim 88 wherein the determining step comprises the step of ascertaining the similarity measure, for the test image, as a function of a difference between corresponding elements in said average vector and said test signature, and a corresponding element in said variance vector.
Claim 101
Dependent Claim
102 words
The method in claim 94 wherein the ascertaining step further comprises the steps of: determining a difference between values in corresponding elements of said average vector and said text signature; squaring a magnitude of the difference so as to yield a squared difference value; dividing said squared difference value by a corresponding element in the variance vector to yield an elemental variability value; summing the elemental variability values across all of the predefined characteristics and for all of the color axes, to yield a difference measure; and determining the similarity measure for the test image as a reciprocal of the difference measure.
Claim 102
Dependent Claim
15 words
A computer-readable medium having computer-executable instructions stored thereon for performing the steps of claim 81.
References Cited (Backward Citations)
US5628003A
Cited
US5581357A
Cited
EP0388677A2
Cited
US5546572A
Cited
US5579471A
Cited
US5644765A
Cited
US5493677A
Cited
US5659626A
Cited
US5729741A
Cited
US5675788A
Cited
US5696964A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2001-12-21 | 2002-06-22 | 2002-12-22 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2005-12-21 | 2006-06-22 | 2006-12-22 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2009-12-21 | 2010-06-22 | 2010-12-22 | $7700.00 | Window Closed / Lapsed |