US6760714B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Representation and retrieval of images using content vectors derived from image information elements
Assignee: Fair Isaac Corp
•
Filing: 2000-09-29
•
Granted: 2004-07-06
•
Expires: 2020-09-29
Abstract
Abstract Image features are generated by performing wavelet transformations at sample points on images stored in electronic form. Multiple wavelet transformations at a point are combined to form an image feature vector. A prototypical set of feature vectors, or atoms, is derived from the set of feature vectors to form an “atomic vocabulary.” The prototypical feature vectors are derived using a vector quantization method (e.g., using neural network self-organization techniques) in which a vector quantization network is also generated. The atomic vocabulary is used to define new images. Meaning is established between atoms in the atomic vocabulary. High-dimensional context vectors are assigned to each atom. The context vectors are then trained as a function of the proximity and co-occurrence of each atom to other atoms in the image. After training, the context vectors associated with the atoms that comprise an image are combined to form a summary vector for the image. Images are retrieved using a number of query methods (e.g., images, image portions, vocabulary atoms, index terms). The user's query is converted into a query context vector. A dot product is calculated between the query vector and the summary vectors to locate images having the closest meaning. The invention is also applicable to video or temporally related images, and can also be used in conjunction with other context vector data domains such as text or audio, thereby linking images to such data domains.
Inventors
- 👤 William R. Caid
- 👤 Robert Hecht-Neilsen
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 21:18
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/50; G06F16/58; G06F16/583; G06F16/5838; G06F16/30; G06V; G06V10/00; G06V10/40; G06V10/44; G06V10/443; G06V10/449; G06V10/451
Claims Analysis
Total Claims: 27
Claim 1
Independent Claim
131 words
A computer-implemented method of retrieving images, the method comprising: receiving a text query; generating a query context vector derived from a context vector of at least one word included in the text query; comparing the query context vector with a plurality of summary vectors, each summary vector associated with image features derived from image data of one of the plurality of images; and retrieving at least one image having a summary vector similar to the query context vector; wherein comparing the query context vector comprises: computing a dot product between the query context vector and the summary vector of each of a plurality of images and sorting said each of a plurality of images by said computed dot product whereby images associated with summary vectors producing high dot products are retrieved.
Claim 2
Dependent Claim
131 words
A computer-implemented method of retrieving images, the method comprising: receiving a text query; generating a query context vector derived from a context vector of at least one word included in the text query; comparing the query context vector with a plurality of summary vectors, each summary vector associated with image features derived from image data of one of the plurality of images; and retrieving at least one image having a summary vector similar to the query context vector; wherein comparing the query context vector comprises: computing a dot product between the query context vector and the summary vector of each of a plurality of images and sorting said each of a plurality of images by said computed dot product whereby images associated with summary vectors producing high dot products are retrieved.
Claim 3
Dependent Claim
14 words
The computer-implemented method of claim 1 , wherein the image data is pixel data.
Claim 4
Dependent Claim
50 words
The computer-implemented method of claim 1 , wherein retrieving at least one image having a summary vector similar to the query context vector comprises: retrieving at least one summary vector with an orientation in a vector space similar to an orientation of the query context vector in the vector space.
Claim 5
Dependent Claim
52 words
The computer-implemented method of claim 3 , wherein the orientation of a vector in the vector space is indicative of the meaning of the word or image with which the vector is associated, such that words or images having similar meaning are associated with vectors having similar orientations in the vector space.
Claim 6
Dependent Claim
56 words
The computer-implemented method of claim 3 , wherein the orientation of a vector in the vector space is determined by the frequency of proximal co-occurrences of words or image features in a corpus of records, such that words or image features that frequently proximally co-occur are associated with vectors having similar orientations in the vector space.
Claim 7
Dependent Claim
19 words
The computer-implemented method of claim 1 , wherein the image features are derived from the images by wavelet transformations.
Claim 8
Dependent Claim
19 words
The computer-implemented method of claim 1 , wherein the image features are coefficients of wavelet transformations on the images.
Claim 9
Dependent Claim
31 words
The computer-implemented method of claim 1 , wherein the summary vectors are oriented in a vector space, the axes of the vector space not associated with selected words or image features.
Claim 10
Dependent Claim
147 words
A computer-implemented process of retrieving images, the process comprising: receiving a query image; deriving at least one feature vector from image data of the query image; generating a query context vector from one or more context vectors associated with the at least one feature vector; comparing the query context vector with a plurality of summary vectors, each summary vector derived from context vectors associated with feature vectors derived from image data of one of the plurality of images; and retrieving at least one image having a summary vector similar to the query context vector; wherein comparing the query context vector comprises: computing a dot product between the query context vector and the summary vector of each of a plurality of images and sorting said each of a plurality of images by said computed dot product whereby images associated with summary vectors producing high dot products are retrieved.
Claim 11
Dependent Claim
147 words
A computer-implemented process of retrieving images, the process comprising: receiving a query image; deriving at least one feature vector from image data of the query image; generating a query context vector from one or more context vectors associated with the at least one feature vector; comparing the query context vector with a plurality of summary vectors, each summary vector derived from context vectors associated with feature vectors derived from image data of one of the plurality of images; and retrieving at least one image having a summary vector similar to the query context vector; wherein comparing the query context vector comprises: computing a dot product between the query context vector and the summary vector of each of a plurality of images and sorting said each of a plurality of images by said computed dot product whereby images associated with summary vectors producing high dot products are retrieved.
Claim 12
Dependent Claim
14 words
The computer-implemented method of claim 9 , wherein the image data is pixel data.
Claim 13
Dependent Claim
50 words
The computer-implemented method of claim 9 , wherein retrieving at least one image having a summary vector similar to the query context vector comprises: retrieving at least one summary vector with an orientation in a vector space similar to an orientation of the query context vector in the vector space.
Claim 14
Dependent Claim
52 words
The computer-implemented method of claim 11 , wherein the orientation of a vector in the vector space is indicative of the meaning of the word or image with which the vector is associated, such that words or images having similar meaning are associated with vectors having similar orientations in the vector space.
Claim 15
Dependent Claim
56 words
The computer-implemented method of claim 11 , wherein the orientation of a vector in the vector space is determined by the frequency of proximal co-occurrences of words or image features in a corpus of records, such that words or image features that frequently proximally co-occur are associated with vectors having similar orientations in the vector space.
Claim 16
Dependent Claim
19 words
The computer-implemented method of claim 9 , wherein the image features are derived from the images by wavelet transformations.
Claim 17
Dependent Claim
19 words
The computer-implemented method of claim 9 , wherein the image features are coefficients of wavelet transformations on the images.
Claim 18
Dependent Claim
29 words
The computer-implemented method of claim 9 , wherein the summary vectors are oriented in a vector space, the axes of the vector space not associated with selected image features.
Claim 19
Dependent Claim
137 words
A computer-implemented process of training context vectors for images within documents, the process comprising: for each of a plurality of images, generating a plurality of feature vectors from image data of the image; for each image, associating each of the image's feature vectors with a context vector; and for each image, aligning each of the context vectors of the image using a context vector of at least one word included in a document containing the image; wherein aligning each of the context vectors of the image comprises: adjusting the image context vector to be more similar to the summary vector of the at least one word included in the document containing the image; and comparing the image context vector with the summary vector by computing a dot product between the query context vector and the summary vector.
Claim 20
Dependent Claim
137 words
A computer-implemented process of training context vectors for images within documents, the process comprising: for each of a plurality of images, generating a plurality of feature vectors from image data of the image; for each image, associating each of the image's feature vectors with a context vector; and for each image, aligning each of the context vectors of the image using a context vector of at least one word included in a document containing the image; wherein aligning each of the context vectors of the image comprises: adjusting the image context vector to be more similar to the summary vector of the at least one word included in the document containing the image; and comparing the image context vector with the summary vector by computing a dot product between the query context vector and the summary vector.
Claim 21
Dependent Claim
32 words
The computer-implemented process of claim 17 , wherein associating each of the image's feature vectors with a context vector further comprises: initializing the context vectors to be substantially orthogonal to each other.
Claim 22
Dependent Claim
31 words
The computer-implemented process of claim 17 , wherein the context vectors are oriented in a vector space, the axes of the vector space not associated with selected words or image features.
Claim 23
Dependent Claim
189 words
A computer-implemented process of training context vectors for images within documents, the process comprising: providing a plurality of word context vectors, each context vector having an orientation in a vector space, such that words having similar meaning have context vectors with similar orientations in the vector space; providing a plurality of image context vectors, each image context vector associated with a feature vector, each feature vector derived from image data of at least one image, each image context vector having an orientation in the vector space; and for each document containing an image, aligning the image context vectors associated with the feature vectors derived from the image, with a summary vector derived from context vectors of selected words contained in the document; wherein aligning the image context vectors with a summary vector derived from context vectors of selected words contained in the document comprises: adjusting the image context vector to be more similar to the summary vector of the selected words included In the document: and comparing the image context vectors with the summary vector by computing a dot product between the query context vectors and the summary vector.
Claim 24
Dependent Claim
189 words
A computer-implemented process of training context vectors for images within documents, the process comprising: providing a plurality of word context vectors, each context vector having an orientation in a vector space, such that words having similar meaning have context vectors with similar orientations in the vector space; providing a plurality of image context vectors, each image context vector associated with a feature vector, each feature vector derived from image data of at least one image, each image context vector having an orientation in the vector space; and for each document containing an image, aligning the image context vectors associated with the feature vectors derived from the image, with a summary vector derived from context vectors of selected words contained in the document; wherein aligning the image context vectors with a summary vector derived from context vectors of selected words contained in the document comprises: adjusting the image context vector to be more similar to the summary vector of the selected words included In the document: and comparing the image context vectors with the summary vector by computing a dot product between the query context vectors and the summary vector.
Claim 25
Dependent Claim
31 words
The computer-implemented process of claim 20 , wherein the context vectors are oriented in the vector space, the axes of the vector space not associated with selected terms or image features.
Claim 26
Dependent Claim
164 words
A computer-implemented method of retrieving records having different media types, the method comprising: providing a plurality of first records, each first record having a first media type; for each of the first record having the first media type, deriving from elements of the first record a context vector, the context vector having an orientation in a vector space; providing a plurality of second records, each second record having a second media type; for each of the second record having the second media type, deriving from elements of the second record a context vector, the context vector having an orientation in the vector space; receiving a query, comprising at least one element of the first media type; deriving a query context vector from the query; and retrieving at least one second record having a context vector similar to the query context vector; wherein comparing the query vector with said context vector comprises computing a dot product between the query context vectors and the summary vector.
Claim 27
Dependent Claim
164 words
A computer-implemented method of retrieving records having different media types, the method comprising: providing a plurality of first records, each first record having a first media type; for each of the first record having the first media type, deriving from elements of the first record a context vector, the context vector having an orientation in a vector space; providing a plurality of second records, each second record having a second media type; for each of the second record having the second media type, deriving from elements of the second record a context vector, the context vector having an orientation in the vector space; receiving a query, comprising at least one element of the first media type; deriving a query context vector from the query; and retrieving at least one second record having a context vector similar to the query context vector; wherein comparing the query vector with said context vector comprises computing a dot product between the query context vectors and the summary vector.
References Cited (Backward Citations)
US4730259A
Cited
US4959870A
Cited
US5287275A
Cited
US5005206A
Cited
US5161204A
Cited
US5274714A
Cited
US5465308A
Cited
US5164992A
Cited
US5325298A
Cited
US5317507A
Cited
US5239594A
Cited
US5263097A
Cited
US5313534A
Cited
US5596659A
Cited
US5579471A
Cited
US5550928A
Cited
US5546477A
Cited
US5647058A
Cited
US5386103A
Cited
US5468069A
Cited
US5619709A
Cited
US5675819A
Cited
US5602589A
Cited
US5748786A
Cited
US5963670A
Cited
US5870754A
Cited
US6298174B1
Cited
US6134532A
Cited
US6389436B1
Cited
US6334129B1
Cited
US6523026B1
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2007-07-06 | 2008-01-05 | 2008-07-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2011-07-06 | 2012-01-05 | 2012-07-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2015-07-06 | 2016-01-05 | 2016-07-06 | $7700.00 | Window Closed / Lapsed |