US6006221A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Multilingual document retrieval system and method using semantic vector matching
Assignee: Syracuse University
•
Filing: 1996-08-14
•
Granted: 1999-12-21
•
Expires: 2016-08-14
Abstract
Abstract A document retrieval system where a user can enter a query, including a natural language query, in a desired one of a plurality of supported languages, and retrieve documents from a database that includes documents in at least one other language of the plurality of supported languages. The user need not have any knowledge of the other languages. Each document in the database is subjected to a set of processing steps to generate a language-independent conceptual representation of the subject content of the document. This is normally done before the query is entered. The query is also subjected to a (possibly different) set of processing steps to generate a language-independent conceptual representation of the subject content of the query. The documents and queries can also be subjected to additional analysis to provide additional term-based representations, such as the extraction of information-rich terms and phrases (such as proper nouns). Documents are matched to queries based on the conceptual-level contents of the document and query, and, optionally, on the basis of the term-based representation. The query's representation is then compared to each document's representation to generate a measure of relevance of the document to the query.
Inventors
- 👤 Elizabeth D. Liddy
- 👤 Woojin Paik
- 👤 Edmund S. Yu
- 👤 Ming Li
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-26 21:20
CPC Classifications
G; G06; G06F; G06F16/00; G06F16/30; G06F16/33; G06F16/20; G06F16/24; G06F16/245; G06F16/2452; G06F16/24526; G06F16/31; G06F16/313; G06F16/3331; G06F16/3332; G06F16/3334; G06F16/3337; G06F16/3338; G06F16/90; G06F16/93; Y; Y10; Y10S; Y10S707/00; Y10S707/99931; Y10S707/99933; Y10S707/99935
Claims Analysis
Total Claims: 38
Claim 1
Independent Claim
97 words
A method of representing documents in a database that includes documents in a plurality of languages, the method carried out for each document, comprising: determining a set of potential conceptual-level meanings of at least some words in the document from a language-independent multilingual concept database that reflects the plurality of languages and comprises a collection of concept groups; mapping the sets of potential conceptual-level meanings, so determined, to respective single language-independent conceptual-level meanings; and generating a language-independent conceptual representation of the subject content of the document based on the language-independent conceptual-level meanings determined in said mapping step.
Claim 2
Independent Claim
97 words
A method of representing documents in a database that includes documents in a plurality of languages, the method carried out for each document, comprising: determining a set of potential conceptual-level meanings of at least some words in the document from a language-independent multilingual concept database that reflects the plurality of languages and comprises a collection of concept groups; mapping the sets of potential conceptual-level meanings, so determined, to respective single language-independent conceptual-level meanings; and generating a language-independent conceptual representation of the subject content of the document based on the language-independent conceptual-level meanings determined in said mapping step.
Claim 3
Dependent Claim
19 words
The method of claim 1, and further comprising, determining the language of the document for at least some documents.
Claim 4
Dependent Claim
20 words
The method of claim 1, and further comprising generating a term-based representation of the document for at least some documents.
Claim 5
Dependent Claim
25 words
The method of claim 3 wherein the term-based representation of the document is a representation of a set of proper nouns found in the document.
Claim 6
Dependent Claim
24 words
The method of claim 4 wherein the set of proper nouns found in the document are represented as categories from a hierarchical classification scheme.
Claim 7
Dependent Claim
25 words
The method of claim 3 wherein the term-based representation of the document is a representation of a set of noun phrases found in the document.
Claim 8
Dependent Claim
101 words
The method of claim 1 wherein a given one of said words is polysemous, giving rise to multiple conceptual-level meanings from the multilingual concept database, and said mapping, for the given word, comprises: disambiguating among the multiple conceptual-level meanings from the multilingual concept database to provide a single multilingual conceptual-level meaning; mapping the single multilingual conceptual-level meaning to a set of monolingual concept categories in a monolingual concept dictionary; and if the set of monolingual concept categories from the monolingual concept dictionary contains multiple monolingual concept categories, disambiguating among the multiple monolingual concept categories to provide the single language-independent conceptual-level meaning.
Claim 9
Dependent Claim
63 words
The method of claim 7 wherein disambiguating includes: analyzing local context information to attempt to determine a single meaning; if a single meaning is not determined from analyzing local context information, analyzing domain knowledge to attempt to determine a single meaning; and if a single meaning is not determined from analyzing domain knowledge, analyzing global information to attempt to determine a single meaning.
Claim 10
Dependent Claim
26 words
The method of claim 7, and further comprising providing a gloss transliteration using the single multilingual conceptual-level meaning derived from disambiguating among the multiple conceptual-level meanings.
Claim 11
Dependent Claim
24 words
The method of claim 1 wherein the collection of concept groups includes words or phrases, from the plurality of languages, that are conceptually synonymous.
Claim 12
Dependent Claim
51 words
The method of claim 1, and further comprising: determining a measure of proximity of the language-independent conceptual representation of each document to the language-independent conceptual representation of the other documents in the plurality; and clustering the documents in the plurality according to the documents' respective measures of proximity to each other.
Claim 13
Independent Claim
176 words
A method of retrieving documents in response to a query, the query being in a user-selected language of a plurality of languages, the method comprising: providing a corpus of documents, each in a language of said plurality of languages, at least one of the documents being in a language other than the user-selected language; for each document; determining a set of multilingual concepts of at least some words in the document using a language-independent multilingual concept database that reflects the plurality of languages and comprises a collection of concept groups; mapping the sets of multilingual concepts, so determined, to respective single language-independent conceptual-level meanings; and generating a language-independent conceptual representation of the subject content of the document based on the language-independent conceptual-level meanings determined in said mapping; generating a language-independent conceptual representation of the subject content of the query; and for each document, generating a measure of relevance of the document to the query using the conceptual representation of the subject content of the document and the conceptual representation of the subject content of the query.
Claim 14
Independent Claim
176 words
A method of retrieving documents in response to a query, the query being in a user-selected language of a plurality of languages, the method comprising: providing a corpus of documents, each in a language of said plurality of languages, at least one of the documents being in a language other than the user-selected language; for each document; determining a set of multilingual concepts of at least some words in the document using a language-independent multilingual concept database that reflects the plurality of languages and comprises a collection of concept groups; mapping the sets of multilingual concepts, so determined, to respective single language-independent conceptual-level meanings; and generating a language-independent conceptual representation of the subject content of the document based on the language-independent conceptual-level meanings determined in said mapping; generating a language-independent conceptual representation of the subject content of the query; and for each document, generating a measure of relevance of the document to the query using the conceptual representation of the subject content of the document and the conceptual representation of the subject content of the query.
Claim 15
Dependent Claim
13 words
The method of claim 12 wherein the query is a natural language query.
Claim 16
Dependent Claim
30 words
The method of claim 12 wherein generating a language-independent conceptual representation of the subject content of the document comprises: generating a conceptual-level vector representing the subject content of the document.
Claim 17
Dependent Claim
69 words
The method of claim 12 wherein; the concept groups include a collection of synonyms and near-synonyms of a given word or phrase in said plurality of languages; a given word in the document is polysemous, giving rise to multiple conceptual-level meanings from the multilingual concept database; and for the given word, mapping the sets of multilingual concepts to respective single language-independent conceptual-level meanings comprises: disambiguating the multiple conceptual-level meanings.
Claim 18
Dependent Claim
80 words
The method of claim 15 wherein disambiguating the multiple conceptual-level meanings comprises: disambiguating among the multiple conceptual-level meanings from the multilingual concept database to provide a single multilingual conceptual-level meaning; mapping the single multilingual conceptual-level meaning to a set of monolingual concept categories in a monolingual concept dictionary; and if the set of monolingual concept categories from the monolingual concept dictionary contains multiple monolingual concept categories, disambiguating among the multiple monolingual concept categories to provide the single language-independent conceptual-level meaning.
Claim 19
Dependent Claim
63 words
The method of claim 16 wherein disambiguating includes: analyzing local context information to attempt to determine a single meaning; if a single meaning is not determined from analyzing local context information, analyzing domain knowledge to attempt to determine a single meaning; and if a single meaning is not determined from analyzing domain knowledge, analyzing global information to attempt to determine a single meaning.
Claim 20
Dependent Claim
26 words
The method of claim 16, and further comprising providing a gloss transliteration using the single multilingual conceptual-level meaning derived in disambiguating among the multiple conceptual-level meanings.
Claim 21
Dependent Claim
26 words
The method of claim 12, and further comprising providing a gloss transliteration of at least some of the words in at least one of the documents.
Claim 22
Dependent Claim
33 words
The method of claim 12 wherein said language-independent conceptual representation of the subject content of the document is augmented by a language-dependent statistical index using words in the document's language as indexing units.
Claim 23
Dependent Claim
29 words
The method of claim 12 wherein said language-independent conceptual representation of the subject content of the document includes a statistical index using N-gram style decomposed words as indexing units.
Claim 24
Dependent Claim
41 words
The method of claim 12 wherein generating a language-independent conceptual representation of the subject content of the query comprises: mapping words or phrases in the query into language-independent concepts; and generating a conceptual-level vector representing the subject content of the query.
Claim 25
Dependent Claim
26 words
The method of claim 12 wherein said language-independent conceptual representation of the subject content of the query includes N-gram style decomposed terms as language-independent query requirements.
Claim 26
Dependent Claim
48 words
The method of claim 12 wherein said language-independent conceptual representation of the subject content of the query is augmented using a language-dependent logic requirement, the logic requirement including terms and logical connectives where the terms include the query term and its synonymous terms in the multilingual concept database.
Claim 27
Dependent Claim
56 words
The method of claim 12, and further comprising: providing a list of at least some of the documents; receiving user input specifying at least one document, or a part thereof, on the list; and generating a revised query representation based on the original query plus a representation of the specified document or documents, or parts thereof.
Claim 28
Dependent Claim
18 words
The method of claim 12, and further providing a relevance-ranked list of at least some of the documents.
Claim 29
Dependent Claim
25 words
The method of claim 26, wherein the number of documents in the relevance-ranked list of documents is calculated based on a user-specified level of recall.
Claim 30
Dependent Claim
36 words
The method of claim 26, wherein the number of documents in the relevance-ranked list of documents is calculated based on a user-specified level of recall and a user-specified level of confidence in that level of recall.
Claim 31
Dependent Claim
25 words
The method of claim 26, wherein retrieved documents in the relevance-ranked list of documents are ranked without regard to the language they are written in.
Claim 32
Dependent Claim
21 words
The method of claim 12, and further determining the language of the document before generating a language-independent representation of the document.
Claim 33
Dependent Claim
51 words
The method of claim 12 wherein generating a measure of relevance for a given document comprises: generating conceptual-level vectors for the given document and for the query; and determining a distance between the vectors, the distance representing the measure of relevance, with a smaller distance representing a higher degree of relevance.
Claim 34
Dependent Claim
58 words
The method of claim 12 wherein generating a measure of relevance for a given document comprises: generating an N-gram decomposed term representation for the given document and for the query; and determining a degree of overlap between the N-gram decomposed terms, the overlap representing the measure of relevance, with a larger overlap representing a higher degree of relevance.
Claim 35
Dependent Claim
68 words
The method of claim 12 wherein generating a measure of relevance for a given document comprises: generating word representations for the given document and for the query; organizing words in the query as logical requirements; and determining a coverage of terms in the documents against the logical requirement of a query, the coverage representing the measure of relevance, with a larger coverage representing a higher degree of relevance.
Claim 36
Independent Claim
145 words
A method of retrieving documents in response to a query in a user-selected language of a plurality of languages, the method comprising: (a) providing a corpus of documents, each in a language of said plurality of languages, at least some of the documents being in a language other than the user-selected language; (b) processing each document by determining the language of the document, determining conceptual-level meaning of at least some words in the document from a language-independent multilingual concept database comprising a collection of concept groups, mapping the conceptual-level meanings into language-independent concepts, and generating a conceptual-level vector representing the subject content of the document; (c) processing the query by mapping words or phrases in the query into language-independent concepts, and generating a conceptual-level vector representing the subject content of the query; and (d) for each document, determining a measure of relevance to the query.
Claim 37
Independent Claim
145 words
A method of retrieving documents in response to a query in a user-selected language of a plurality of languages, the method comprising: (a) providing a corpus of documents, each in a language of said plurality of languages, at least some of the documents being in a language other than the user-selected language; (b) processing each document by determining the language of the document, determining conceptual-level meaning of at least some words in the document from a language-independent multilingual concept database comprising a collection of concept groups, mapping the conceptual-level meanings into language-independent concepts, and generating a conceptual-level vector representing the subject content of the document; (c) processing the query by mapping words or phrases in the query into language-independent concepts, and generating a conceptual-level vector representing the subject content of the query; and (d) for each document, determining a measure of relevance to the query.
Claim 38
Dependent Claim
27 words
The method of claim 34 wherein mapping the conceptual-level meanings into language-independent concepts comprises disambiguating multiple senses of polysemous words and phrases to generate the language-independent concepts.
References Cited (Backward Citations)
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-12-20 | 2003-06-21 | 2003-12-21 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-12-20 | 2007-06-21 | 2007-12-21 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-12-20 | 2011-06-21 | 2011-12-21 | $7700.00 | Window Closed / Lapsed |