US6009382A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Word storage table for natural language determination
Assignee: International Business Machines Corp
•
Filing: 1996-09-30
•
Granted: 1999-12-28
•
Expires: 2016-09-30
Abstract
Abstract A language in which a document is written is identified through the use of sets of most frequently used words in each of a plurality of candidate languages. Each set of most frequently used words in a respective set of word tables for a respective candidate language according to letter pairs in each set of most frequently used words. In the preferred embodiment, each word table is an N×N bit table, where each bit represents a given letter pair at a particular place in one of the most frequently used words in one of the candidate languages. Words from the document are compared to the most frequently used words stored in the word tables. A count of the number of matches between the words from the document and the words stored in each respective set of word tables is kept for each respective language. The language of the document as the respective candidate language having the greatest number of matches.
Inventors
- 👤 Michael John Martino
- 👤 Robert Charles Paulsen, Jr.
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-03 12:19
CPC Classifications
G; G06; G06F; G06F40/00; G06F40/20; G06F40/205; G06F40/216; G06F40/263
Claims Analysis
Total Claims: 21
Claim 1
Independent Claim
117 words
A method for identifying a plurality of character strings, comprising the steps of: selecting a set of character strings; storing the set of character strings in a word table as a set of ordered character pairs, wherein each word table is an N×N bit table, wherein each bit represents a given character pair at a particular place in one of the character strings; comparing new character strings to the character strings stored in the word tables; counting a number of matches between the new character strings and the character strings in the word table; identifying the new character strings as related to the character strings in the word table if there are a sufficient number of matches.
Claim 2
Independent Claim
117 words
A method for identifying a plurality of character strings, comprising the steps of: selecting a set of character strings; storing the set of character strings in a word table as a set of ordered character pairs, wherein each word table is an N×N bit table, wherein each bit represents a given character pair at a particular place in one of the character strings; comparing new character strings to the character strings stored in the word tables; counting a number of matches between the new character strings and the character strings in the word table; identifying the new character strings as related to the character strings in the word table if there are a sufficient number of matches.
Claim 3
Dependent Claim
62 words
The method as recited in claim 1 wherein the character strings are words, the set of character strings in the word table are words from a natural language, the new character strings are words from a document and if there are a sufficient number of matches the new character strings are identified as being in the natural language of the word table.
Claim 4
Independent Claim
154 words
A method for identifying a language in which a document is written, comprising the steps of: selecting a set of most frequently used words in each of a plurality of candidate languages; storing each set of most frequently used words in a respective set of word tables for a respective candidate language according to letter pairs in each set of most frequently used words, wherein each word table is an N×N bit table, wherein each bit represents a given letter pair at a Particular place in one of the most frequently used words in one of the candidate language; comparing words from the document to the most frequently used words stored in the word tables; counting a number of matches between the words from the document and the words stored in each respective set of word tables; identifying the language of the document as the respective candidate language having the greatest number of matches.
Claim 5
Independent Claim
154 words
A method for identifying a language in which a document is written, comprising the steps of: selecting a set of most frequently used words in each of a plurality of candidate languages; storing each set of most frequently used words in a respective set of word tables for a respective candidate language according to letter pairs in each set of most frequently used words, wherein each word table is an N×N bit table, wherein each bit represents a given letter pair at a Particular place in one of the most frequently used words in one of the candidate language; comparing words from the document to the most frequently used words stored in the word tables; counting a number of matches between the words from the document and the words stored in each respective set of word tables; identifying the language of the document as the respective candidate language having the greatest number of matches.
Claim 6
Dependent Claim
17 words
The method as recited in claim 1 wherein special characters are also represented in the word tables.
Claim 7
Dependent Claim
36 words
The method as recited in claim 4 wherein one of the special characters is a blank character so that most frequently used words of different lengths can be stored in the same set of word tables.
Claim 8
Dependent Claim
38 words
The method as recited in claim 1 wherein each respective set of tables contains tableaus of tables for words of a respective length so that all the words in a given tableau of tables are a given length.
Claim 9
Dependent Claim
23 words
The method as recited in claim 6 wherein some of the most frequently used words in at least one candidate language are truncated.
Claim 10
Independent Claim
149 words
A system including processor and memory for identifying a language in which a document is written, comprising: a plurality of word tables, each for a respective candidate language in which a set of most frequently used words are stored according to letter pairs in each of the most frequently used words, wherein each word table is an N×N bit table, wherein each bit represents a given letter pair at a Particular place in one of the most frequently used words in one of the candidate language; a comparator for comparing words from the document to the most frequently used words stored in the word tables; an accumulator for counting a number of matches between the words from the document and the words stored in each respective set of word tables; means for identifying the language of the document as the respective candidate language having the greatest number of matches.
Claim 11
Independent Claim
149 words
A system including processor and memory for identifying a language in which a document is written, comprising: a plurality of word tables, each for a respective candidate language in which a set of most frequently used words are stored according to letter pairs in each of the most frequently used words, wherein each word table is an N×N bit table, wherein each bit represents a given letter pair at a Particular place in one of the most frequently used words in one of the candidate language; a comparator for comparing words from the document to the most frequently used words stored in the word tables; an accumulator for counting a number of matches between the words from the document and the words stored in each respective set of word tables; means for identifying the language of the document as the respective candidate language having the greatest number of matches.
Claim 12
Dependent Claim
39 words
The system as recited in claim 8 wherein the plurality of word tables further comprises a plurality of tableaus of word tables for each respective candidate language, each tableau of word tables for storing words of a respective length.
Claim 13
Dependent Claim
41 words
The system as recited in claim 9 further comprising: length determining means for determining lengths of words in the document; word routing means for sending each word in the document to the tableaus according to the determined length of the word.
Claim 14
Dependent Claim
53 words
The system as recited in claim 10 further comprising: means for truncating words from the document which exceed the length of the words stored in the tableaus storing words of the longest respective length; wherein the word routing means routes the truncated words to the tableaus storing words of the longest respective length.
Claim 15
Dependent Claim
24 words
The system as recited in claim 8 wherein the words in the word tables were chosen to avoid strong aliasing between respective candidate languages.
Claim 16
Dependent Claim
28 words
The system as recited in claim 8 wherein the words in the word tables for each respective candidate language cover a substantially equivalent portion of each candidate language.
Claim 17
Dependent Claim
148 words
A computer program product on a computer readable medium for identifying a language in which a document is written, comprising: means for providing a plurality of word tables, each arranged in tableaus for storing words of a respective length and in a respective candidate language according to letter pairs in each of the stored words, wherein each word table is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in one of the candidate language; means for comparing words from the document to the words stored in the word tables; means for counting a number of matches between the words from the document and the words stored in each respective set of word tables; means for identifying the language of the document as the respective candidate language having the greatest number of matches.
Claim 18
Dependent Claim
148 words
A computer program product on a computer readable medium for identifying a language in which a document is written, comprising: means for providing a plurality of word tables, each arranged in tableaus for storing words of a respective length and in a respective candidate language according to letter pairs in each of the stored words, wherein each word table is an N×N bit table, wherein each bit represents a given letter pair at a particular place in one of the most frequently used words in one of the candidate language; means for comparing words from the document to the words stored in the word tables; means for counting a number of matches between the words from the document and the words stored in each respective set of word tables; means for identifying the language of the document as the respective candidate language having the greatest number of matches.
Claim 19
Dependent Claim
28 words
The program as recited in claim 14 wherein the words stored in the word tables are a set of most frequently used words in a respective candidate language.
Claim 20
Dependent Claim
32 words
The product as recited in claim 14 further comprising means for stopping the comparing and counting means when the number of matches between respective candidate languages reaches a predetermined degree of divergence.
Claim 21
Dependent Claim
21 words
The product as recited in claim 14 further comprising means for transmitting the product over a network to a computer system.
References Cited (Backward Citations)
US4610025A
Cited
US4750122A
Cited
US4698758A
Cited
US5175803A
Cited
US4829580A
Cited
US4773009A
Cited
WO1991002347A1
Cited
US5062143A
Cited
US5182708A
Cited
US5392419A
Cited
US5371807A
Cited
US5418951A
Cited
US5548507A
Cited
US5551020A
Cited
US5715446A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-12-27 | 2003-06-28 | 2003-12-28 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-12-27 | 2007-06-28 | 2007-12-28 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-12-27 | 2011-06-28 | 2011-12-28 | $7700.00 | Window Closed / Lapsed |