US5946648A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Identification of words in Japanese text by a computer system
Assignee: Microsoft Technology Licensing LLC
•
Filing: 1998-07-24
•
Granted: 1999-08-31
•
Expires: 2018-07-24
Abstract
Abstract A word breaking facility operates to identify words within a Japanese text string. The word breaking facility performs morphological processing to identify postfix bound morphemes and prefix bound morphemes. The word breaking facility also performs opheme matching to identify likely stem characters. A scoring heuristic is applied to determine an optimal analysis that includes a postfix analysis, a stem analysis, and a prefix analysis. The morphological analyses are stored in an efficient compressed format to minimize the amount of memory they occupy and maximize the analysis speed. The morphological analyses of postfixes, stems, and prefixes is performed in a right-to-left fashion. The word breaking facility may be used in applications that demand identity of selection granularity, autosummarization applications, content indexing applications, and natural language processing applications.
Inventors
- 👤 Patrick H. Halstead, Jr.
- 👤 Hisami Suzuki
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 17:37
CPC Classifications
G; G06; G06F; G06F40/00; G06F40/20; G06F40/268; G06F40/253; G06F40/279; G06F40/284; G06F40/40; G06F40/53
Claims Analysis
Total Claims: 19
Claim 1
Independent Claim
136 words
In a computer system, a method of processing, an input string of characters, comprising the computer-implemented steps of: identifying phrase breaks in the input string by applying a statistical technique, such that the phrase breaks delimit phrases; for each phrase within the input string, identifying possible postfix bound morphemes and creating a graphical representation of such morphemes; identifying possible stems in the phrase by comparing characters in the input string with stem character templates that each identify a pattern of character types in a stem; identifying possible prefix bound morphemes for each possible stem and creating a graphical representation of the prefix bound morphemes; and choosing a best combination of the possible postfix bound morphemes, stems and prefix bound morphemes as an analysis of the phrase in the input string that best characterizes the input string.
Claim 2
Dependent Claim
136 words
In a computer system, a method of processing, an input string of characters, comprising the computer-implemented steps of: identifying phrase breaks in the input string by applying a statistical technique, such that the phrase breaks delimit phrases; for each phrase within the input string, identifying possible postfix bound morphemes and creating a graphical representation of such morphemes; identifying possible stems in the phrase by comparing characters in the input string with stem character templates that each identify a pattern of character types in a stem; identifying possible prefix bound morphemes for each possible stem and creating a graphical representation of the prefix bound morphemes; and choosing a best combination of the possible postfix bound morphemes, stems and prefix bound morphemes as an analysis of the phrase in the input string that best characterizes the input string.
Claim 3
Dependent Claim
12 words
The method of claim 1 wherein the input string includes Japanese characters.
Claim 4
Dependent Claim
23 words
The method of claim 1 wherein the statistical technique used in identifying phrase breaks relies upon probabilities of breaking unigrams and breaking bigrams.
Claim 5
Dependent Claim
20 words
The method of claim 1 wherein the identifying of possible stems relies on probabilities of the stem character templates occurring.
Claim 6
Dependent Claim
126 words
In a computer system, a computer-readable storage medium holding instructions for: identifying phrase breaks in the input string by applying a statistical technique, such that the phrase breaks delimit phrases; for each phrase within the input string, identifying possible postfix bound morphemes and creating a graphical representation of such morphemes; identifying possible stems in the phrase by comparing characters in the input string with stem character templates that each identify a pattern of character types in a stem; identifying possible prefix bound morphemes for each possible stem and creating a graphical representation of the prefix bound morphemes; and choosing a best combination of the possible postfix bound morphemes, stems and prefix bound morphemes as an analysis of the input string that best characterizes the input string.
Claim 7
Dependent Claim
126 words
In a computer system, a computer-readable storage medium holding instructions for: identifying phrase breaks in the input string by applying a statistical technique, such that the phrase breaks delimit phrases; for each phrase within the input string, identifying possible postfix bound morphemes and creating a graphical representation of such morphemes; identifying possible stems in the phrase by comparing characters in the input string with stem character templates that each identify a pattern of character types in a stem; identifying possible prefix bound morphemes for each possible stem and creating a graphical representation of the prefix bound morphemes; and choosing a best combination of the possible postfix bound morphemes, stems and prefix bound morphemes as an analysis of the input string that best characterizes the input string.
Claim 8
Dependent Claim
25 words
The computer-readable storage medium of claim 5 wherein the statistical technique used in identifying phrase breaks relies upon probabilities of breaking unigrams and breaking bigrams.
Claim 9
Dependent Claim
22 words
The computer-readable storage medium of claim 5 wherein the identifying of possible stems relies on probabilities of the stem character templates occurring.
Claim 10
Dependent Claim
14 words
The computer-readable storage medium of claim 5 wherein the input string includes Japanese characters.
Claim 11
Dependent Claim
138 words
A computer system comprising: a phrase break analysis component for identifying hard phrase breaks in a character input string; an analyzer for analyzing the input string to identify possible stems and affixes in the input string; a tree generator for generating a tree representing the possible stems and affixes identified by the analyzer, the generated tree being comprised of nodes including a root node and leaf nodes, each path in the tree from the root node to a leaf node representing an alternative analysis of the input string; and a selector for using the generated tree to choose a stem and one or more affixes from among those that have been identified as a representation of at least a portion of the input string, and for identifying the combination of the chosen stem and affixes as a word.
Claim 12
Dependent Claim
138 words
A computer system comprising: a phrase break analysis component for identifying hard phrase breaks in a character input string; an analyzer for analyzing the input string to identify possible stems and affixes in the input string; a tree generator for generating a tree representing the possible stems and affixes identified by the analyzer, the generated tree being comprised of nodes including a root node and leaf nodes, each path in the tree from the root node to a leaf node representing an alternative analysis of the input string; and a selector for using the generated tree to choose a stem and one or more affixes from among those that have been identified as a representation of at least a portion of the input string, and for identifying the combination of the chosen stem and affixes as a word.
Claim 13
Dependent Claim
14 words
The computer system of claim 9 wherein the input string includes primarily Japanese characters.
Claim 14
Dependent Claim
53 words
The computer system of claim 9 wherein the computer system lacks a fully specified dictionary, the computer system performing a method comprising the steps of: providing an input string of characters, said characters lacking white space between characters; and processing the input string to identify words and stems and affixes in the words.
Claim 15
Dependent Claim
20 words
The computer system of claim 11 wherein the processing includes identifying postfix bound morphemes that are bound to the stem.
Claim 16
Dependent Claim
20 words
The computer system of claim 11 wherein the processing includes identifying prefix bound morphemes that are bound to the stems.
Claim 17
Dependent Claim
24 words
The computer system of claim 11, further comprising the step of using the identification of the words, stems and affixes in natural language processing.
Claim 18
Dependent Claim
27 words
The computer system of claim 11, further comprising the step of using the identification of the words, stems and affixes in content indexing of the input string.
Claim 19
Dependent Claim
34 words
The computer system of claim 11, further comprising the step of using the identification of the words, stem and affixes in an autosummarization application that seeks to produce a summary of the input string.
References Cited (Backward Citations)
US5490061A
Cited
EP0547222A1
Cited
US5497319A
Cited
US5590257A
Cited
US5268840A
Cited
US5535121A
Cited
US5799269A
Cited
US5794177A
Cited
US5778361A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-08-30 | 2003-03-01 | 2003-08-31 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-08-30 | 2007-03-01 | 2007-08-31 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-08-30 | 2011-03-01 | 2011-08-31 | $7700.00 | Window Closed / Lapsed |