Text storage and retrieval system and method

Claim 2

A computer-based system for storing text, comprising: a lexical parser which divides said text into search terms and all other information; a word vector table which stores said search terms solely as an inverted structure, wherein each said search term is stored uniquely and each said search term is coupled with a list of its positions within said text; and a first auxiliary data structure which stores only said other information; wherein said word vector table and said first auxiliary data structure contain sufficient information substantially to reproduce said text.

Claim 3

The system of claim 1, wherein said positions are counted from the beginning of the text, each search term in said text is counted as one position, and each instance of other information between said search terms is counted as one or more positions.

Claim 4

The system of claim 1, wherein said text contains words, and said search terms are the approximate lemmas of said words.

Claim 5

The system of claim 1, wherein said text contains words, and said search terms are compressed representations of said words.

Claim 6

The system of claim 1, wherein said lists of positions are compressed before storage in said word vector table.

Claim 7

The system of claim 5, wherein said lists are compressed at least in part by sequential differencing.

Claim 8

The system of claim 1, wherein said search terms are stored in said word vector table by reference to a second auxiliary data structure, said second auxiliary data structure storing said search terms.

Claim 9

The system of claim 7, wherein said text contains words, and said search terms are the approximate lemmas of said words.

Claim 10

The system of claim 7, wherein said text contains words, and said search terms are compressed representations of said words.

Claim 11

The system of claim 7, wherein said lists of positions are compressed before storage in said word vector table.

Claim 12

The system of claim 10, wherein said lists of positions are compressed at least in part by sequential differencing.

Claim 13

The system of claim 1, wherein said text comprises one or more paragraphs having paragraph formats, said formats including at least the margins of said paragraphs, and wherein said paragraph formats are stored in said first auxiliary data structure as a table of said formats and a list of references to said table.

Claim 14

A computer-based method for storing text, comprising the steps of: dividing said text into search terms and all other information; storing said search terms solely as an inverted structure, wherein each said search term is stored uniquely and each said search term is coupled with a list of its positions within said text; and storing only said other information in a first auxiliary data structure.

Claim 15

A computer-based method for storing text, comprising the steps of: dividing said text into search terms and all other information; storing said search terms solely as an inverted structure, wherein each said search term is stored uniquely and each said search term is coupled with a list of its positions within said text; and storing only said other information in a first auxiliary data structure.

Claim 16

The method of claim 13, wherein said text contains words, and said search terms are the approximate lemmas of said words.

Claim 17

The method of claim 13, additionally comprising the step of compressing said inverted structure after said first storing step.

Claim 18

A computer-based method for compressing a particular string of symbols, comprising the steps of: providing a large number of strings similar to said particular string; assigning a value to nearly all substrings contained within said large number of strings, said value being equal to the frequency of occurrence of said substring in said large number of strings, multiplied by the length of said substring; storing the highest valued substrings in a list; assigning to each substring in said list a unique fixed-length code; and replacing each substring contained both within said particular string and said list with the code corresponding to said substring.

Claim 19

A computer-based method for compressing a particular string of symbols, comprising the steps of: providing a large number of strings similar to said particular string; assigning a value to nearly all substrings contained within said large number of strings, said value being equal to the frequency of occurrence of said substring in said large number of strings, multiplied by the length of said substring; storing the highest valued substrings in a list; assigning to each substring in said list a unique fixed-length code; and replacing each substring contained both within said particular string and said list with the code corresponding to said substring.

Claim 20

A computer-based method for compressing a particular string of symbols, comprising the steps of: providing a large number of strings similar to said particular string; providing a temporary list of substrings contained within said large number of strings; assigning a value to all substrings in said temporary list, said value being equal to the frequency of occurrence of said substring in said large number of strings, multiplied by the length of said substring; subtracting from each string in said large number of strings each substring contained in said temporary list; assigning a value to all remaining substrings in said large number of strings, said value being equal to the frequency of occurrence of said substring in said large number of strings, multiplied by the length of said substring; replacing all substrings in said temporary list having a value lower than any said remaining substring with the highest valued said remaining substrings; iterating said assigning, subtracting, assigning, and replacing steps until said temporary list is substantially unchanged from the previous iteration; assigning to each substring in said temporary list a unique fixed-length code; and replacing each substring contained both within said particular string and said list with the code corresponding to said substring.

Claim 21

A computer-based method for compressing a particular string of symbols, comprising the steps of: providing a large number of strings similar to said particular string; providing a temporary list of substrings contained within said large number of strings; assigning a value to all substrings in said temporary list, said value being equal to the frequency of occurrence of said substring in said large number of strings, multiplied by the length of said substring; subtracting from each string in said large number of strings each substring contained in said temporary list; assigning a value to all remaining substrings in said large number of strings, said value being equal to the frequency of occurrence of said substring in said large number of strings, multiplied by the length of said substring; replacing all substrings in said temporary list having a value lower than any said remaining substring with the highest valued said remaining substrings; iterating said assigning, subtracting, assigning, and replacing steps until said temporary list is substantially unchanged from the previous iteration; assigning to each substring in said temporary list a unique fixed-length code; and replacing each substring contained both within said particular string and said list with the code corresponding to said substring.

Claim 22

A computer-based method for storing text, comprising the steps of: dividing said text into search terms and all other information; compressing said search terms using the method of claim 16; storing said search terms solely as an inverted structure, wherein each said search term is stored uniquely and each said search term is coupled with a list of its positions within said text; and storing said other information in a first auxiliary data structure.

Claim 23

A computer-based method for storing text, comprising the steps of: dividing said text into search terms and all other information; storing said other information in a first auxiliary data structure; compressing said search terms using the method of claim 16; storing said search terms in a second auxiliary data structure; and storing said search terms in an inverted structure, wherein each said search term is stored uniquely by reference to said second auxiliary data structure, and each said search term is coupled with a list of its positions within said text.

Claim 24

A computer-based method for storing text, comprising the steps of: dividing said text into search terms and all other information; storing said search terms solely as an inverted structure, wherein each said search term is stored uniquely and each said search term is coupled with a list of its positions within said text; compressing at least part of said other information using the method of claim 16; and storing said other information in a first auxiliary data structure.