Real-time document collection search engine with phrase indexing

Claim 2

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a collection index including first predetermined single word and multiple word phrases as indexed terms occurring in said collection of documents; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list using conditional indexing of said word phrases.

Claim 3

The collection search system of claim 1 wherein said collection is fractionally stored in a plurality of distributed servers.

Claim 4

The collection search system of claim 1 wherein said linguistic parser provides for eliding predetermined punctuation marks and stop list words, wherein said collection search engine.

Claim 5

The search system of claim 1 wherein said phrase document terms include proximity related single word terms, and wherein phrase document terms exclude a predetermined set of words.

Claim 6

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a collection index including first predetermined single word and multiple word phrases as indexed terms occurring in said collection of documents, said collection index storing one-word and two-word indexes of the frequency of occurrence of search phrases; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list.

Claim 7

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a collection index including first predetermined single word and multiple word phrases as indexed terms occurring in said collection of documents, said collection index storing one-word and two-word indexes of the frequency of occurrence of search phrases; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list.

Claim 8

The search system of claim 5 wherein three-word and more phrases are inferred from said one-word and two-word indexes of the frequency of occurrence of search phrases.

Claim 9

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a collection index including first predetermined single word and multiple word phrases as indexed terms occurring in said collection of documents, wherein said collection index is distributed in different subgroups which share common group statistics; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list.

Claim 10

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a collection index including first predetermined single word and multiple word phrases as indexed terms occurring in said collection of documents, wherein said collection index is distributed in different subgroups which share common group statistics; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list.

Claim 11

The search system of claim 7 wherein one or more of said subgroups is stored within high-speed memory whereby searches of said one or more of said subgroups do not require accesses to slower-speed memory.

Claim 12

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a distributed collection index including first predetermined one-word and two-word phrases as indexed terms occurring in said collection of documents; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list.

Claim 13

A collection search system responsive to a user query against a collection of documents to provide a search report, said collection search system comprising: a) a distributed collection index including first predetermined one-word and two-word phrases as indexed terms occurring in said collection of documents; b) a linguistic parser that identifies a list of search terms from a user query, said linguistic parser identifying said list from second predetermined single words and multiple word phrases; and c) a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify a predetermined document from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score related to the intersection of said predetermined document with said list.

Claim 14

A collection search system responsive to a user query, Q, for searching against a collection of documents, where each document is described as a string of words, S, where for an M-word document, S=S 1 , S 2 ,S 3 , . . . ,S m ,S m+1 ,S m+2 ,S m+3 , . . . ,SM, where m=1,2, . . . , M, where for a number, N g , of documents in a group g of documents in the collection, the documents are designated S1 g ,S2 g , . . . ,Sn g , . . . ,SN g for n g =1,2, . . . , N g , and where a typical predetermined document, Sn g , of said documents in the group g is given by, Sn g =Sn g ,1,Sn g ,2,Sn g ,3, . . . ,Sn g ,m3 Sn g ,m+1,Sn g ,m+2,Sn g ,m+3, . . . ,Sn g ,Mm, said collection search system comprising: a collection index including first predetermined one-word and two-word phrases as indexed terms occurring in said collection of documents; a linguistic parser that identifies from the user query, Q, a list of search terms Q 1 , Q 2 , . . . , Q p , . . . , Q p each having a weighted value, W Q1 ,W Q2 , . . . ,W Qp3 . . . , WQP, said linguistic parser identifying said list of search terms from second predetermined one-word and two-word phrases; and a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify the predetermined document Sn g from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score, score(Sn g ) Q , for the document Sn g based on the query Q as follows: ##EQU5## where (A Ng ) Qp =a document value relative to the number of occurrences of the term Qp in the documents in the group g, where W Qp =the value of the query term relative to the number of occurrences of the term Qp in the particular document Sn g .

Claim 15

A collection search system responsive to a user query, Q, for searching against a collection of documents, where each document is described as a string of words, S, where for an M-word document, S=S 1 , S 2 ,S 3 , . . . ,S m ,S m+1 ,S m+2 ,S m+3 , . . . ,SM, where m=1,2, . . . , M, where for a number, N g , of documents in a group g of documents in the collection, the documents are designated S1 g ,S2 g , . . . ,Sn g , . . . ,SN g for n g =1,2, . . . , N g , and where a typical predetermined document, Sn g , of said documents in the group g is given by, Sn g =Sn g ,1,Sn g ,2,Sn g ,3, . . . ,Sn g ,m3 Sn g ,m+1,Sn g ,m+2,Sn g ,m+3, . . . ,Sn g ,Mm, said collection search system comprising: a collection index including first predetermined one-word and two-word phrases as indexed terms occurring in said collection of documents; a linguistic parser that identifies from the user query, Q, a list of search terms Q 1 , Q 2 , . . . , Q p , . . . , Q p each having a weighted value, W Q1 ,W Q2 , . . . ,W Qp3 . . . , WQP, said linguistic parser identifying said list of search terms from second predetermined one-word and two-word phrases; and a search engine coupled to receive said list from said linguistic parser, said search engine intersecting said list with said collection index to identify the predetermined document Sn g from said collection of documents, said search engine including an accumulator for summing for said predetermined document a relevancy score, score(Sn g ) Q , for the document Sn g based on the query Q as follows: ##EQU5## where (A Ng ) Qp =a document value relative to the number of occurrences of the term Qp in the documents in the group g, where W Qp =the value of the query term relative to the number of occurrences of the term Qp in the particular document Sn g .

Claim 16

The search system of claim 10 wherein said documents are in different subgroups, sg p of the group, g, where g=sg 1 , sg 2 , . . . ,sg j , . . . ,sg G and the scoring is performed separately for each subgroup j using the statistics of the master subgroup, g, where score, score(Sn g ) Q {sg j }, for the particular document Sn g for a particular one subgroup {sg j } is determined as follows, ##EQU6## where W Qp {sg j }=the weighted value for the query collectively considering all the Nsg j documents in the subgroup sp j .

Claim 17

The search system of claim 11 wherein the total score SCORE(Sn g ) Q for the document Sn g for all the subgroups {sg j } for j=1,2, . . . , P is determined as follows: ##EQU7##