US5913208A — Identifying duplicate documents from search resul…

Claim 1

Independent Claim 143 words

A method of automatically determining duplicate documents on a hit-list containing one or more duplicate documents and document instances, the hit-list having a hit-list record for each instance of the documents, each hit-list record having one or more attribute fields, each attribute field containing one or more attributes of the documents, the method comprising the steps of: selecting one or more of the attributes that are intrinsic attributes, the intrinsic attributes being established at a time of document creation and that are invariant with a location and replication of the document; generating a pair of the hit-list records associated with the documents and intrinsic attributes; comparing one or more of the intrinsic attributes of the pair of hit-list records; using the comparison of the intrinsic attributes of the pair of hit-list records to determine if the documents are instances of the same document.

Claim 2

Independent Claim 143 words

A method of automatically determining duplicate documents on a hit-list containing one or more duplicate documents and document instances, the hit-list having a hit-list record for each instance of the documents, each hit-list record having one or more attribute fields, each attribute field containing one or more attributes of the documents, the method comprising the steps of: selecting one or more of the attributes that are intrinsic attributes, the intrinsic attributes being established at a time of document creation and that are invariant with a location and replication of the document; generating a pair of the hit-list records associated with the documents and intrinsic attributes; comparing one or more of the intrinsic attributes of the pair of hit-list records; using the comparison of the intrinsic attributes of the pair of hit-list records to determine if the documents are instances of the same document.

Claim 3

Dependent Claim 70 words

A method, as in claim 1, further comprising the steps of: repeating steps a through c for one or more pairs of hit-list records, each pair of hit-list records associated with a pair of compared documents, and noting that compared documents with attributes that are not equal are not instances of the same document; determining that documents in the pair that are not noted are duplicate instances of one another.

Claim 4

Dependent Claim 26 words

A method, as in claim 1, where one of the intrinsic attributes is a score that is a function of one or more other intrinsic attributes.

Claim 5

Dependent Claim 21 words

A method, as in claim 3, where the other intrinsic attributes are those intrinsic attributes based on content of the document.

Claim 6

Dependent Claim 27 words

A method, as in claim 4, where the other intrinsic attributes include any one of the following: document length, title, concepts, author, date of publication, and abstract.

Claim 7

Dependent Claim 20 words

A method, as in claim 3, where the function includes any one of the following: probabilistic measures, cosine similarity measures.

Claim 8

Dependent Claim 18 words

A method, as in claim 3, where the other intrinsic attributes are those intrinsic attributes on the hit-list.

Claim 9

Dependent Claim 42 words

A method, as in claim 1, where one of the intrinsic attributes is a score that is a function of one or more other intrinsic attributes and one or more query elements of a query that is used to generate the hit-list.

Claim 10

Dependent Claim 56 words

A method, as in claim 8, where the score also includes a portion that is based on properties that are not intrinsic attributes of the document but that are the same for every document of the hit-list because the properties are properties of the entire collection of documents and are determined at a time of query.

Claim 11

Independent Claim 189 words

A method of automatically determining duplicate documents on a hit-list containing one or more documents and document instances, the hit-list having a hit-list record for each instance of the documents, each hit-list record having one or more attribute fields, each attribute field containing one or more attribute of the documents, the method comprising the steps of: selecting one or more of the attributes that are intrinsic attributes, the intrinsic attributes being established at a time of document creation and that are invariant with a location and replication of the document; generating a pair of the hit-list records associated with the documents and the intrinsic attributes; comparing one or more of the intrinsic attributes of the pair of hit-list records selecting one or more of the attributes that are non intrinsic attributes, the non intrinsic attributes being variable with one or more document instance; comparing one or more of the non intrinsic attributes of the pair of hit-list records previously compared; noting that the pair of hit-list records failing a comparison test of the comparing of the intrinsic and non intrinsic attributes results in instances of not the same document.

Claim 12

Independent Claim 189 words

A method of automatically determining duplicate documents on a hit-list containing one or more documents and document instances, the hit-list having a hit-list record for each instance of the documents, each hit-list record having one or more attribute fields, each attribute field containing one or more attribute of the documents, the method comprising the steps of: selecting one or more of the attributes that are intrinsic attributes, the intrinsic attributes being established at a time of document creation and that are invariant with a location and replication of the document; generating a pair of the hit-list records associated with the documents and the intrinsic attributes; comparing one or more of the intrinsic attributes of the pair of hit-list records selecting one or more of the attributes that are non intrinsic attributes, the non intrinsic attributes being variable with one or more document instance; comparing one or more of the non intrinsic attributes of the pair of hit-list records previously compared; noting that the pair of hit-list records failing a comparison test of the comparing of the intrinsic and non intrinsic attributes results in instances of not the same document.

Claim 13

Dependent Claim 40 words

A method, as in claim 10, further comprising the steps of: repeating steps a through for one or more pairs of hit-list records; f. determining that documents in the pair that are not noted are duplicate instances of one another.

Claim 14

Dependent Claim 22 words

A method, as in claim 10, where one of the non intrinsic attributes is document location that includes a file name component.

Claim 15

Dependent Claim 21 words

A method, as in claim 12, where the comparison test is whether the file name component of the pair are equivalent.

Claim 16

Dependent Claim 21 words

A method, as in claim 12, where the file name component is last component of a URL in a network environment.

Claim 17

Dependent Claim 23 words

A method, as in claim 12, where the file name component is an implicit last component of a URL in a network environment.

Claim 18

Independent Claim 141 words

A method of automatically determining duplicate documents on a hit-list containing one or more documents and document instances, the hit-list having a hit-list record for each instance of the documents, each hit-list record having one or more attribute fields, each attribute field containing an attribute of the documents, the method comprising the steps of: a. selecting one or more of the attributes that are intrinsic attributes, intrinsic attributes being attributes that are established at a time of document creation and that are invariant with a location and replication of the document; b. sorting the hit-list using all of the intrinsic attributes as sort keys; c. comparing one or more intrinsic attributes of one or more of the adjacent documents on the sorted hit-list; and d. noting that documents with attributes that are not equal are not instances of the same document.

Claim 19

Independent Claim 141 words

A method of automatically determining duplicate documents on a hit-list containing one or more documents and document instances, the hit-list having a hit-list record for each instance of the documents, each hit-list record having one or more attribute fields, each attribute field containing an attribute of the documents, the method comprising the steps of: a. selecting one or more of the attributes that are intrinsic attributes, intrinsic attributes being attributes that are established at a time of document creation and that are invariant with a location and replication of the document; b. sorting the hit-list using all of the intrinsic attributes as sort keys; c. comparing one or more intrinsic attributes of one or more of the adjacent documents on the sorted hit-list; and d. noting that documents with attributes that are not equal are not instances of the same document.

Claim 20

Dependent Claim 20 words

A method, as in claim 16, where an additional sort key includes a non intrinsic attribute that is file name.

Claim 21

Dependent Claim 215 words

A computer system of one or more computers comprising: one or more memory storage devices containing a document collection of one or more documents; an index including an inverted file with one or more terms, each term associated with one or more document identifiers, the index further including a document catalog that associates each of the document identifiers with one or more attributes; a search engine process that processes a query with one or more query elements to produce a hit list having one or more hit list entry, each hit list entry associated with one of the documents that is determined by the search engine to be relevant to the query; one or more hit list attributes associated with each of the hit list entries, each of the hit list attributes being one of the attributes; and a formatter processor that identifies duplicate hit list entries by selecting one or more of the hit list attributes as selected attributes, identified by a hit list attribute selector, the formatter processor further selecting two or more hit list entries, called compared entries, that are each associated with one of the documents, each called a compared document, and the formatter process comparing the selected attributes to determine if the compared documents are duplicate instances of one another.

Claim 22

Dependent Claim 215 words

A computer system of one or more computers comprising: one or more memory storage devices containing a document collection of one or more documents; an index including an inverted file with one or more terms, each term associated with one or more document identifiers, the index further including a document catalog that associates each of the document identifiers with one or more attributes; a search engine process that processes a query with one or more query elements to produce a hit list having one or more hit list entry, each hit list entry associated with one of the documents that is determined by the search engine to be relevant to the query; one or more hit list attributes associated with each of the hit list entries, each of the hit list attributes being one of the attributes; and a formatter processor that identifies duplicate hit list entries by selecting one or more of the hit list attributes as selected attributes, identified by a hit list attribute selector, the formatter processor further selecting two or more hit list entries, called compared entries, that are each associated with one of the documents, each called a compared document, and the formatter process comparing the selected attributes to determine if the compared documents are duplicate instances of one another.

Claim 23

Dependent Claim 39 words

A computer system, as in claim 18, where there is one computer that is a single stand alone work station including the memory storage devices, the index, the search engine process, the hit list attributes, and the formatter processor.

Claim 24

Dependent Claim 19 words

A computer system, as in claim 19, where the single stand alone work station is connected to a network.

Claim 25

Dependent Claim 20 words

A computer system, as in claim 20, where the single stand alone work station can be disconnected from the network.

Claim 26

Dependent Claim 33 words

A computer system, as in claim 18, where one or more of the computers is a client on a network and one or more of the computers is a server on the network.

Claim 27

Dependent Claim 28 words

A computer system, as in claim 22, where one or more of the servers has one of the memory storage devices with one or more of the documents.

Claim 28

Dependent Claim 18 words

A computer system, as in claim 22, where one or more of the servers has the formatter processor.

Claim 29

Dependent Claim 25 words

A computer system, as in claim 24, where one or more of the servers further has one or more indexes and the search engine processor.

Claim 30

Dependent Claim 18 words

A computer system, as in claim 22, where one or more of the clients has the formatter processor.

Claim 31

Dependent Claim 44 words

A computer system, as in claim 18, where one or more of the selected attributes is an intrinsic attribute, intrinsic attributes being attributes that are established at a time of document creation and that are invariant with a location and replication of the document.

Claim 32

Dependent Claim 23 words

A computer system, as in claim 27, where the intrinsic attributes include any one or more of the following: title and document size.

Claim 33

Dependent Claim 35 words

A computer system, as in claim 27, where one or more of the selected attributes is a non intrinsic attribute, non intrinsic attributes being attributes that vary with respect to one or more document instance.

Claim 34

Dependent Claim 25 words

A computer system, as in claim 29, where one of the non intrinsic attributes include is a document location that includes a file name component.

Claim 35

Dependent Claim 22 words

A computer system, as in claim 30, where the file name component is last component of a URL in a network environment.

Claim 36

Dependent Claim 24 words

A computer system, as in claim 30, where the file name component is an implicit last component of a URL in a network environment.

Claim 37

Dependent Claim 24 words

A computer system, as in claim 18, where one or more of the selected attributes is a function of one or more other attributes.

Claim 38

Dependent Claim 42 words

A computer system, as in claim 18, where a duplicate field is associated with one or more of the hit list entries and the formatter processor uses the duplicate field to mark whether or not the hit list entry is a duplicate.

Claim 39

Dependent Claim 221 words

A computer system of one or more computers comprising: one or more memory storage devices containing a document collection of one or more documents; an index including an inverted file with one or more terms, each term associated with one or more document identifiers, the index further including a document catalog that associates each of the document identifiers with one or more attributes; search engine process means for processing a query with one or more query elements to produce hit list means having one or more hit list entries, each hit list entry associated with one of the documents that is determined by the search engine to be relevant to the query; one or more hit list attributes means for identifying the documents and associated with each of the hit list entries, each of the hit list attributes being one of the attributes; and formatter processor means for identifying duplicate hit entries by selecting one or more of the hit list attributes as selected attributes, identified by a hit list attribute selector, the formatter processor means further selecting two or more hit list entries, called compared entries, that are each associated with one of the documents, each called a compared document, and the formatter process comparing the selected attributes to determine if the compared documents are duplicate instances of one another.

Claim 40

Dependent Claim 221 words

A computer system of one or more computers comprising: one or more memory storage devices containing a document collection of one or more documents; an index including an inverted file with one or more terms, each term associated with one or more document identifiers, the index further including a document catalog that associates each of the document identifiers with one or more attributes; search engine process means for processing a query with one or more query elements to produce hit list means having one or more hit list entries, each hit list entry associated with one of the documents that is determined by the search engine to be relevant to the query; one or more hit list attributes means for identifying the documents and associated with each of the hit list entries, each of the hit list attributes being one of the attributes; and formatter processor means for identifying duplicate hit entries by selecting one or more of the hit list attributes as selected attributes, identified by a hit list attribute selector, the formatter processor means further selecting two or more hit list entries, called compared entries, that are each associated with one of the documents, each called a compared document, and the formatter process comparing the selected attributes to determine if the compared documents are duplicate instances of one another.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2002-06-14	2002-12-14	2003-06-15	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2006-06-14	2006-12-14	2007-06-15	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2010-06-14	2010-12-14	2011-06-15	$7700.00	Window Closed / Lapsed

Identifying duplicate documents from search results without comparing document content

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag