US5842199A — System, method and article of manufacture for usi…

Claim 1

Independent Claim 104 words

An electronic information system for determining predictive utility of recommendation techniques in ascertaining which items are preferred by a user, comprising: (a) input means for obtaining a set of measurements of user preference of items recommended to a user, the items presented to the user being derived from a plurality of items by utilizing recommendations from an adaptive recommendation technique; and (b) processing means, operatively coupled to the input means, generating a quality metric receiver operating curve (ROC) from the set of measurements which corresponds to a measure of effectiveness of the adaptive recommendation technique in predicting which items are preferred by the user.

Claim 2

Dependent Claim 104 words

An electronic information system for determining predictive utility of recommendation techniques in ascertaining which items are preferred by a user, comprising: (a) input means for obtaining a set of measurements of user preference of items recommended to a user, the items presented to the user being derived from a plurality of items by utilizing recommendations from an adaptive recommendation technique; and (b) processing means, operatively coupled to the input means, generating a quality metric receiver operating curve (ROC) from the set of measurements which corresponds to a measure of effectiveness of the adaptive recommendation technique in predicting which items are preferred by the user.

Claim 3

Dependent Claim 119 words

The information system of claim 1 wherein: (a) the input means comprises means for obtaining a first and a second set of measurements of actual preference of a first and a second set of items recommended to the user, respectively, each set of items presented to the user being derived from the plurality of items by utilizing recommendations from different adaptive recommendation techniques; and (b) the processing means comprises means for generating a first and a second quality metric ROC from the first and the second set of measurements, respectively, and comparing the first and the second quality metric ROC to compare effectiveness of each different adaptive recommendation technique to each other in recommending items preferred by the user.

Claim 4

Dependent Claim 37 words

The information system of claim 2 wherein the processing means comprises means for selecting one of the different adaptive recommendation techniques for subsequent use based on the comparison of the first and the second quality metric ROC.

Claim 5

Dependent Claim 38 words

The information system of claim 3 wherein the different adaptive recommendation techniques utilize different underlying recommendation algorithms such that the comparison of the first and the second quality metric ROC selects which underlying recommendation algorithm is subsequently used.

Claim 6

Dependent Claim 46 words

The information system of claim 3 wherein the different adaptive recommendation techniques utilize a single underlying recommendation algorithm with different operating parameters such that the comparison of the first and the second quality metric ROC selects which operating parameters are subsequently used with the recommendation algorithm.

Claim 7

Dependent Claim 18 words

The information system of claim 5 wherein the operating parameters comprise partitioning schemes for the plurality of items.

Claim 8

Dependent Claim 37 words

The information system of claim 1 wherein the input means comprises means for generating a set of implicit measures from user input as the set of measurements of actual user preference of items recommended to the user.

Claim 9

Dependent Claim 51 words

The information system of claim 7 wherein the input means comprises means for generating the set of implicit measures based on criteria selected from a group consisting of: user time spent viewing a particular item, user time spent listening to a particular item, and user time spent sampling a particular item.

Claim 10

Dependent Claim 78 words

The information system of claim 7 wherein the input means comprises means for generating the set of implicit measures based on subsequent actions taken, the subsequent actions taken being selected from a group consisting of: the user viewing details of a particular item after viewing only a title, saving a particular item, forwarding a particular item, saving a return address of a particular item, deleting a particular item, replying to a particular item, and printing a particular item.

Claim 11

Dependent Claim 150 words

The information system of claim 1 wherein the input means and processing means together form a quality metric device, the information system further comprising a client device operatively coupled to the quality metric device, the client device comprising: (a) user interface means for recommending the items to the user; (b) user input means for receiving data input by the user from which a set of measurements of actual user preference of items recommended to a user is obtained; (c) client processing means, operatively coupled to the user interface means and user input means, for receiving items to recommend to the user from an output mechanism of the quality metric device, obtaining the set of measurements of actual user preference of items based on the data input by the user, and providing the set of measurements of actual user preference of items to the input means of the quality metric device.

Claim 12

Dependent Claim 21 words

The information system of claim 10 wherein the quality metric device and the client device are located within a single apparatus.

Claim 13

Dependent Claim 47 words

The information system of claim 10 wherein the quality metric device and the client device each comprise means for communicating with each other through an interactive network such that the quality metric device and the client device can be remotely located from one another in separate apparatuses.

Claim 14

Independent Claim 114 words

A method of operating an electronic information system in order to determine predictive utility of recommendation techniques in ascertaining which items are preferred by a user, comprising: (a) deriving particular items from a plurality of items by utilizing recommendations from an adaptive recommendation technique; (b) presenting the particular items to a user on a user interface device; (c) obtaining a set of measurements of actual value of the particular items presented to a user through an input device; and (d) generating a quality metric receiver operating curve (ROC) from the set of measurements which corresponds to a measure of effectiveness of the adaptive recommendation technique in predicting which items are preferred by the user.

Claim 15

Independent Claim 114 words

A method of operating an electronic information system in order to determine predictive utility of recommendation techniques in ascertaining which items are preferred by a user, comprising: (a) deriving particular items from a plurality of items by utilizing recommendations from an adaptive recommendation technique; (b) presenting the particular items to a user on a user interface device; (c) obtaining a set of measurements of actual value of the particular items presented to a user through an input device; and (d) generating a quality metric receiver operating curve (ROC) from the set of measurements which corresponds to a measure of effectiveness of the adaptive recommendation technique in predicting which items are preferred by the user.

Claim 16

Dependent Claim 146 words

The method of claim 13: (a) wherein the deriving step comprises deriving a first and a second set of particular items from a plurality of items by utilizing recommendations from different adaptive recommendation techniques; (b) wherein the presenting step comprises presenting the first and the second set of particular items to the user; (c) wherein the obtaining step comprises obtaining a first and a second set of measurements of actual user preference of the first and the second set of items presented to the user, respectively; (d) wherein the generating step comprises generating a first and a second quality metric ROC from the first and the second set of measurements, respectively; and (e) further comprises a step of comparing the first and the second quality metric ROC to compare effectiveness of each different adaptive recommendation technique to each other in predicting items preferred by the user.

Claim 17

Dependent Claim 26 words

The method of claim 14 further comprising a step of selecting one of the different adaptive recommendation techniques for subsequent use based on the comparing step.

Claim 18

Dependent Claim 30 words

The method of claim 15 wherein the different adaptive recommendation techniques utilize different underlying recommendation algorithms such that the selecting step comprises selecting which underlying recommendation algorithm is subsequently used.

Claim 19

Dependent Claim 38 words

The method of claim 15 wherein the different adaptive recommendation techniques utilize a single underlying recommendation algorithm with different operating parameters such that the selecting step comprises selecting which operating parameters are subsequently used with the recommendation algorithm.

Claim 20

Dependent Claim 17 words

The method of claim 17 wherein the operating parameters comprise partitioning schemes for the plurality of items.

Claim 21

Dependent Claim 38 words

The method of claim 18 wherein the obtaining step comprises generating a set of implicit measures from user input through the input device as the set of measurements of actual user preference of items presented to the user.

Claim 22

Dependent Claim 46 words

The method of claim 19 wherein the obtaining step comprises deriving the implicit measure based on criteria selected from a group consisting of: user time spent viewing a particular item, user time spent listening to a particular item, and user time spent sampling a particular item.

Claim 23

Dependent Claim 73 words

The method of claim 19 wherein the obtaining step comprises deriving the implicit measure based on subsequent actions taken, the subsequent actions taken being selected from a group consisting of: a user viewing details of a particular item after viewing only a title, saving a particular item, forwarding a particular item, saving a return address of a particular item, deleting a particular item, replying to a particular item, and printing a particular item.

Claim 24

Dependent Claim 141 words

A program storage device readable by a computer system tangibly embodying a program of instructions executable by the computer system to perform a method of operating an electronic information system in order to determine predictive utility of recommendation techniques in ascertaining which items are preferred by a user, the method comprising steps of: (a) deriving particular items from a plurality of items by utilizing recommendations from an adaptive recommendation technique; (b) presenting the particular items to a user on a user interface device; (c) obtaining a set of measurements of actual user preference of the particular items presented to a user through an input device; and (d) generating a quality metric receiver operating curve (ROC) from the set of measurements which corresponds to a measure of effectiveness of the adaptive recommendation technique in predicting which items are preferred by the user.

Claim 25

Dependent Claim 141 words

A program storage device readable by a computer system tangibly embodying a program of instructions executable by the computer system to perform a method of operating an electronic information system in order to determine predictive utility of recommendation techniques in ascertaining which items are preferred by a user, the method comprising steps of: (a) deriving particular items from a plurality of items by utilizing recommendations from an adaptive recommendation technique; (b) presenting the particular items to a user on a user interface device; (c) obtaining a set of measurements of actual user preference of the particular items presented to a user through an input device; and (d) generating a quality metric receiver operating curve (ROC) from the set of measurements which corresponds to a measure of effectiveness of the adaptive recommendation technique in predicting which items are preferred by the user.

Claim 26

Dependent Claim 150 words

The program storage device of claim 22 wherein: (a) the method deriving step comprises deriving a first and a second set of particular items from a plurality of items by utilizing recommendations from different adaptive recommendation techniques; (b) the method presenting step comprises presenting the first and the second set of particular items to the user; (c) the method obtaining step comprises obtaining a first and a second set of measurements of actual user preference of the first and the second set of items presented to the user, respectively; (d) the method generating step comprises generating a first and a second quality metric ROC from the first and the second set of measurements, respectively; and (e) the method further comprises a step of comparing the first and the second quality metric ROC to compare effectiveness of each different recommendation technique to each other in predicting items preferred by the user.

Claim 27

Dependent Claim 30 words

The program storage device of claim 23 wherein the method further comprises a step of selecting one of the different recommendation techniques for subsequent use based on the comparing step.

Annuity Stage	Window Opens	Payment Deadline	Surcharge Period Until	Estimated Fee	Status
3.5 Years (Year 4)	2001-11-23	2002-05-25	2002-11-24	$2000.00	Window Closed / Lapsed
7.5 Years (Year 8)	2005-11-23	2006-05-25	2006-11-24	$3760.00	Window Closed / Lapsed
11.5 Years (Year 12)	2009-11-23	2010-05-25	2010-11-24	$7700.00	Window Closed / Lapsed

System, method and article of manufacture for using receiver operating curves to evaluate predictive utility

Abstract

Inventors

Bibliographic Details

Claims Analysis

References Cited (Backward Citations)

USPTO Annuity & Maintenance Windows

Assigned Categories & Tags

Assign Category Tag