US5987302A
Expired / Lapsed
US
⏱️ 0.0 years remaining
On-line essay evaluation system
Assignee: Educational Testing Service
•
Filing: 1998-03-20
•
Granted: 1999-11-16
•
Expires: 2018-03-20
Abstract
Abstract Systems and methods for on-line essay evaluation offer students the opportunity to prepare practice essays, submit the essays to trained, expert readers, and retrieve an evaluation at the student's convenience. The system provides the capability for a user or test taker to submit essays at any time during the year, independent of the timing of an actual testing event, and to receive prompt, consistent evaluations of the submitted essays. Further, the system provides the capability to prioritize essays and schedule readers so that essays can be evaluated on a rush basis. The essays are evaluated in a manner that provides useful instructional feedback to students about their skills relative to any assessment or test that the student wishes to take.
Inventors
- 👤 Gary Driscoll
- 👤 Lou A. Hatfield
- 👤 Arlene A. Johnson
- 👤 Helen D. Kahn
- 👤 Theodore E. Kessler
- 👤 David L. Kuntz
- 👤 Cheryl A. Pocino
- 👤 Michele Rosenthal
- 👤 Patricia G. Williams
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-29 19:08
CPC Classifications
G; G09; G09B; G09B5/00; G09B5/06; G09B5/065; G09B5/08; G09B5/14; G09B7/00; G09B7/02
Claims Analysis
Total Claims: 19
Claim 1
Independent Claim
152 words
A method for on-line evaluation of a constructed response to an examination question, comprising the steps of: accessing a web site via the Internet, ordering an evaluation account, and selecting an examination question; an examinee constructing an essay response to said examination question; electronically submitting said essay response to said web site for evaluation; positioning the submitted essay response in a queue of essay responses that need to be evaluated by a grader, the essay being placed in the queue based upon at least one of the time of submission of said essay response and the date an evaluation of the essay response is due to the examinee; the grader evaluating the essay response in accordance with calibrated grading guidelines, selecting an overall evaluation, at least one pre-defined feedback comment, and a score; and releasing for retrieval by the examinee the overall evaluation and the pre-defined feedback comments regarding said essay response.
Claim 2
Independent Claim
152 words
A method for on-line evaluation of a constructed response to an examination question, comprising the steps of: accessing a web site via the Internet, ordering an evaluation account, and selecting an examination question; an examinee constructing an essay response to said examination question; electronically submitting said essay response to said web site for evaluation; positioning the submitted essay response in a queue of essay responses that need to be evaluated by a grader, the essay being placed in the queue based upon at least one of the time of submission of said essay response and the date an evaluation of the essay response is due to the examinee; the grader evaluating the essay response in accordance with calibrated grading guidelines, selecting an overall evaluation, at least one pre-defined feedback comment, and a score; and releasing for retrieval by the examinee the overall evaluation and the pre-defined feedback comments regarding said essay response.
Claim 3
Dependent Claim
147 words
An on-line essay evaluation system for submitting essay responses for evaluation by graders and retrieving an evaluation of the essay response from the grader for submission to the examinee, comprising: a database which stores a plurality of questions which elicit essay responses from an examinee; an essay response submission system which selects a question from said database and which submits an essay response to the selected question; an essay response administrative system which stores essay responses from a plurality of examinees in a queue and selectively distributes the queued essay responses to graders for evaluation; an automated grading system which enables a grader to evaluate an essay response, to select an overall evaluation, at least one pre-defined additional comment, and a score; and an evaluation delivery system which releases to the examinee for the examinee's retrieval said overall evaluation and any pre-defined additional comments from the grader.
Claim 4
Dependent Claim
147 words
An on-line essay evaluation system for submitting essay responses for evaluation by graders and retrieving an evaluation of the essay response from the grader for submission to the examinee, comprising: a database which stores a plurality of questions which elicit essay responses from an examinee; an essay response submission system which selects a question from said database and which submits an essay response to the selected question; an essay response administrative system which stores essay responses from a plurality of examinees in a queue and selectively distributes the queued essay responses to graders for evaluation; an automated grading system which enables a grader to evaluate an essay response, to select an overall evaluation, at least one pre-defined additional comment, and a score; and an evaluation delivery system which releases to the examinee for the examinee's retrieval said overall evaluation and any pre-defined additional comments from the grader.
Claim 5
Dependent Claim
40 words
The system of claim 2 wherein said essay response administrative system schedules graders to evaluate queued essay responses so that essay responses are evaluated and feedback is stored and released for retrieval by the examinee within a predetermined time period.
Claim 6
Dependent Claim
42 words
The system of claim 2 wherein said essay response administrative system prioritizes the presentation and distribution of essay responses to graders so that essay responses are evaluated and feedback is stored and released for retrieval by examinees within a predetermined time period.
Claim 7
Dependent Claim
29 words
The system of claim 2 wherein said essay response administrative system automatically calculates any compensation for the graders based upon the number of constructed responses the grader has evaluated.
Claim 8
Dependent Claim
38 words
The system of claim 2 wherein said automated grading system enables a grader leader to view the overall evaluation and pre-defined additional comments of the grader and assign a new overall evaluation and pre-defined additional comments if desired.
Claim 9
Dependent Claim
32 words
The system of claim 2 wherein said evaluation delivery system releases to the examinee for the examinee's retrieval at least one exemplary essay response, said overall evaluation, and said pre-defined additional comments.
Claim 10
Dependent Claim
148 words
The system of claim 2 wherein said essay response administrative system includes software which: identifies the number of essay responses which need to be evaluated in an upcoming time period; identifies the number of essay responses which are scheduled to be evaluated in the upcoming time period; if the number of essay responses that need to be evaluated in the upcoming time period is less than the number of essay responses that are scheduled to be evaluated in the upcoming time period, identifies the number of essay responses which need to be evaluated in the upcoming time period; and if the number of essay responses that need to be evaluated in the upcoming time period is greater than the number of essays that are scheduled to be evaluated in the upcoming time period, electronically notifies backup graders of the need to provide assistance in the upcoming time period.
Claim 11
Dependent Claim
106 words
The system of claim 2 wherein said essay response administrative system includes software which: identifies the date and time an essay response was submitted; identifies whether the essay response is to be graded on a regular schedule or a rush schedule; calculates a date for which an evaluation of the essay response is due based upon the date the essay response was submitted and whether the essay response is to be evaluated on a regular schedule or a rush schedule; and places the essay response in a queue of essay responses that is ordered by the date on which an evaluation of each response is due.
Claim 12
Dependent Claim
44 words
The system of claim 2 wherein said essay response administrative system includes software which: identifies a grader; identifies the total number of essay responses read by the grader; and calculates the compensation for the grader as function of the number of essay responses read.
Claim 13
Dependent Claim
81 words
The system of claim 2 wherein said essay response administrative system includes software which: identifies that a grader has submitted an overall evaluation and score for an essay response; forwards the essay response and overall evaluation to a grader leader for review; and if the overall evaluation and score are not consistent with scoring and evaluation standards, allows grader leader to assign a new score and evaluation, so long as the supervisory grader performs the review in a predetermined time period.
Claim 14
Dependent Claim
42 words
The system of claim 2 wherein said automated grading system presents an essay response to a grader so that the grader may evaluate the essay response; and presents evaluation categories to the grader so that the grader may select an overall evaluation.
Claim 15
Dependent Claim
44 words
The system of claim 2 wherein said evaluation delivery system groups the examination question, scoring guidelines, sample essay responses, the overall evaluation, and pre-defined additional comments into a feedback package; stores said feedback package; and releases said feedback package for retrieval by the examinee.
Claim 16
Dependent Claim
153 words
An Internet based system for submitting and evaluating essay responses to examination questions, comprising: a server computer which forwards examination questions to examinees and accepts essay responses to said examination questions for evaluation, said server being connected to the Internet; a first computer connected to said server computer so as to permit examinees to set up an account for purchasing access to an examination question and evaluation of an essay response submitted in response to said examination question; and a second computer connected to said server computer so as to permit graders to read an essay response that was submitted by an examinee, evaluate the essay response by selecting an overall comment and at least one pre-defined feedback comment, store said overall comment and said at least one re-defined feedback comments, and release said overall comment and said at least one pre-defined feedback comment for retrieval by the examinee via said server computer.
Claim 17
Dependent Claim
153 words
An Internet based system for submitting and evaluating essay responses to examination questions, comprising: a server computer which forwards examination questions to examinees and accepts essay responses to said examination questions for evaluation, said server being connected to the Internet; a first computer connected to said server computer so as to permit examinees to set up an account for purchasing access to an examination question and evaluation of an essay response submitted in response to said examination question; and a second computer connected to said server computer so as to permit graders to read an essay response that was submitted by an examinee, evaluate the essay response by selecting an overall comment and at least one pre-defined feedback comment, store said overall comment and said at least one re-defined feedback comments, and release said overall comment and said at least one pre-defined feedback comment for retrieval by the examinee via said server computer.
Claim 18
Independent Claim
176 words
A method for on-line evaluation of constructed responses to an examination question over a system having a server computer connected to the Internet, a first computer electronically connected to said server computer, and second computer connected to said server computer, comprising the following steps: an examinee accessing the server computer from said first computer to set up an account for purchasing access to an examination question and evaluation of an essay response which is submitted in response to said examination question; an examinee accessing the server computer from said first computer to submit an essay response in response to said examination question; accepting said essay responses at the server computer; and a grader accessing the server computer from the second computer to read the essay response submitted by the examinee, evaluate the essay response by providing an overall comment and at least one pre-defined feedback comment, storing said overall comment and said at least one pre-defined feedback comment, and releasing said overall comment and said at least one pre-defined feedback comment for retrieval by the examinee.
Claim 19
Independent Claim
176 words
A method for on-line evaluation of constructed responses to an examination question over a system having a server computer connected to the Internet, a first computer electronically connected to said server computer, and second computer connected to said server computer, comprising the following steps: an examinee accessing the server computer from said first computer to set up an account for purchasing access to an examination question and evaluation of an essay response which is submitted in response to said examination question; an examinee accessing the server computer from said first computer to submit an essay response in response to said examination question; accepting said essay responses at the server computer; and a grader accessing the server computer from the second computer to read the essay response submitted by the examinee, evaluate the essay response by providing an overall comment and at least one pre-defined feedback comment, storing said overall comment and said at least one pre-defined feedback comment, and releasing said overall comment and said at least one pre-defined feedback comment for retrieval by the examinee.
References Cited (Backward Citations)
US4768087A
Cited
US4671772A
Cited
US4867685A
Cited
US4895518A
Cited
US5247670A
Cited
US4953209A
Cited
US4958284A
Cited
US5002491A
Cited
US4978305A
Cited
US5211564A
Cited
US5011413A
Cited
US5033969A
Cited
US5059127A
Cited
US5318450A
Cited
US5176520A
Cited
US5195033A
Cited
US5204813A
Cited
US5180309A
Cited
US5170362A
Cited
US5211563A
Cited
US5524193A
Cited
US5309564A
Cited
US5442759A
Cited
US5672060A
Cited
US5287519A
Cited
US5565316A
Cited
US5432932A
Cited
US5509127A
Cited
US5437554A
Cited
US5466159A
Cited
US5433615A
Cited
US5558521A
Cited
US5321611A
Cited
US5513994A
Cited
WO1995015654A1
Cited
US5829983A
Cited
US5772446A
Cited
US5788504A
Cited
US5782642A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-11-15 | 2003-05-17 | 2003-11-16 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-11-15 | 2007-05-17 | 2007-11-16 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-11-15 | 2011-05-17 | 2011-11-16 | $7700.00 | Window Closed / Lapsed |