US6183261B1
Expired / Lapsed
US
⏱️ 0.0 years remaining
Collaborative and quality control scoring system and method
Assignee: NCS Pearson Inc
•
Filing: 1998-08-28
•
Granted: 2001-02-06
•
Expires: 2018-08-28
Abstract
Abstract A method and system for managing resolvers of test answers to maintain consistent and efficient scoring of test answers. Collaborative scoring allows two test resolvers to score the same test answer and resolves potential discrepancies in the scores provided by the resolvers. Quality items monitor a test resolver's scoring performance against a known standard. A quality-control test item, having a known score, is transmitted to a test resolver. After receiving a score from the resolver, the known score is compared with the resolver's score and a quality profile is generated for the resolver.
Inventors
- 👤 Jay V. Clark
- 👤 William L. Bramlett, Jr.
- 👤 Judith Moyer
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-10-29 05:11
CPC Classifications
G; G09; G09B; G09B7/00; G09B7/02; G09B7/06; G09B7/066; Y; Y10; Y10S; Y10S706/00; Y10S706/902; Y10S706/927
Claims Analysis
Total Claims: 48
Claim 1
Independent Claim
127 words
A system for verifying test answers between two or more test resolvers, the system comprising: a) first receive means for electronically receiving an answer to a test question; b) transmit means for electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) second receive means for electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) verification means for electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 2
Independent Claim
127 words
A system for verifying test answers between two or more test resolvers, the system comprising: a) first receive means for electronically receiving an answer to a test question; b) transmit means for electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) second receive means for electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) verification means for electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 3
Dependent Claim
30 words
The system of claim 1 further comprising display means for electronically displaying the test question to the first and second test resolvers at the first and second test resolver terminals.
Claim 4
Dependent Claim
76 words
The system of claim 1 wherein the verification means farther comprises collaboration means for providing an electronic communication link between the first and second test resolver terminals and, if the first and second test scores contain the discrepancy based on the predefined criteria, for signaling the first and second test resolvers to interactively determine and enter a joint score using the electronic communication link, the verification means further comprising recording means for recording the joint score.
Claim 5
Dependent Claim
53 words
The system of claim 3 wherein the collaboration means further comprises means for tracking average scores of the first and second test resolvers and for preventing the first and second test resolvers from scoring a same test item if the first and second test resolvers each have average scores within predefined scoring criteria.
Claim 6
Dependent Claim
40 words
The system of claim 3 wherein the verification means further comprises means for preventing the first and second test resolver terminals from receiving another answer to be scored until the first and second test resolvers have entered the joint score.
Claim 7
Dependent Claim
49 words
The system of claim 3 wherein the verification means further comprises means for transmitting the test questions to a third test resolver terminal to be scored by a third test resolver, if the first and second test resolvers do not enter the joint score within a predetermined time period.
Claim 8
Dependent Claim
66 words
The system of claim 1 wherein the verification means further comprises means for transmitting the answer to a third test resolver terminal to be scored by a third test resolver, if the first and second test scores contain the discrepancy based on the predefined criteria, and for recording a third test score, as determined by the third test resolver, received from the third test resolver terminal.
Claim 9
Dependent Claim
35 words
The system of claim 7 wherein the means for transmitting the answer to a third test resolver terminal further comprises means for providing the third test resolver terminal with the first and second test scores.
Claim 10
Dependent Claim
51 words
The system of claim 7 wherein the means for transmitting the answer to a third test resolver terminal her comprises means for preventing the third test resolver from viewing the first and second test scores on the third test resolver terminal before the third test resolver records the third test score.
Claim 11
Dependent Claim
36 words
The system of claim 1 , wherein the verification means further comprises means for electronically linking the first and second test resolver terminals when the first and second test resolvers are located in different geographical locations.
Claim 12
Dependent Claim
45 words
The system of claim 1 wherein the verification means comprises means for selectively recording, as the value, a greater of the first and second test scores, a lower of the first and second test scores, and an average of the first and second test scores.
Claim 13
Independent Claim
116 words
A method for verifying test answers between two or more test resolvers, the method comprising the steps of: a) electronically receiving an answer to a test question; b) electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 14
Independent Claim
116 words
A method for verifying test answers between two or more test resolvers, the method comprising the steps of: a) electronically receiving an answer to a test question; b) electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 15
Dependent Claim
30 words
The method of claim 12 further comprising the step of electronically displaying the test question to the first and second test resolvers at the first and second test resolver terminals.
Claim 16
Dependent Claim
78 words
The method of claim 12 wherein the step of electronically comparing the first and second test scores further comprises the steps of: a) providing an electronic communication link between the first and second test resolver terminals and, if the first and second test scores contain the discrepancy based on the predefined criteria, signaling the first and second test resolvers to interactively determine and enter a joint score using the electronic communication link; and b) recording the joint score.
Claim 17
Dependent Claim
49 words
The method of claim 14 further comprising the step of tracking average scores of the first and second test resolvers and preventing the first and second test resolvers from scoring a same test item if the first and second test resolvers each have average scores within predefined scoring criteria.
Claim 18
Dependent Claim
49 words
The method of claim 14 wherein the step of electronically comparing the first and second test scores further comprises the step of preventing the first and second test resolver terminals from receiving another answer to be scored until the first and second test resolvers have entered the joint score.
Claim 19
Dependent Claim
58 words
The method of claim 14 wherein the step of electronically comparing the first and second test scores further comprises the step of transmitting the test item to a third test resolver terminal to be scored by a third test resolver, if the first and second test resolvers do not enter the joint score within a predetermined time period.
Claim 20
Dependent Claim
76 words
The method of claim 12 wherein the step of electronically comparing the first and second test scores further comprises the steps of: a) transmitting the answer to a third test resolver terminal to be scored by a third test resolver, if the first and second test scores contain the discrepancy based on the predefined criteria; and b) recording a third test score, as determined by the third test resolver, received from the third test resolver terminal.
Claim 21
Dependent Claim
36 words
The method of claim 12 wherein the step of transmitting the answer to a third test resolver terminal further comprises the step of providing the third test resolver terminal with the first and second test scores.
Claim 22
Dependent Claim
52 words
The method of claim 12 wherein the step of transmitting the answer to a third test resolver terminal further comprises the step of preventing the third test resolver from viewing the first and second test scores on the third test resolver terminal before the third test resolver records the third test score.
Claim 23
Dependent Claim
45 words
The method of claim 12 , wherein the step of electronically comparing the first and second test scores further comprises the step of electronically linking the first and second test resolver terminals when the first and second test resolvers are located in different geographical locations.
Claim 24
Dependent Claim
54 words
The method of claim 12 wherein the step of electronically comparing the first and second test scores comprises the step of selectively recording, as the value, a greater of the first and second test scores, a lower of the first and second test scores, and an average of the first and second test scores.
Claim 25
Independent Claim
134 words
A system for verifying test answers between two or more test resolvers, the system comprising: a) a receiving subsystem configured to electronically receive an answer to a test question; b) a transmit subsystem configured to electronically transmit the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) a test score subsystem configured to electronically receive first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) a comparison subsystem configured to electronically compare the first and second test scores and to selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 26
Independent Claim
134 words
A system for verifying test answers between two or more test resolvers, the system comprising: a) a receiving subsystem configured to electronically receive an answer to a test question; b) a transmit subsystem configured to electronically transmit the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) a test score subsystem configured to electronically receive first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) a comparison subsystem configured to electronically compare the first and second test scores and to selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 27
Dependent Claim
33 words
The system of claim 23 wherein the receiving subsystem is further configured to electronically display the test question to the first and second test resolvers at the first and second test resolver terminals.
Claim 28
Dependent Claim
70 words
The system of claim 23 wherein the comparison subsystem is further configured to: a) provide an electronic communication link between the first and second test resolver terminals and, if the first and second test scores contain the discrepancy based on the predefined criteria; b) signal the first and second test resolvers to interactively determine and enter a joint score using the electronic communication link; and c) record the joint score.
Claim 29
Dependent Claim
53 words
The system of claim 25 wherein the comparison subsystem is further configured to track average scores of the first and second test resolvers and to prevent the first and second test resolvers from scoring a same test item if the first and second test resolvers each have average scores within predefined scoring criteria.
Claim 30
Dependent Claim
40 words
The system of claim 25 wherein the comparison subsystem is further configured to prevent the first and second test resolver terminals from receiving another answer to be scored until the first and second test resolvers have entered the joint score.
Claim 31
Dependent Claim
49 words
The system of claim 25 wherein the comparison subsystem is further configured to transmit the test item to a third test resolver terminal to be scored by a third test resolver, if the first and second test resolvers do not enter the joint score within a predetermined time period.
Claim 32
Dependent Claim
67 words
The system of claim 23 wherein the comparison subsystem is further configured to: a) transmit the answer to a third test resolver terminal to be scored by a third test resolver, if the first and second test scores contain the discrepancy based on the predefined criteria; and b) record a third test score, as determined by the third test resolver, received from the third test resolver terminal.
Claim 33
Dependent Claim
36 words
The system of claim 23 wherein the transmitting the answer to a third test resolver terminal by the comparison subsystem further comprises the providing the third test resolver terminal with the first and second test scores.
Claim 34
Dependent Claim
52 words
The system of claim 23 wherein the transmitting the answer to a third test resolver terminal by the comparison subsystem further comprises the preventing the third test resolver from viewing the first and second test scores on the third test resolver terminal before the third test resolver records the third test score.
Claim 35
Dependent Claim
36 words
The system of claim 23 , wherein the comparison subsystem is further configured to electronically link the first and second test resolver terminals when the first and second test resolvers are located in different geographical locations.
Claim 36
Dependent Claim
47 words
The system of claim 23 wherein the comparison subsystem is further configured to selectively record, as the joint score, a greater of the first and second test scores, a lower of the first and second test scores, and an average of the first and second test scores.
Claim 37
Dependent Claim
133 words
A computer-readable program storage medium tangibly embodying a data package and associated verification instructions executable by a computing system for verifying test answers between two or more test resolvers, the method comprising the steps of: a) electronically receiving an answer to a test question; b) electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 38
Dependent Claim
133 words
A computer-readable program storage medium tangibly embodying a data package and associated verification instructions executable by a computing system for verifying test answers between two or more test resolvers, the method comprising the steps of: a) electronically receiving an answer to a test question; b) electronically transmitting the answer to first and second test resolver terminals to be scored by first and second test resolvers; c) electronically receiving first and second test scores for the answer, as determined by the first and second test resolvers, from the first and second test resolver terminals; and d) electronically comparing the first and second test scores and for selectively recording a value related to the first and second test scores if the first and second test scores did not contain a discrepancy based on predefined criteria.
Claim 39
Dependent Claim
33 words
The computer-readable program storage medium of claim 34 further comprising the step of electronically displaying the test question to the first and second test resolvers at the first and second test resolver terminals.
Claim 40
Dependent Claim
81 words
The computer-readable program storage medium of claim 34 wherein the step of electronically comparing the first and second test scores further comprises the steps of: a) providing an electronic communication link between the first and second test resolver terminals and, if the first and second test scores contain the discrepancy based on the predefined criteria, signaling the first and second test resolvers to interactively determine and enter a joint score using the electronic communication link; and b) recording the joint score.
Claim 41
Dependent Claim
52 words
The computer-readable program storage medium of claim 36 further comprising the step of tracking average scores of the first and second test resolvers and preventing the first and second test resolvers from scoring a same test item if the first and second test resolvers each have average scores within predefined scoring criteria.
Claim 42
Dependent Claim
52 words
The computer-readable program storage medium of claim 36 wherein the step of electronically comparing the first and second test scores further comprises the step of preventing the first and second test resolver terminals from receiving another answer to be scored until the first and second test resolvers have entered the joint score.
Claim 43
Dependent Claim
61 words
The computer-readable program storage medium of claim 36 wherein the step of electronically comparing the first and second test scores further comprises the step of transmitting the test item to a third test resolver terminal to be scored by a third test resolver, if the first and second test resolvers do not enter the joint score within a predetermined time period.
Claim 44
Dependent Claim
79 words
The computer-readable program storage medium of claim 34 wherein the step of electronically comparing the first and second test scores further comprises the steps of: a) transmitting the answer to a third test resolver terminal to be scored by a third test resolver, if the first and second test scores contain the discrepancy based on the predefined criteria; and b) recording a third test score, as determined by the third test resolver, received from the third test resolver terminal.
Claim 45
Dependent Claim
39 words
The computer-readable program storage medium of claim 34 wherein the step of transmitting the answer to a third test resolver terminal further comprises the step of providing the third test resolver terminal with the first and second test scores.
Claim 46
Dependent Claim
55 words
The computer-readable program storage medium of claim 34 wherein the step of transmitting the answer to a third test resolver terminal further comprises the step of preventing the third test resolver from viewing the first and second test scores on the third test resolver terminal before the third test resolver records the third test score.
Claim 47
Dependent Claim
48 words
The computer-readable program storage medium of claim 34 , wherein the step of electronically comparing the first and second test scores farther comprises the step of electronically linking the first and second test resolver terminals when the first and second test resolvers are located in different geographical locations.
Claim 48
Dependent Claim
57 words
The computer-readable program storage medium of claim 34 wherein the step of electronically comparing the first and second test scores comprises the step of selectively recording, as the value, a greater of the first and second test scores, a lower of the first and second test scores, and an average of the first and second test scores.
References Cited (Backward Citations)
US3405457A
Cited
US3538626A
Cited
US3762072A
Cited
US4004354A
Cited
US3932948A
Cited
US4205780A
Cited
US4151659A
Cited
US4518267A
Cited
US4518361A
Cited
US4798543A
Cited
US4553261A
Cited
US4478584A
Cited
US4705479A
Cited
EP0171663B1
Cited
US4694352A
Cited
US4789543A
Cited
US4648062A
Cited
US4845739A
Cited
US4908759A
Cited
US4671772A
Cited
US4741047A
Cited
US4764120A
Cited
US4930077A
Cited
US4760246A
Cited
US4867685A
Cited
US4996642A
Cited
US4895518A
Cited
US4878175A
Cited
US5147205A
Cited
US4937439A
Cited
WO1990005970A1
Cited
US5072383A
Cited
US4958284A
Cited
US5003613A
Cited
US5105354A
Cited
US5140650A
Cited
US5002491A
Cited
US4978305A
Cited
US5011413A
Cited
US5035625A
Cited
US5101447A
Cited
US5059127A
Cited
US5344132A
Cited
US5038392A
Cited
US5023435A
Cited
US5151948A
Cited
US5119433A
Cited
US5176520A
Cited
USRE34476E
Cited
US5204813A
Cited
US5195033A
Cited
US5134669A
Cited
US5103490A
Cited
US5100329A
Cited
JPH04147288A
Cited
US5180309A
Cited
US5261823A
Cited
US5496175A
Cited
US5258855A
Cited
US5634101A
Cited
US5259766A
Cited
US5294229A
Cited
US5267865A
Cited
US5418865A
Cited
US5302132A
Cited
US5387104A
Cited
US5672060A
Cited
US5565316A
Cited
US5596698A
Cited
US5321611A
Cited
US5558521A
Cited
US5437554A
Cited
US5458493A
Cited
US5433615A
Cited
US5466159A
Cited
US5690497A
Cited
US5709551A
Cited
US5716213A
Cited
US5718591A
Cited
US5735694A
Cited
US5752836A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2004-02-06 | 2004-08-07 | 2005-02-06 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2008-02-06 | 2008-08-07 | 2009-02-06 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2012-02-06 | 2012-08-07 | 2013-02-06 | $7700.00 | Window Closed / Lapsed |