Patent US5870709A successfully refreshed from Google Patents.
US5870709A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Method and apparatus for combining information from speech signals for adaptive interaction in teaching and testing
Assignee: Ordinate Corp
•
Filing: 1996-11-25
•
Granted: 1999-02-09
•
Expires: 2016-11-25
Abstract
Abstract A computer system with a speech recognition component provides a method and apparatus for instructing and evaluating the proficiency of human users in skills that can be exhibited through speaking. The computer system tracks linguistic, indexical and paralinguistic characteristics of the spoken input of users, and implements games, data access, instructional systems, and tests. The computer system combines characteristics of the spoken input automatically to select appropriate material and present it in a manner suitable for the user. In one embodiment, the computer system measures the response latency and speaking rate of the user and presents its next spoken display at an appropriate speaking rate. In other embodiments, the computer system identifies the gender and native language of the user, and combines that information with the relative accuracy of the linguistic content of the user's utterance to select and display material that may be easier or more challenging for speakers with these characteristics.
Inventors
- 👤 Jared C. Bernstein
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 14:26
CPC Classifications
G; G09; G09B; G09B7/00; G09B7/02; G09B7/04; G09B19/00; G09B19/04; G10; G10L; G10L15/00; G10L15/22
Claims Analysis
Total Claims: 65
Claim 1
Independent Claim
72 words
A computer-assisted method, comprising selecting a next prompt for presentation to a user based at least in part on one or more extra-linguistic measures estimated from one or more spoken responses received from the user in response to one or more prior prompts, the extra-linguistic measures being chosen from a group including the latency of the spoken response, the amplitude of the spoken response and fundamental frequency values of the spoken response.
Claim 2
Dependent Claim
72 words
A computer-assisted method, comprising selecting a next prompt for presentation to a user based at least in part on one or more extra-linguistic measures estimated from one or more spoken responses received from the user in response to one or more prior prompts, the extra-linguistic measures being chosen from a group including the latency of the spoken response, the amplitude of the spoken response and fundamental frequency values of the spoken response.
Claim 3
Dependent Claim
68 words
The computer-assisted method of claim 1 wherein the next prompt is selected based at least in part on the one or more extra-linguistic measures and one or more prosodic measures estimated from the one or more spoken responses, the prosodic measures being chosen from a group including the rate of speech of the user during the period of the spoken response and the fluency of the spoken response.
Claim 4
Dependent Claim
66 words
The computer-assisted method of claim 2 wherein the next prompt is selected based at least in part on the extra-linguistic and prosodic measures and one or more indexical values, the indexical values being chosen from a group including speaker identity, speaker native language, speaker age, and speaker gender, the indexical values being estimated from the one or more spoken responses or directly provided by the user.
Claim 5
Dependent Claim
56 words
The computer-assisted method of claim 3 wherein the next prompt is selected based at least in part on the extra-linguistic measures, prosodic measures and indexical values and one or more production quality measures estimated from the one or more spoken responses, the production quality measures from a group including the pronunciation quality of the spoken response.
Claim 6
Dependent Claim
45 words
The computer-assisted method of claim 4 wherein the next prompt is selected based at least in part on the indexical values, the extra-linguistic, prosodic, and production quality measures and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 7
Dependent Claim
43 words
The computer-assisted method of claim 3 wherein the next prompt is selected based at least in part on the extra-linguistic measures, prosodic measures and indexical values and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 8
Dependent Claim
55 words
The computer-assisted method of claim 2 wherein the next prompt is selected based at least in part on the extra-linguistic and prosodic measures and one or more production quality measures estimated from the one or more spoken responses, the production quality measures being chosen from a group including the pronunciation quality of the spoken response.
Claim 9
Dependent Claim
40 words
The computer-assisted method of claim 2 wherein the next prompt is selected based at least in part on the extra-linguistic and prosodic measures and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 10
Dependent Claim
67 words
The computer-assisted method of claim 1 wherein the next prompt is selected based at least in part on the one or more extra-linguistic measures and one or more indexical values, the indexical values being chosen from a group including speaker identity, speaker native language, speaker age, and speaker gender, the indexical values being estimated from the one or more spoken responses or directly provided by the user.
Claim 11
Dependent Claim
56 words
The computer-assisted method of claim 1 wherein the next prompt is selected based at least in part on the one or more extra-linguistic measures and one or more production quality measures estimated from the one or more spoken responses, the production quality measures being chosen from a group including the pronunciation quality of the spoken response.
Claim 12
Dependent Claim
41 words
The computer-assisted method of claim 1 wherein the next prompt is selected based at least in part on the one or more extra-linguistic measures and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 13
Dependent Claim
45 words
The computer-assisted method of claim 1 wherein the next prompt comprises at least one of: a request for information; a request to read a linguistic unit; a request to repeat a linguistic unit; or a request to complete, fill in or identify a verbal aggregate.
Claim 14
Dependent Claim
32 words
The computer-assisted method of claim 1 wherein the one or more spoken responses from the user are received at an interactive computer system via telephone or other telecommunication or data information network.
Claim 15
Dependent Claim
29 words
The computer-assisted method of claim 1 wherein the next prompt comprises at least one of: a graphical prompt, an audio prompt, or a combination of verbal and graphical elements.
Claim 16
Dependent Claim
50 words
A computer-assisted method, comprising selecting a next prompt for presentation to a user based at least in part on a value of the user's native language, age or gender as estimated at least in part from one or more spoken responses of the user to one or more prior prompts.
Claim 17
Dependent Claim
50 words
A computer-assisted method, comprising selecting a next prompt for presentation to a user based at least in part on a value of the user's native language, age or gender as estimated at least in part from one or more spoken responses of the user to one or more prior prompts.
Claim 18
Dependent Claim
45 words
The computer-assisted method of claim 15 wherein the next prompt comprises at least one of: a request for information; a request to read a linguistic unit; a request to repeat a linguistic unit; or a request to complete, fill in or identify a verbal aggregate.
Claim 19
Dependent Claim
27 words
The computer-assisted method of claim 15 wherein the user's spoken responses are received at an interactive computer system via telephone or other telecommunication or data information network.
Claim 20
Dependent Claim
29 words
The computer-assisted method of claim 15 wherein the next prompt comprises at least one of: a graphical prompt, an audio prompt, or a combination of verbal and graphical elements.
Claim 21
Dependent Claim
59 words
The computer-assisted method of claim 15 wherein the next prompt is further selected based at least in part on one or more measures of extra-linguistic, prosodic or production quality measures or linguistic units estimated from the one or more spoken responses, and the user's identity estimated from the one or more spoken responses or provided directly by the user.
Claim 22
Dependent Claim
111 words
A computer-assisted method, comprising selecting a next prompt for presentation to a user based at least in part on prosodic measures estimated from one or more spoken responses received from the user in response to one or more prior prompts and indexical values, the prosodic measures being chosen from a group including the rate of speech of the user during the period of the spoken response and the fluency of the spoken response, and the indexical values being chosen from a group including user identity, user native language, user age, and user gender, the indexical values being estimated from the one or more spoken responses or directly provided by the user.
Claim 23
Dependent Claim
111 words
A computer-assisted method, comprising selecting a next prompt for presentation to a user based at least in part on prosodic measures estimated from one or more spoken responses received from the user in response to one or more prior prompts and indexical values, the prosodic measures being chosen from a group including the rate of speech of the user during the period of the spoken response and the fluency of the spoken response, and the indexical values being chosen from a group including user identity, user native language, user age, and user gender, the indexical values being estimated from the one or more spoken responses or directly provided by the user.
Claim 24
Dependent Claim
45 words
The computer-assisted method of claim 20 wherein the next prompt comprises at least one of: a request for information; a request to read a linguistic unit; a request to repeat a linguistic unit; or a request to complete, fill in or identify a verbal aggregate.
Claim 25
Dependent Claim
29 words
The computer-assisted method of claim 20 wherein the spoken responses from the user are received at an interactive computer system via telephone or other telecommunication or data information network.
Claim 26
Dependent Claim
29 words
The computer-assisted method of claim 20 wherein the next prompt comprises at least one of: a graphical prompt, an audio prompt, or a combination of verbal and graphical elements.
Claim 27
Dependent Claim
74 words
A computer-assisted method, comprising estimating skills, traits or states of a user from extra-linguistic measures derived from one or more spoken responses received from the user in response to one or more prompts, the extra-linguistic measures being chosen from a group including the latency of at least one of the spoken responses, the amplitude of at least one of the spoken responses and fundamental frequency values of at least one of the spoken responses.
Claim 28
Dependent Claim
74 words
A computer-assisted method, comprising estimating skills, traits or states of a user from extra-linguistic measures derived from one or more spoken responses received from the user in response to one or more prompts, the extra-linguistic measures being chosen from a group including the latency of at least one of the spoken responses, the amplitude of at least one of the spoken responses and fundamental frequency values of at least one of the spoken responses.
Claim 29
Dependent Claim
30 words
The computer-assisted method of claim 24 wherein the skills, traits or states are chosen from a group including language proficiency, subject matter skill, cognitive skill, verbal skill and vocal skill.
Claim 30
Dependent Claim
48 words
The computer-assisted method of claim 25 wherein at least one of the prompts comprises at least one of: a request for information; a request to read a linguistic unit; a request to repeat a linguistic unit; or a request to complete, fill in or identify a verbal aggregate.
Claim 31
Dependent Claim
29 words
The computer-assisted method of claim 25 wherein the spoken responses from the user are received at an interactive computer system via telephone or other telecommunication or data information network.
Claim 32
Dependent Claim
36 words
The computer-assisted method of claim 25 wherein at least one of the prompts comprises at least one of: one or more graphical prompts, one or more audio prompts, or a combination of verbal and graphical elements.
Claim 33
Dependent Claim
65 words
The computer-assisted method of claim 24 wherein the skills, traits or states are estimated from the one or more extra-linguistic measures and one or more prosodic measures derived from the one or more spoken responses, the prosodic measures being chosen from a group including the rate of speech of the user during the period of the spoken response and the fluency of the spoken response.
Claim 34
Dependent Claim
63 words
The computer-assisted method of claim 29 wherein the skills, traits or states are estimated from the extra-linguistic and prosodic measures and one or more indexical values, the indexical values being chosen from a group including speaker identity, speaker native language, speaker age, and speaker gender, the indexical values being estimated from the one or more spoken responses or directly provided by the user.
Claim 35
Dependent Claim
53 words
The computer-assisted method of claim 30 wherein the skills, traits or states are estimated from the extra-linguistic measures, prosodic measures and indexical values and one or more production quality measures derived from the one or more spoken responses, the production quality measures from a group including the pronunciation quality of the spoken response.
Claim 36
Dependent Claim
40 words
The computer-assisted method of claim 30 wherein the skills, traits or states are estimated from the extra-linguistic measures, prosodic measures and indexical values and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 37
Dependent Claim
42 words
The computer-assisted method of claim 32 wherein the skills, traits or states are estimated from the indexical values, the extra-linguistic, prosodic, and production quality measures and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 38
Dependent Claim
52 words
The computer-assisted method of claim 29 wherein the skills, traits or states are estimated from the extra-linguistic and prosodic measures and one or more production quality measures derived from the one or more spoken responses, the production quality measures being chosen from a group including the pronunciation quality of the spoken response.
Claim 39
Dependent Claim
37 words
The computer-assisted method of claim 29 wherein the skills, traits or states are estimated from the extra-linguistic and prosodic measures and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 40
Dependent Claim
64 words
The computer-assisted method of claim 24 wherein the skills, traits or states are estimated from the one or more extra-linguistic measures and one or more indexical values, the indexical values being chosen from a group including speaker identity, speaker native language, speaker age, and speaker gender, the indexical values being estimated from the one or more spoken responses or directly provided by the user.
Claim 41
Dependent Claim
53 words
The computer-assisted method of claim 24 wherein the skills, traits or states are estimated from the one or more extra-linguistic measures and one or more production quality measures derived from the one or more spoken responses, the production quality measures being chosen from a group including the pronunciation quality of the spoken response.
Claim 42
Dependent Claim
38 words
The computer-assisted method of claim 24 wherein the skills, traits or states are estimated from the one or more extra-linguistic measures and the identity of one or more linguistic units which comprise the one or more spoken responses.
Claim 43
Dependent Claim
148 words
A computer-assisted method, comprising using one or more semaphore values measured from one or more spoken responses of a user received in response to one or more prompts to select extra-linguistic characteristics, prosodic characteristics, indexical values or production quality characteristics with which to present linguistic units in a next prompt, the extra-linguistic characteristics being chosen from a group including the latency of the spoken response, the amplitude of the spoken response and fundamental frequency values of the spoken response, the prosodic characteristics being chosen from a group including the rate of speech of the user during the period of the spoken response and the fluency of the spoken response, the indexical values being chosen from a group including speaker identity, speaker native language, speaker age, and speaker gender, and the production quality characteristics being chosen from a group including the pronunciation style or quality of the spoken response.
Claim 44
Dependent Claim
148 words
A computer-assisted method, comprising using one or more semaphore values measured from one or more spoken responses of a user received in response to one or more prompts to select extra-linguistic characteristics, prosodic characteristics, indexical values or production quality characteristics with which to present linguistic units in a next prompt, the extra-linguistic characteristics being chosen from a group including the latency of the spoken response, the amplitude of the spoken response and fundamental frequency values of the spoken response, the prosodic characteristics being chosen from a group including the rate of speech of the user during the period of the spoken response and the fluency of the spoken response, the indexical values being chosen from a group including speaker identity, speaker native language, speaker age, and speaker gender, and the production quality characteristics being chosen from a group including the pronunciation style or quality of the spoken response.
Claim 45
Dependent Claim
27 words
The computer-assisted method of claim 39 wherein the semaphore values comprise at least one of: extra-linguistic measures, prosodic measures, indexical values, production quality measures or linguistic units.
Claim 46
Dependent Claim
29 words
The computer-assisted method of claim 39 wherein the next prompt comprises at least one of: a graphical prompt, an audio prompt, or a combination of verbal and graphical elements.
Claim 47
Dependent Claim
45 words
The computer-assisted method of claim 39 wherein the next prompt comprises at least one of: a request to repeat a linguistic unit; a request to complete, fill in or identify a verbal aggregate; a request to read a linguistic unit; or a request for information.
Claim 48
Dependent Claim
81 words
An interactive computer-based system wherein spoken responses are elicited from a user in response to prompts presented by the system, the system comprising: a) means for extracting linguistic, indexical, or paralinguistic values in the user's spoken response; and b) means for automatically selecting a next prompt to be presented to the user according to combined values of (i) linguistic units, including words, phrases or sentences contained in the response and (ii) a latency of the spoken response relative to the prompt.
Claim 49
Dependent Claim
81 words
An interactive computer-based system wherein spoken responses are elicited from a user in response to prompts presented by the system, the system comprising: a) means for extracting linguistic, indexical, or paralinguistic values in the user's spoken response; and b) means for automatically selecting a next prompt to be presented to the user according to combined values of (i) linguistic units, including words, phrases or sentences contained in the response and (ii) a latency of the spoken response relative to the prompt.
Claim 50
Dependent Claim
47 words
The system of claim 43 wherein the means for extracting indexical or paralinguistic values comprises means for extracting semaphore values including speaker identity, fundamental frequency values, speech signal amplitudes, pronunciation quality, fluency, speech rate, speaker native language, speaker age or speaker gender from the user's spoken responses.
Claim 51
Dependent Claim
58 words
The system of claim 44 wherein the means for automatically selecting the next prompt comprises means for combining (a) one or more of the extracted semaphore values from the user's spoken responses with (b) a measure of the latency of the user's response and with (c) the linguistic units of a previous response to select the next prompt.
Claim 52
Dependent Claim
36 words
The system of claim 44 wherein the means for automatically selecting the next prompt comprises means for estimating a user state by combining two or more of the extracted semaphore values to select the next prompt.
Claim 53
Dependent Claim
39 words
The system of claim 44 wherein the means for automatically selecting the next prompt further comprises means for estimating a user skill or trait by combining two or more of the extracted semaphore values to select the next prompt.
Claim 54
Dependent Claim
26 words
The system of claim 47 wherein the user skill or trait comprises at least one of: language proficiency, subject matter knowledge, user age or user gender.
Claim 55
Dependent Claim
37 words
The system of claim 48 further comprising means for selecting the linguistic, paralinguistic or indexical characteristics of the prompt, at least in part, by the linguistic, paralinguistic or indexical content of a spoken response from the user.
Claim 56
Dependent Claim
43 words
The system of claim 44 wherein the linguistic, paralinguistic or indexical characteristics of the prompt presented include linguistic units, latency relative to the user's response, speech rate, fundamental frequency values, speech signal amplitudes, pronunciation quality, fluency, speaker identity, speaker age or speaker gender.
Claim 57
Dependent Claim
13 words
The system of claim 44 wherein the prompt comprises a request for information.
Claim 58
Dependent Claim
16 words
The system of claim 44 wherein the prompt comprises a request to read a linguistic unit.
Claim 59
Dependent Claim
16 words
The system of claim 44 wherein the prompt comprises a request to repeat a linguistic unit.
Claim 60
Dependent Claim
20 words
The system of claim 44 wherein the prompt comprises a request to complete, fill in or identify a verbal aggregate.
Claim 61
Dependent Claim
34 words
The system of claim 43 wherein the means for extracting indexical or paralinguistic values comprises means for extracting semaphore values including user's native language derived from the user's spoken responses in a target language.
Claim 62
Dependent Claim
28 words
The system of claim 43 wherein the spoken responses from the user are received at the interactive computer system via telephone or other telecommunication or data information network.
Claim 63
Dependent Claim
11 words
The system of claim 43 wherein the prompts are graphical prompts.
Claim 64
Dependent Claim
11 words
The system of claim 43 wherein the prompts are audio prompts.
Claim 65
Dependent Claim
13 words
The system of claim 43 wherein the prompts combine verbal and graphical elements.
References Cited (Backward Citations)
WO1990001203A1
Cited
US5065345A
Cited
US5036539A
Cited
US5255925A
Cited
US5268990A
Cited
US5387104A
Cited
US5475792A
Cited
US5393236A
Cited
WO1994010666A1
Cited
WO1994017508A1
Cited
US5487671A
Cited
WO1994020952A1
Cited
US5634086A
Cited
US5458494A
Cited
US5540589A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-02-08 | 2002-08-10 | 2003-02-09 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-02-08 | 2006-08-10 | 2007-02-09 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-02-08 | 2010-08-10 | 2011-02-09 | $7700.00 | Window Closed / Lapsed |