US6012030A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Management of speech and audio prompts in multimodal interfaces
Assignee: Avaya Inc
•
Filing: 1998-04-21
•
Granted: 2000-01-04
•
Expires: 2018-04-21
Abstract
Abstract Management of speech and audio prompts, and interface presence, in multimodal user interfaces is provided. A communications device having a multimodal user interface including a speech interface, and a non-speech interface, e.g. a graphical or tactile user interface, comprises means for dynamically switching between a background state of the speech interface and a foreground state of the speech interface in accordance with a users input modality choice. Preferably, in the foreground state speech prompts and speech based error recovery are fully implemented and in a background state speech prompts are replaced by earcons, and no speech based error recovery is implemented. Thus there is provided a device which automatically subdue the speech prompts when a user selects a non-speech input/output mechanism. Also provided is a method for dynamic adjustment of audio prompts and speech prompts by switching from a foreground state to a background state of a speech interface in response to a users current interaction modality, by selecting alternative states for speech and audio interfaces that represent users needs for speech prompts. This type of system and method is particularly useful and applicable to hand held Internet access communication devices.
Inventors
- 👤 Marilyn French-St. George
- 👤 Nicola Fumai
- 👤 Henry Adam Pasternack
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 22:12
CPC Classifications
H; H04; H04M; H04M3/00; H04M3/42; H04M3/487; H04M3/493; H04M3/4936; G; G10; G10L; G10L15/00; G10L15/26; H04M1/00; H04M1/72; H04M1/724; H04M1/72403; H04M1/72448; H04M1/26; H04M1/27; H04M1/271; H04M1/72445; H04M2201/00; H04M2201/40; H04M2250/00; H04M2250/56; H04M2250/74
Claims Analysis
Total Claims: 39
Claim 1
Independent Claim
59 words
A communications device having a multimodal user interface offering a user a choice of input modalities, comprising a speech interface for accessing a speech recognizer, and a graphical user interface, and comprising means for dynamically switching between a foreground state of a speech interface and a background state of a speech interface in accordance with a users input modality.
Claim 2
Dependent Claim
59 words
A communications device having a multimodal user interface offering a user a choice of input modalities, comprising a speech interface for accessing a speech recognizer, and a graphical user interface, and comprising means for dynamically switching between a foreground state of a speech interface and a background state of a speech interface in accordance with a users input modality.
Claim 3
Dependent Claim
21 words
A device according to claim 1 wherein, in the foreground state audio prompts and full speech based error recovery are implemented.
Claim 4
Dependent Claim
21 words
A device according to claim 2 wherein audio prompts comprise one of speech prompts, earcons, and both speech prompts and earcons.
Claim 5
Dependent Claim
23 words
A device according to claim 1 wherein in the background state audio prompts comprising earcons, and no speech based error recovery are implemented.
Claim 6
Dependent Claim
25 words
A device according to claim 3 wherein in the background state speech prompts are replaced by earcons, and no speech based error recovery are implemented.
Claim 7
Dependent Claim
55 words
A device according to claim 1 wherein, in the foreground state, audio prompts comprising one of speech prompts, earcons, and both speech prompts and earcons are implemented and full speech based error recovery is implemented, and in the background state, audio prompts comprising earcons replacing speech prompts, and no speech based error recovery are implemented.
Claim 8
Dependent Claim
21 words
A device according to claim 6 wherein the speech prompts are automatically subdued when a user selects a non-speech input modality.
Claim 9
Dependent Claim
25 words
A device according to claim 1 wherein the device also includes at least one of keypad interface, a tactile interface and a pen input interface.
Claim 10
Dependent Claim
10 words
A device according to claim 1 comprising a mobile telephone.
Claim 11
Dependent Claim
13 words
A device according to claim 1 comprising a hand held Internet communication device.
Claim 12
Dependent Claim
16 words
A device according to claim 1 wherein the speech recognizer is accessed on a network system.
Claim 13
Dependent Claim
15 words
A device according to claim 1 wherein the speech recognizer is provided by the device.
Claim 14
Independent Claim
81 words
A method for dynamic adjustment of audio prompts in response to a users interaction modality in a communications system having a multimodal interface comprising a speech interface for accessing a speech recognizer and another interface, the method comprising: after prompting a user for input and capturing input, generating an input identifier associated with the user input modality, determining the mode of user input modality and selecting a corresponding one of a foreground state and a background state of the speech interface.
Claim 15
Independent Claim
81 words
A method for dynamic adjustment of audio prompts in response to a users interaction modality in a communications system having a multimodal interface comprising a speech interface for accessing a speech recognizer and another interface, the method comprising: after prompting a user for input and capturing input, generating an input identifier associated with the user input modality, determining the mode of user input modality and selecting a corresponding one of a foreground state and a background state of the speech interface.
Claim 16
Dependent Claim
38 words
A method according to claim 13 comprising when the users input modality is speech, selecting the foreground state of the speech interface, and when the users input modality is non-speech, selecting a background mode of the speech interface.
Claim 17
Dependent Claim
60 words
A method according to claim 14 wherein in the foreground state of the speech interface, implementing audio prompts comprising one of speech prompts, earcons, and earcons and speech prompts, and implementing full speech based error recovery, and, in the background state of the speech interface, implementing audio prompts comprising earcons replacing speech prompts, and implementing no speech based error recovery.
Claim 18
Dependent Claim
16 words
A method according to claim 14, wherein in a foreground state, audio prompts include speech prompts.
Claim 19
Dependent Claim
38 words
A method according to claim 13 wherein when the user's input modality comprises more than one input mode, the method comprises selecting an appropriate foreground state or background state of the speech interface according to a precedence system.
Claim 20
Dependent Claim
40 words
A method according to claim 13 wherein when the user's input modality comprises more than one input mode, the method comprises selecting an appropriate foreground state or background state of the speech interface according to a context of the application.
Claim 21
Dependent Claim
38 words
A method according to claim 13 wherein when the user's input modality comprises more than one input mode, the method comprises selecting an appropriate foreground state or background state of the speech interface according to a selection system.
Claim 22
Dependent Claim
44 words
A method according to claim 13 wherein when the user's input modality comprises more than one input mode, the method comprises selecting an appropriate foreground state or background state of the speech interface based on predetermined set of priorities associated with each input modality.
Claim 23
Dependent Claim
32 words
A method according to claim 13 wherein the method comprises determining an appropriate selection of the background or foreground state of the speech interface based on a sequence of previous user inputs.
Claim 24
Dependent Claim
27 words
A method according to claim 13 wherein selection of the background or foreground state of the speech interface is based on the expected needs of the user.
Claim 25
Dependent Claim
23 words
A method according to claim 13 wherein when the user selected a non-speech input/output mechanism, switching the speech interface to the background state.
Claim 26
Dependent Claim
41 words
A method according to claim 13 wherein in the background state a speech recognizer of the speech interface is put into a conservative recognition state, in which the recognizer responds only to a very limited set of speech and audio input.
Claim 27
Dependent Claim
34 words
A method according to claim 23 wherein the background state is selected until speech input is recognized by the recognizer and then switching from the background state to the foreground state of the interface.
Claim 28
Dependent Claim
87 words
Software on a computer readable medium for carrying out a method for dynamic adjustment of speech prompts in response to a users interaction modality in a communications system having a multimodal interface comprising a speech interface for accessing a speech recognizer, and another interface, the method comprising: after prompting a user for input and capturing input, generating an identifier associated with the user input modality, determining the user input modality and selecting a corresponding one of a foreground state and a background state of the speech interface.
Claim 29
Dependent Claim
87 words
Software on a computer readable medium for carrying out a method for dynamic adjustment of speech prompts in response to a users interaction modality in a communications system having a multimodal interface comprising a speech interface for accessing a speech recognizer, and another interface, the method comprising: after prompting a user for input and capturing input, generating an identifier associated with the user input modality, determining the user input modality and selecting a corresponding one of a foreground state and a background state of the speech interface.
Claim 30
Independent Claim
62 words
A system for dynamic adjustment of audio prompts in response to a users interaction modality with a communications device having a multimodal interface comprising a speech interface for accessing a speech recognizer and another interface, comprising: means for determining the mode of user input modality and selecting a corresponding one of a foreground state and a background state of the speech interface.
Claim 31
Independent Claim
62 words
A system for dynamic adjustment of audio prompts in response to a users interaction modality with a communications device having a multimodal interface comprising a speech interface for accessing a speech recognizer and another interface, comprising: means for determining the mode of user input modality and selecting a corresponding one of a foreground state and a background state of the speech interface.
Claim 32
Dependent Claim
59 words
A system according to claim 27 wherein the foreground state of the speech interface provides audio prompts comprising one of speech prompts, earcons, and earcons and speech prompts, and provides full speech based error recovery, and in the background mode of the speech interface provides audio prompts comprising earcons replacing speech prompts, and provides no speech based error recovery.
Claim 33
Dependent Claim
16 words
A system according to claim 28, wherein in a foreground state, audio prompts include speech prompts.
Claim 34
Dependent Claim
34 words
A system according to claim 27 comprising means for selecting an appropriate foreground state or background state of the speech interface according to a precedence system when input is captured for multiple input modalities.
Claim 35
Dependent Claim
36 words
A system according to claim 27 comprising means for selecting an appropriate foreground state or background state of the speech interface based on a context of the application when input is captured for multiple input modalities.
Claim 36
Dependent Claim
34 words
A system according to claim 27 comprising means for selecting an appropriate foreground state or background state of the speech interface according to a selection system when input is captured for multiple input modalities.
Claim 37
Dependent Claim
40 words
A system according to claim 27 comprising means for selecting an appropriate foreground state or background state of the speech interface based on predetermined set of priorities associated with each input modality when input is captured for multiple input modalities.
Claim 38
Dependent Claim
29 words
A system according to claim 27 comprising means for selecting an appropriate foreground state or background state of the speech interface based on a sequence of previous user inputs.
Claim 39
Dependent Claim
29 words
A system according to claim 27 comprising means for selecting an appropriate foreground state or background state of the speech interface based on the expected needs of the user.
References Cited (Backward Citations)
US5632002A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2003-01-03 | 2003-07-05 | 2004-01-04 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2007-01-03 | 2007-07-05 | 2008-01-04 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2011-01-03 | 2011-07-05 | 2012-01-04 | $7700.00 | Window Closed / Lapsed |