US5864819A
Expired / Lapsed
US
⏱️ 0.0 years remaining
Internal window object tree method for representing graphical user interface applications for speech navigation
Assignee: International Business Machines Corp
•
Filing: 1996-11-08
•
Granted: 1999-01-26
•
Expires: 2016-11-08
Abstract
Abstract Method for representing a target software application program to a voice recognition navigator program on a computer system. The method requires analyzing an application program to determine a plurality of application states. Each of the application states is defined as a set of window objects within the application for performing a specific user task. According to the invention, each of the application states is preferably represented by a sub-context tree, comprised of a plurality of sub-context objects. The tree allows the navigator to associate decoded spoken commands to specific window objects.
Inventors
- 👤 Mario E. De Armas
- 👤 Harvey M. Ruback
Bibliographic Details
Application Number
—
Jurisdiction
US
Data Source
google_patents
Last Synchronized
2026-11-09 22:15
CPC Classifications
G; G06; G06F; G06F3/00; G06F3/16; G06F8/00; G06F8/30; G06F8/38; G10; G10L; G10L15/00; G10L15/22; G10L2015/226; G10L2015/228
Claims Analysis
Total Claims: 18
Claim 1
Independent Claim
224 words
Method for representing a target software application program to a voice recognition navigator program on a computer system, said method comprising: analyzing an application program to determine a plurality of application states, each of said application states being comprised of a plurality of window objects; organizing each of said application states in the form of a sub-context tree, each of said sub-context trees being comprised of a plurality of sub-context objects defining the hierarchical relationship among each of said sub-context objects; determining a set of attributes associated with each of said window objects, and assigning a matching set of attributes to each of the sub-context objects; storing the sub-context trees in a memory device of said computer system as a context data file; executing said voice recognition navigator program on said computer system concurrently with said target application program; analyzing said target application with said voice navigator program to determine a current target application state of said target application program and matching said current target application state to a corresponding sub-context tree by comparing a window object from the current target application state to at least one of said sub-context objects stored as part of said context data file; defining an active vocabulary set for the current target application state by retrieving from said context data file at least one sub-context object vocabulary set.
Claim 2
Dependent Claim
224 words
Method for representing a target software application program to a voice recognition navigator program on a computer system, said method comprising: analyzing an application program to determine a plurality of application states, each of said application states being comprised of a plurality of window objects; organizing each of said application states in the form of a sub-context tree, each of said sub-context trees being comprised of a plurality of sub-context objects defining the hierarchical relationship among each of said sub-context objects; determining a set of attributes associated with each of said window objects, and assigning a matching set of attributes to each of the sub-context objects; storing the sub-context trees in a memory device of said computer system as a context data file; executing said voice recognition navigator program on said computer system concurrently with said target application program; analyzing said target application with said voice navigator program to determine a current target application state of said target application program and matching said current target application state to a corresponding sub-context tree by comparing a window object from the current target application state to at least one of said sub-context objects stored as part of said context data file; defining an active vocabulary set for the current target application state by retrieving from said context data file at least one sub-context object vocabulary set.
Claim 3
Dependent Claim
41 words
The method according to claim 1 wherein matching of said current target application state to a corresponding sub-context tree is accomplished by matching a first window object from the current application state to a first sub-context object from the sub-context trees.
Claim 4
Dependent Claim
44 words
The method according to claim 2 wherein matching said first window object to a first sub-context object is accomplished by matching a sub-set of window attributes of said first window object to a set of attributes of a first one of said sub-context objects.
Claim 5
Dependent Claim
60 words
The method according to claim 3 further comprising the steps of matching a set of window objects which are children of said first window object to a set of sub-context objects which are children of said first sub-context object, when a unique match cannot be found by comparing the attributes of said first window object to said first sub-context object.
Claim 6
Dependent Claim
24 words
The method according to claim 1, wherein each of said application states is a set of window objects for performing a specific user task.
Claim 7
Dependent Claim
76 words
The method according to claim 1, wherein said plurality of application states are determined by: a. executing the target application to display all of said application states; and b. for each application state of said application program, creating a target application sub-context tree comprised of a plurality of sub-context objects, the set of attributes of each of said sub-context objects being defined by the attributes of one of said window objects of the current application state.
Claim 8
Dependent Claim
37 words
The method of claim 6 wherein said sub-context tree is further comprised of sub-context action objects being defined by a set of attributes describing actions which are independent of said window objects of the current application state.
Claim 9
Dependent Claim
17 words
The method of claim 7 wherein said actions are comprised of macros which simulate user interface events.
Claim 10
Dependent Claim
38 words
The method of claim 1, wherein said target application sub-context tree is organized in a tree structure wherein the parent-child relationships between sub-context objects is the same as the parent-child relationships between the corresponding target application window objects.
Claim 11
Dependent Claim
60 words
The method according to claim 1, wherein said current target application state is matched to said corresponding sub-context tree by comparing their respective sets of attributes, and said sets of attributes are comprised of at least one of: a. a window class name; b. a window identification number; c. a window text string; and d. a window enumeration index value.
Claim 12
Dependent Claim
35 words
The method according to claim 1, wherein said active vocabulary set for the current target application state is created by retrieving and combining all vocabulary sets from each sub-context object of the current sub-context tree.
Claim 13
Dependent Claim
101 words
The method according to claim 11, wherein said vocabulary set for each sub-context object is comprised of the following vocabulary sub-sets which correspond to spoken commands which are understood by the voice navigator: a. a sub-context object name which is the spoken command associated with activating the window object represented by the sub-context object; b. a private vocabulary which is active when the window object represented by the sub-context object is the foreground window; c. a sub-context object command vocabulary for window objects which are treated by the navigator as active, although the associated window object is not the foreground window.
Claim 14
Dependent Claim
46 words
The method according to claim 1 further comprising the steps of a. providing the active vocabulary to a voice recognition engine; b. receiving from said voice recognition engine a decoded phrase which is an element of the active vocabulary, and which matches a user voice utterance.
Claim 15
Dependent Claim
49 words
The method according to claim 1 wherein a decoded phrase which is an element of the active vocabulary is received by said voice navigator program, and said voice navigator program associates said decoded phrase with a matching target sub-context object containing a vocabulary element corresponding to said decoded phrase.
Claim 16
Dependent Claim
25 words
The method according to claim 14 further comprising the step of executing the desired action represented by the decoded phrase within the target sub-context object.
Claim 17
Dependent Claim
37 words
The method according to claim 15 wherein said step of executing said desired action represented by the decoded phrase is performed by matching said target sub-context object to a target window object of the current application state.
Claim 18
Dependent Claim
129 words
The method according to claim 16 wherein said matching of said sub-context object to a window object is performed by the steps of: defining a path between said target sub-context object and a foreground sub-context object by traversing up said sub-context tree one level at a time, beginning with said target sub-context object, and comparing the attributes of each of said sub-context objects along said path to the attributes of a foreground window object until a match is determined; traversing down said path, one level at a time, beginning with said foreground sub-context window object, matching at each level the attributes of one of said sub-context objects to a corresponding window object which is a descendent of the previously matched window object, until a target window object is identified.
References Cited (Backward Citations)
US4821211A
Cited
JPH02281326A
Cited
US5263117A
Cited
US5374924A
Cited
US5386494A
Cited
US5412776A
Cited
EP0627691A1
Cited
US5461399A
Cited
WO1995018413A1
Cited
JPH07325797A
Cited
USPTO Annuity & Maintenance Windows
| Annuity Stage | Window Opens | Payment Deadline | Surcharge Period Until | Estimated Fee | Status |
|---|---|---|---|---|---|
| 3.5 Years (Year 4) | 2002-01-25 | 2002-07-27 | 2003-01-26 | $2000.00 | Window Closed / Lapsed |
| 7.5 Years (Year 8) | 2006-01-25 | 2006-07-27 | 2007-01-26 | $3760.00 | Window Closed / Lapsed |
| 11.5 Years (Year 12) | 2010-01-25 | 2010-07-27 | 2011-01-26 | $7700.00 | Window Closed / Lapsed |