System and method for textual editing of structurally-represented computer programs with on-the-fly typographical display

Claim 2

A process for editing structurally represented computer programs with on-the-fly prettyprinting comprising the steps of: maintaining a token stream representation of a computer program after each input event that occurs as said program is entered, wherein said token stream representation comprises a sequence of tokens, each of said tokens characterized according to a set of extended lexical properties relative to the language of said computer program, said maintaining step requiring no maintenance of a text stream representation of said program; and typographically displaying said program as said program is entered based on said token stream representation of said program, wherein said extended lexical properties of said tokens influence said typographical display of said tokens.

Claim 3

The editing process of claim 1, further comprising the step of: generating an editable text stream corresponding to said token stream representation of said typographically displayed program.

Claim 4

The editing process of claim 1, further comprising the step of: maintaining a structured representation of said program, wherein said structured representation additionally characterizes said tokens according to a set of extended syntactical properties; wherein said typographical displaying step comprises: typographically displaying said program as said program is entered based on said structured and said token stream representations of said program, wherein said extended lexical and syntactical properties of said tokens influence said typographical display of said tokens.

Claim 5

The editing process of claim 3, further comprising the step of: generating an editable text stream corresponding to said token stream and said structured representations of said typographically displayed program.

Claim 6

A process for editing structurally represented computer programs with on-the-fly prettyprinting comprising the steps of: tokenizing a computer program after each input event that occurs as said program is entered, wherein said tokenizing involves generating a token stream representation of said computer program and characterizing each token in said token stream as at least one of a catalog of extended lexical properties relative to the language of said computer program, said tokenizing step requiring no maintenance of a text stream representation of said program; typographically displaying said program as said program is entered based on said token stream representation of said program, wherein said extended lexical properties of said tokens influence said typographical display of said tokens; and generating an editable text stream corresponding to said token stream representation of said typographically displayed program.

Claim 7

A process for editing structurally represented computer programs with on-the-fly prettyprinting comprising the steps of: tokenizing a computer program after each input event that occurs as said program is entered, wherein said tokenizing involves generating a token stream representation of said computer program and characterizing each token in said token stream as at least one of a catalog of extended lexical properties relative to the language of said computer program, said tokenizing step requiring no maintenance of a text stream representation of said program; typographically displaying said program as said program is entered based on said token stream representation of said program, wherein said extended lexical properties of said tokens influence said typographical display of said tokens; and generating an editable text stream corresponding to said token stream representation of said typographically displayed program.

Claim 8

The editing process of claim 5, wherein said extended lexical properties associated with each of said tokens include a lexical class and, when said token is not well-formed, an extended type selected from ill-formed or incomplete.

Claim 9

The editing process of claim 6, wherein well-formed tokens, ill-formed tokens and incomplete tokens are typographically displayed in a visually distinct manner associated with their respective extended-types.

Claim 10

The editing process of claim 7, wherein said ill-formed tokens are typographically displayed with a first preselected color overlay and said incomplete tokens are displayed with a second preselected color overlay.

Claim 11

The editing process of claim 7, wherein said tokens are typographically displayed in a visually distinct manner tied to their respective lexical classes.

Claim 12

The editing process of claim 5, wherein said tokenizing step comprises the steps of: receiving said input event generated by at least one input device as said program is entered; determining an insertion point in said token stream corresponding to said input event; incrementally updating said token stream by lexically analyzing a token stream segment consisting of tokens from said token stream likely to be affected by said input event; and updating said insertion point in light of said incrementally updated token stream; wherein said typographical displaying step includes displaying a cursor providing user interactability with said typographically displayed program at a screen position corresponding to said insertion point.

Claim 13

The editing process of claim 10, wherein said token stream segment consists of a left token immediately to the left of said insertion point, all right tokens immediately to the right of said token, and when said insertion point is inside said token, said token including said insertion point.

Claim 14

The editing process of claim 5, further comprising the step of: parsing said token stream to form a structured representation of said computer program, said structured representation reflecting the syntax of a computer language in which said computer program is expressed; and wherein said typographical displaying step comprises: typographically displaying said program as said program is entered based on said structured representation and said token stream representation of said program; wherein said extended lexical properties and said structured representation influence said typographical display of said tokens.

Claim 15

The editing process of claim 12, wherein said parsing step is executed either opportunistically or in response to predetermined user actions, where opportunistically means when computer resources are available.

Claim 16

An editor for structurally represented computer programs comprising: at least one input device configured to receive a computer program into said editor, said at least one input device generating a stream of input events as said program is entered; an extended lexical analyzer that represents said program as a token stream and characterizes each token in said token stream as at least one of a catalog of extended lexical properties relative to the language of said computer program without being required to maintain a text stream representation of said program, said extended lexical analyzer updating said token stream representation and characterization after each said input event; a typographical display processor configured to prettyprint said token stream based on said extended lexical properties of said tokens as said program is entered.

Claim 17

An editor for structurally represented computer programs comprising: at least one input device configured to receive a computer program into said editor, said at least one input device generating a stream of input events as said program is entered; an extended lexical analyzer that represents said program as a token stream and characterizes each token in said token stream as at least one of a catalog of extended lexical properties relative to the language of said computer program without being required to maintain a text stream representation of said program, said extended lexical analyzer updating said token stream representation and characterization after each said input event; a typographical display processor configured to prettyprint said token stream based on said extended lexical properties of said tokens as said program is entered.

Claim 18

The editor of claim 14, wherein said extended lexical properties associated with each of said tokens include a lexical class and, when said token is not well-formed, an extended type selected from ill-formed or incomplete.

Claim 19

The editor of claim 15, wherein well-formed tokens, ill-formed tokens and incomplete tokens are typographically displayed in a visually distinct manner associated with their respective extended types.

Claim 20

The editor of claim 16, wherein said typographical display processor displays said ill-formed tokens with a first preselected color overlay and said incomplete tokens with a second preselected color overlay.

Claim 21

The editor of claim 17, wherein said typographical display processor displays said tokens in a visually distinct manner tied to their respective lexical classes.

Claim 22

The editor of claim 14, wherein said extended lexical analyzer comprises: a keystroke executive configured to receive each input event generated by said at least one input device as said program is entered, said keystroke executive maintaining said token stream representation and an insertion point as said program is entered; and a tokenizer callable by said keystroke executive, said tokenizer being an expert at incremental, lexical analysis of token streams, said keystroke executive passing said tokenizer a token stream segment consisting of a tokens proximal to said insertion point likely to be affected by said input event and asking said tokenizer to suggest an update to said token steam segment based on said most recent input event, said update to said token stream segment including an updated cursor position; such that when said keystroke executive adopts said suggestion, said keystroke executive updates said token stream with said updated token stream segment and updates said insertion point to be consistent with said updated cursor position; wherein said typographical display processor displays said cursor at a screen position corresponding to said insertion point.

Claim 23

The editor of claim 19, wherein said tokenizer analyzes said token stream segment based on extended lexical rules for the programming language in which said program is written.

Claim 24

The editor of claim 20, wherein, when said programming language is selected from C or C++, said token stream segment consists of a left token immediately to the left of said insertion point, all right tokens immediately to the right of said token, and when said insertion point is inside said token, said token including said insertion point.

Claim 25

The editor of claim 14, further comprising: a structural analyzer configured to parse said token stream to form a structured representation of said computer program, said structured representation reflecting the syntax of a computer language in which said computer program is expressed; wherein said typographical display processor typographically displays said program as said program is entered based on said structured representation and said token stream representation of said program; wherein said extended lexical properties influence said typographical display of said tokens.

Claim 26

The editor of claim 22, wherein said parser executes either opportunistically or in response to predetermined user actions, where opportunistically means when computer resources are available.

Claim 27

A computer system, comprising: a processor; at least one input device configured to receive a computer program, said at least one input device generating a stream of input events as said program is entered; a display on which said program is displayed as said program is being edited, said display and said at least one input device being coupled to said processor and controlled by applications running in said processor; a primary memory coupled to said processor, and storing said applications and data for said applications; a secondary memory coupled to said processor storing said applications and said data for said applications, said secondary memory being slower than said primary memory; and a computer program editor in which a computer program being edited is internally represented as a stream of tokens, said computer program editor being one of said applications running in said processor, said computer program being edited being one of said data, said program editor including: an extended lexical analyzer configured to represent said program as a token stream and to characterize each token in said token stream as at least one of a catalog of extended lexical properties relative to the language of said computer program without being required to maintain a text stream representation of said program, said extended lexical analyzer updating said token stream representation and characterization after each said input event; and a typographical display processor configured to prettyprint said token stream to said display based on said extended lexical properties of said tokens as said program is entered.

Claim 28

A computer system, comprising: a processor; at least one input device configured to receive a computer program, said at least one input device generating a stream of input events as said program is entered; a display on which said program is displayed as said program is being edited, said display and said at least one input device being coupled to said processor and controlled by applications running in said processor; a primary memory coupled to said processor, and storing said applications and data for said applications; a secondary memory coupled to said processor storing said applications and said data for said applications, said secondary memory being slower than said primary memory; and a computer program editor in which a computer program being edited is internally represented as a stream of tokens, said computer program editor being one of said applications running in said processor, said computer program being edited being one of said data, said program editor including: an extended lexical analyzer configured to represent said program as a token stream and to characterize each token in said token stream as at least one of a catalog of extended lexical properties relative to the language of said computer program without being required to maintain a text stream representation of said program, said extended lexical analyzer updating said token stream representation and characterization after each said input event; and a typographical display processor configured to prettyprint said token stream to said display based on said extended lexical properties of said tokens as said program is entered.

Claim 29

The computer system of claim 24, wherein said extended lexical properties associated with each of said tokens include a lexical class and, when said token is not well-formed, an extended type selected from ill-formed or incomplete.

Claim 30

The computer system of claim 25, wherein well-formed tokens, ill-formed tokens and incomplete tokens are typographically displayed in a visually distinct manner associated with their respective extended types.

Claim 31

The computer system of claim 26, wherein said typographical display processor displays said ill-formed tokens with a first preselected color overlay and said incomplete tokens with a second preselected color overlay.

Claim 32

The computer system of claim 27, wherein said typographical display processor displays said tokens in a visually distinct manner tied to their respective lexical classes.

Claim 33

The computer system of claim 24, wherein said extended lexical analyzer comprises: a keystroke executive configured to receive each input event generated by said at least one input device as said program is entered, said keystroke executive maintaining said token stream representation and an insertion point as said program is entered; and a tokenizer callable by said keystroke executive, said tokenizer being an expert at incremental, lexical analysis of token streams, said keystroke executive passing said tokenizer a token stream segment consisting of a tokens proximal to said insertion point likely to be affected by said input event and asking said tokenizer to suggest an update to said token steam segment based on said most recent input event, said update to said token stream segment including an updated cursor position; such that when said keystroke executive adopts said suggestion, said keystroke executive updates said token stream with said updated token stream segment and updates said insertion point to be consistent with said updated cursor position; wherein said typographical display processor displays said cursor at a screen position corresponding to said insertion point.

Claim 34

The computer system of claim 29, wherein said tokenizer analyzes said token stream segment based on extended lexical rules for the programming language in which said program is written.

Claim 35

The computer system of claim 30, wherein, when said programming language is selected from C or C++, said token stream segment consists of a left token immediately to the left of said insertion point, all right tokens immediately to the right of said token, and when said insertion point is inside said token, said token including said insertion point.

Claim 36

The computer system of claim 24, further comprising: a structural analyzer configured to parse said token stream to form a structured representation of said computer program, said structured representation reflecting the syntax of a computer language in which said computer program is expressed; wherein said typographical display processor typographically displays said program as said program is entered based on said structured representation and said token stream representation of said program; wherein said extended lexical properties influence said typographical display of said tokens.

Claim 37

The computer system of claim 32, wherein said parser is executable either opportunistically or in response to predetermined user actions, where opportunistically means when computer resources are available.