System and method for inter-token whitespace representation and textual editing behavior in a program editor

Claim 2

A process for visual whitespace representation and cursor behavior in a computer program editor configured to edit a computer program represented as a stream of tokens from a display of said program as if said computer program were represented as a text flow, said process comprising the steps of: displaying each line of said computer program as it is entered by a user, wherein a displayed line comprises a sequence of tokens and visual whitespace separating adjacent tokens, said visual whitespace being based on lexical types of a first and a second token composing said adjacent tokens and not on spaces entered by said user, said spaces not being recorded in the token stream; displaying a cursor in said displayed line of said computer program, said cursor providing user interactability with said displayed line, said cursor having a cursor position corresponding to an insertion point in said stream lying between editable elements composing said tokens, between said tokens or at the beginning or end of said stream; maintaining boundaries between said tokens and updating said insertion position as said user edits said line of said computer program; and updating said cursor position, as said user edits said line of said computer program, in accordance with a set of cursor display rules as applied to ones of said tokens adjacent said cursor position such that said cursor position is updated as though said user were editing said line of said computer program with a text editor.

Claim 3

The process of claim 1 wherein said maintaining step further comprises the steps of: internally representing unambiguous boundaries between tokens with an implicit token boundary; wherein said implicit token boundary is not represented by an editable element in said token stream; internally representing ambiguous boundaries between said tokens with a separator, or ordinary separator, represented as an editable element in said token stream, wherein said ambiguous boundaries are between said adjacent tokens that could be legitimately joined into a single token according to the rules of the computer language in which said computer program was written; and internally representing a potentially ambiguous boundary with a provisional separator, said potentially ambiguous boundary corresponding to a boundary between tokens where a user typed a space and where no additional separator was necessary.

Claim 4

The process of claim 2 wherein: said insertion point is classified as having one of a multiplicity of predefined distinct insertion point contexts; and said insertion point updating step includes executing one of a plurality of distinct insertion point update methods selected in accordance with said classified insertion point context and said user editing actions.

Claim 5

The process of claim 3 wherein said multiplicity of predefined contexts comprises six general categories: "1"--between said editable elements comprising one of said tokens, "2"--between said adjacent tokens sharing said implicit token boundary, "3"--between said adjacent tokens and to the left of said separator, "4"--between said adjacent tokens and to the right of said separator, "5"--between said adjacent tokens, with said provisional separator to the left of said insertion point but no separator to the right of said insertion point and "6"--between tokens, with said provisional separator to the left of said insertion point and said separator to the right of said insertion point.

Claim 6

The process of claim 4 wherein said editing actions comprise: editable element deletion, character insertion, space insertion, editing of string literals, and navigating over said editable elements comprising said stream.

Claim 7

The process of claim 5, wherein said editing action is said character insertion, said step of insertion point updating and token boundary maintaining further comprising the steps of: when said insertion point context is selected from said insertion point contexts 1, 2, 4, or 5, performing said insertion, re-analyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1, 2 or 3 immediately following said character just inserted based on said re-analyzing of said token stream following said insertion operation; and when said insertion point context is selected from said insertion point contexts 3 or 6, performing said insertion, re-analyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 2 or 3 immediately following said character just inserted based on said re-analyzing of said token stream following said insertion operation.

Claim 8

The process of claim 5, wherein said editing action is said editable element deletion, said step of insertion point updating and token boundary maintaining further comprising the steps of: when said insertion point context is said insertion point context 1, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1, 2 or 3 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; when said insertion point context is said insertion point context 2, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 2 or 3 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; when said insertion point context is said insertion point context 3, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1 or 4 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; when said insertion point context is said insertion point context 4 or 5, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 2 or 4 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; and when said insertion point context is said insertion point context 6, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1 or 4 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation.

Claim 9

The process of claim 5, wherein said editing action is said space insertion, said step of insertion point updating and token boundary maintaining further comprising the steps of: when said insertion point context is said insertion point context 1, splitting said token between said editable elements to form two tokens, reanalyzing said resulting token stream and placing said insertion point in said insertion point context 4; when said insertion point context is said insertion point context 2, inserting a provisional separator between said tokens and placing said insertion point in said insertion point context 4; when said insertion point context is said insertion point context 3, moving said insertion point over said separator and placing said insertion point in said insertion point context 4, to the right of said separator; when said insertion point context is selected from said insertion point contexts 4, 5 or 6, not changing said cursor position or said token stream.

Claim 10

The process of claim 5 wherein said editing action is said editing of string literals, said step of insertion point updating and token boundary maintaining further comprising the steps of: detecting string delimiting characters indicating boundaries of said string literals; matching said string delimiting characters, wherein said matching involves identifying pairs of said string delimiting characters, wherein said pair includes a string beginning delimiter and a string ending delimiter; forming a token representing all characters, wherein said characters includes spaces, inserted by said user between said string beginning delimiter and said string ending delimiter; and when said tokens exist between said delimiters, including said tokens in said string as editable elements of said string and deleting said included tokens from said token stream.

Claim 11

The process of claim 5 wherein said editing action is navigating over said editable elements of said token stream, said step of insertion point updating comprising moving said cursor over said editable elements in response to left cursor and right cursor keystrokes typed by said user and mouseclicks.

Claim 12

The process of claim 2, wherein said separators do not exist between said adjacent tokens in said token stream, but instead are inferable from an approximate separator table defining whether said separator exists between adjacent separators having a range of extended lexical types, said separator table being conservative, meaning said table prescribes separators between first and second adjacent tokens when tokens sharing said extended lexical types of said first and second tokens could legitimately be joined according to said computer language rules in at least some situations.

Claim 13

The process of claim 2, wherein said separators are not inserted between said adjacent tokens in said token stream, but instead are inferable from an exact separator table defining whether said separator exists between said adjacent tokens based on individual and lexical attributes of said adjacent tokens.

Claim 16

The system of claim 13 wherein said keystroke executive maintains said token stream by: internally representing unambiguous boundaries between said tokens with an implicit token boundary; wherein said implicit token boundary is not represented by an editable element in said token stream; internally representing ambiguous boundaries between said tokens with a separator, or ordinary separator, represented as an editable element in said token stream, wherein said ambiguous boundaries are between said adjacent tokens that could be legitimately joined into a single token according to the rules of the computer language in which said computer program was written; and internally representing a potentially ambiguous boundary with a provisional separator, said potentially ambiguous boundary corresponding to a boundary between tokens where a user typed a space and where no additional separator was necessary.

Claim 17

The system of claim 14 wherein: said insertion point is classified as having one of a multiplicity of predefined distinct insertion point contexts; and said keystroke executive updates said token stream according to one of a plurality of said token stream updating rules selected in accordance with said classified insertion point context and said user editing actions.

Claim 18

The system of claim 15 wherein said multiplicity of predefined contexts comprises six general categories: "1"--between said editable elements comprising one of said tokens, "2"--between said adjacent tokens sharing said implicit token boundary, "3"--between said adjacent tokens and to the left of said separator, "4"--between said adjacent tokens and to the right of said separator, "5"--between said adjacent tokens, with said provisional separator to the left of said insertion point but no separator to the right of said insertion point and "6"--between tokens, with a said provisional separator to the left of said insertion point and no separator to the right of said insertion point.

Claim 19

The system of claim 16 wherein said editing actions comprise: (1) editable element deletion, (2) character insertion, (3) space insertion; (4) editing of string literals; and (5) navigating over said editable elements comprising said stream.

Claim 20

The system of claim 17, wherein, when said editing action is said character insertion, said keystroke executive updates said token stream by: when said insertion point context is selected from said insertion point contexts 1, 2, 4, or 5, perform said insertion, re-analyzing said token stream according to said token stream updating rules, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1, 2 or 3 immediately following said character just inserted based on said re-analyzing of said token stream following said insertion operation; and when said insertion point context is selected from said insertion point contexts 3 or 6, performing said insertion, re-analyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 2 or 3 immediately following said character just inserted based on said re-analyzing of said token stream following said insertion operation.

Claim 21

The system of claim 17, wherein, when said editing action is said editable element deletion, said keystroke executive updates said token stream by: when said insertion point context is said insertion point context 1, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1, 2 or 3 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; when said insertion point context is said insertion point context 2, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1, 2 or 3 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; when said insertion point context is said insertion point context 3, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1 or 4 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; when said insertion point context is selected from said insertion point contexts 4 or 5, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 2 or 4 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation; and when said insertion point context is said insertion point context 6, performing said deletion, reanalyzing said token stream, and placing said insertion point and said cursor in a post-analysis position selected from insertion point contexts 1 or 4 with respect to said editable elements adjacent to deleted editable element based on said reanalysis of said token stream following said deletion operation.

Claim 22

The system of claim 17, wherein, when said editing action is said space insertion, said keystroke executive updates said token stream by: when said insertion point context is said insertion point context 1, splitting said token between said editable elements to form two tokens, reanalyzing said resulting token stream and placing said insertion point in said insertion point context 4; when said insertion point context is said insertion point context 2, inserting a provisional separator between said tokens and placing said insertion point in said insertion point context 4; when said insertion point context is said insertion point context 3, moving said insertion point over said separator and placing said insertion point in said insertion point context 4, to the right of said separator; when said insertion point context is selected from said insertion point contexts 4, 5 or 6, not changing said cursor position or said token stream.

Claim 23

The system of claim 17 wherein, when said editing action is said editing of string literals, said keystroke executive updates said token stream by: detecting string delimiting characters indicating boundaries of said string literals; matching said string delimiting characters, wherein said matching involves identifying pairs of said string delimiting characters, wherein said pair includes a string beginning delimiter and a string ending delimiter; forming a token representing all characters, wherein said characters includes spaces, inserted by said user between said string beginning delimiter and said string ending delimiter; and when said tokens exist between said delimiters, including said tokens in said string as editable elements of said string and deleting said included tokens from said token stream.

Claim 24

The system of claim 17 wherein, when said editing action is navigating over said editable elements of said token stream, said keystroke executive moves said cursor over said editable elements in response to left cursor and right cursor keystrokes typed by said user on said input device.

Claim 25

The system of claim 14, further comprising an approximate separator table accessible by said keystroke executive and said typographical display processor; wherein said separators do not exist between said adjacent tokens in said token stream, but instead are inferable from said approximate separator table defining whether said separator exists between adjacent separators having a range of extended lexical types, said separator table being conservative, meaning said table prescribes separators between first and second adjacent tokens when tokens sharing said extended lexical types of said first and second tokens could legitimately be joined according to said computer language rules in at least some situations.

Claim 26

The system of claim 14, further comprising an exact separator table accessible by said keystroke executive and said typographical display processor; wherein said separators are not inserted between said adjacent tokens in said token stream, but instead are inferable from an exact separator table defining whether said separator exists between said adjacent tokens based on individual and lexical attributes of said adjacent tokens.