Packages  This Package  Prev  Next  Index  

public  class  java.io.StreamTokenizer
    extends  java.lang.Object  (I-§1.12)
{
        // Fields
    public double nval;	§2.22.1
    public String sval;	§2.22.2
    public int ttype;	§2.22.3

        // possible values for the ttype field
    public final static int TT_EOF;	§2.22.4
    public final static int TT_EOL;	§2.22.5
    public final static int TT_NUMBER;	§2.22.6
    public final static int TT_WORD;	§2.22.7

        // Constructors
    public StreamTokenizer(InputStream  I);	§2.22.8

        // Methods
    public void commentChar(int  ch);	§2.22.9
    public void eolIsSignificant(boolean  flag);	§2.22.10
    public int lineno();	§2.22.11
    public void lowerCaseMode(boolean  fl);	§2.22.12
    public int nextToken();	§2.22.13
    public void ordinaryChar(int  ch);	§2.22.14
    public void ordinaryChars(int  low, int  hi);	§2.22.15
    public void parseNumbers();	§2.22.16
    public void pushBack();	§2.22.17
    public void quoteChar(int  ch);	§2.22.18
    public void resetSyntax();	§2.22.19
    public void whitespaceChars(int  low, int  hi);	§2.22.20
    public void slashStarComments(boolean  flag);	§2.22.21
    public String toString();	§2.22.22
    public void whitespaceChars(int  low, int  hi);	§2.22.23
    public void wordChars(int  low, int  hi);	§2.22.24
}

Each byte read from the input stream is regarded as a character in the range '\u0000' through '\u00FF'. The character value is used to look up five possible attributes of the character: whitespace, alphabetic, numeric, string quote, and comment character. Each character can have zero or more of these attributes.

In addition an instance has four flags. These flags indicate:

• Whether line terminators are to be returned as tokens or treated as whitespace that merely separates tokens.
• Whether C-style comments are to be recognized and skipped.
• Whether C++-style comments are to be recognized and skipped.
• Whether the characters of identifiers are converted to lowercase.

If the current token is a number, this field contains the value of that number. The current token is a number when the value of the ttype field (I-§2.22.3) is TT_NUMBER (I-§2.22.6).

If the current token is a word token, this field contains a string giving the characters of the word token. When the current token is a quoted string token, this field contains the body of the string.

The current token is a word when when the value of the ttype field (I-§2.22.3) is TT_WORD (I-§2.22.7). The current token is a quoted string token when the value of the ttype field is a quote character (see I-§2.22.18).

After a call to the nextToken method (I-§2.22.13), this field contains the type of the token just read. For a single character token, its value is the single character, converted to an integer. For a quoted string token (see I-§2.22.18), its value is the quote character. Otherwise, its value is one of the following:

• TT_WORD (I-§2.22.7) indicates that the token is a word.

• TT_NUMBER (I-§2.22.6) indicates that the token is a number
• TT_EOL (I-§2.22.5) indicates that the end of line has been read. The field can only have this value if the eolIsSignficant method (I-§2.22.10) has been called with the argument true.
• TT_EOF (I-§2.22.4) indicates that the end of the input stream has been reached.

A constant indicating that the end of the stream has been read.

A constant indicating that the end of line has been read.

A constant indicating that a number has been read.

A constant indicating that a token has been read.

Creates a stream tokenizer that parses the specified input stream. The stream tokenizer is initialized to the following default state:

• All byte values 'A' through 'Z', 'a' through 'z', and '\u00A0' through '\u00FF' are considered to be alphabetic.

• All byte values '\u0000' through '\u0020' are considered to be whitespace.
• '/' is a comment character.
• Single quote '\'' and double quote '"' are string quote characters.
• Numbers are parsed
• End of lines are not treated as whitespace, not as separate tokens.
• C-style and C++-style comments are not recognized.

Parameters:

I - the input stream

Specified that the character argument starts a single line comment. All characters from the comment character to the end of the line are ignored by this stream tokenizer.

Parameters:

ch - the character

If the flag argument is true, this tokenizer treats end of lines as tokens; the nextToken method (I-§2.22.13) returns TT_EOL (I-§2.22.5) and also sets the ttype field (I-§2.22.3) to this value when an end of line is read.

A line is a sequence of characters ending with either a carriage return character ('\r') or a newline character ('\n'). In addition, a carriage return character followed immediately by a newline character is treated as a single end- of-line token.

If the flag is false, end of line characters are treated as whitespace and serve only to separate tokens.

Parameters:

flag - true indicates that end-of-line characters are separate tokens; false indicates that end-of-line characters are whitespace

Returns:

the current line number of this stream tokenizer

If the flag argument is true, then the value in the sval field is lowercased whenever a word token is returned (the ttype field (I-§2.22.3) has the value TT_WORD (I-§2.22.7)) by the nextToken method (I-§2.22.13) of this tokenizer.

If the flag argument is false, then the sval field is not modified.

Parameters:

fl - true indicates that all word tokens should be lowercased

Parses the next token from the input stream of this tokenizer. The type of the next token is returned in the ttype field (I-§2.22.3). Additional information about the token may be in the nval field (I-§2.22.1) or the sval field (I-§2.22.2) of this tokenizer.

Returns:

The value of the ttype field (I-§2.22.3).

Throws

IOException (I-§2.29)

If an I/O error occurs.

Specifies that the character argument is "ordinary" in this tokenizer. It removes any special significance the character has as a comment character, word component, string delimiter, whitespace, or number character. When such a character is encountered by the parser, the parser treates it as a single-character token and sets ttype field (I-§2.22.3) to the character value.

Parameters:

ch - the character

Specifies that all characters c in the range are "ordinary" in this tokenizer. See the ordinaryChar method (I-§2.22.14) for more information on a character being ordinary.

Parameters:

low - the low end of the range

hi - the high end of the range

Specifies that numbers should be parsed by this tokenizer. The syntax table of this tokenizer is modified so that each of the twelve characters

0 1 2 3 4 5 6 7 8 9 . -

has the "numeric" attribute.

When the parser encounters a word token that has the format of a double precision floating point number, it treates the token as a number rather than a word, by setting the the ttype field (I-§2.22.3) to the value TT_NUMBER (I-§2.22.6) and putting the numeric value of the token into the nval field (I-§2.22.1).

Causes the next call to the nextToken method (I-§2.22.13) of this tokenizer to return the current value in the ttype field (I-§2.22.3), and not to modify the value in the nval (I-§2.22.1) or sval (I-§2.22.2) field.

Specifies that matching pairs of this character delimit string constants in this tokenizer.

When the nextToken method (I-§2.22.13) encounters a string constant, the ttype field (I-§2.22.3) field is set to the string delimiter and the sval field (I-§2.22.2) is set to the body of the string.

If a string quote character is encountered, then a string is recognized, consisting of all characters after (but not including) the string quote character, up to (but not including) the next occurrence of that same string quote character, or a line terminator, or end of file. The usual escape sequences such as "\n" and "\t" are recognized and converted to single characters as the string is parsed.

Parameters:

ch - the character

Resets this tokenizer's syntax table so that all characters are "ordinary." See the ordinaryChar method (I-§2.22.14) for more information on a character being ordinary.

If the flag argument is true, this stream tokenizer recognizes C++ style comments. Any occurrence of two consecutive slash characters ('/') is treated as the beginning of a comment that extends to the end of the line.

If the flag argument is false, then C++ style comments are not treated specially.

Parameters:

flag - true indicates to recognize and ignore C++ style comments

If the flag argument is true, this stream tokenizer recognizes C style comments. All text between successive occurrences of /* and */ are discarded.

If the flag argument is false, then C style comments are not treated specially.

Parameters:

flag - true indicates to recognize and ignore C style comments

Returns the string representation of the current stream token.

Returns:

a string representation of the token specified by the ttype (I-§2.22.3), nval (I-§2.22.1), and sval (I-§2.22.2) fields.

Overrides:

toString in class Object (I-§1.12.9).

Specifies that all characters c in the range are whitespace character. Whitepsace characters serve only to separate tokens in the input stream.

Parameters:

low - the low end of the range

hi - the high end of the range

Specifies that all characters c in the range are word constituents. A word token consists of a word constitutent followed by zero or more word constituents or number constituents.

Parameters:

low - the low end of the range

hi - the high end of the range

Packages  This Package  Prev  Next  Index