public abstract class TokenStream
extends java.lang.Object
This is an abstract class. Concrete subclasses are:
Tokenizer
, a TokenStream
whose input is a Reader; and
TokenFilter
, a TokenStream
whose input is another TokenStream.
next(Token)
. It's
also OK to instead override next()
but that
method is now deprecated in favor of next(Token)
.Constructor and Description |
---|
TokenStream() |
Modifier and Type | Method and Description |
---|---|
void |
close()
Releases resources associated with this stream.
|
Token |
next()
Deprecated.
The returned Token is a "full private copy" (not
re-used across calls to next()) but will be slower
than calling
next(Token) instead.. |
Token |
next(Token reusableToken)
Returns the next token in the stream, or null at EOS.
|
void |
reset()
Resets this stream to the beginning.
|
public Token next() throws java.io.IOException
next(Token)
instead..java.io.IOException
public Token next(Token reusableToken) throws java.io.IOException
This implicitly defines a "contract" between consumers (callers of this method) and producers (implementations of this method that are the source for tokens):
Token.clear()
before setting the fields in it & returning itTokenFilter
is considered a consumer.reusableToken
- a Token that may or may not be used to
return; this parameter should never be null (the callee
is not required to check for null before using it, but it is a
good idea to assert that it is not null.)java.io.IOException
public void reset() throws java.io.IOException
java.io.IOException
public void close() throws java.io.IOException
java.io.IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.