public class KeywordTokenizer extends Tokenizer
Constructor and Description |
---|
KeywordTokenizer(Reader input) |
KeywordTokenizer(Reader input,
int bufferSize) |
public KeywordTokenizer(Reader input)
public KeywordTokenizer(Reader input, int bufferSize)
public Token next(Token reusableToken) throws IOException
TokenStream
This implicitly defines a "contract" between consumers (callers of this method) and producers (implementations of this method that are the source for tokens):
Token.clear()
before setting the fields in it & returning itTokenFilter
is considered a consumer.next
in class TokenStream
reusableToken
- a Token that may or may not be used to
return; this parameter should never be null (the callee
is not required to check for null before using it, but it is a
good idea to assert that it is not null.)IOException
public void reset(Reader input) throws IOException
Tokenizer
reset
in class Tokenizer
IOException
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.