public class SinkTokenizer extends Tokenizer
TeeTokenFilter
Modifier and Type | Field and Description |
---|---|
protected java.util.Iterator |
iter |
protected java.util.List |
lst |
Constructor and Description |
---|
SinkTokenizer() |
SinkTokenizer(int initCap) |
SinkTokenizer(java.util.List input) |
Modifier and Type | Method and Description |
---|---|
void |
add(Token t)
Override this method to cache only certain tokens, or new tokens based
on the old tokens.
|
void |
close()
By default, closes the input Reader.
|
java.util.List |
getTokens()
Get the tokens in the internal List.
|
Token |
next(Token reusableToken)
Returns the next token out of the list of cached tokens
|
void |
reset()
Reset the internal data structures to the start at the front of the list of tokens.
|
next
public SinkTokenizer(java.util.List input)
public SinkTokenizer()
public SinkTokenizer(int initCap)
public java.util.List getTokens()
reset()
method to be called in order for them
to be made available. Also, this Tokenizer does nothing to protect against ConcurrentModificationException
s
in the case of adds happening while next(org.apache.lucene.analysis.Token)
is being called.
WARNING: Since this SinkTokenizer can be reset and the cached tokens made available again, do not modify them. Modify clones instead.Token
spublic Token next(Token reusableToken) throws java.io.IOException
next
in class TokenStream
reusableToken
- a Token that may or may not be used to
return; this parameter should never be null (the callee
is not required to check for null before using it, but it is a
good idea to assert that it is not null.)Token
in the Sink.java.io.IOException
public void add(Token t)
t
- The Token
to add to the sinkpublic void close() throws java.io.IOException
Tokenizer
public void reset() throws java.io.IOException
next(Token)
reset
in class TokenStream
java.io.IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.