public class NGramTokenFilter extends TokenFilter
Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_MAX_NGRAM_SIZE |
static int |
DEFAULT_MIN_NGRAM_SIZE |
input
Constructor and Description |
---|
NGramTokenFilter(TokenStream input)
Creates NGramTokenFilter with default min and max n-grams.
|
NGramTokenFilter(TokenStream input,
int minGram,
int maxGram)
Creates NGramTokenFilter with given min and max n-grams.
|
Modifier and Type | Method and Description |
---|---|
Token |
next(Token reusableToken)
Returns the next token in the stream, or null at EOS.
|
close, reset
next
public static final int DEFAULT_MIN_NGRAM_SIZE
public static final int DEFAULT_MAX_NGRAM_SIZE
public NGramTokenFilter(TokenStream input, int minGram, int maxGram)
input
- TokenStream holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic NGramTokenFilter(TokenStream input)
input
- TokenStream holding the input to be tokenizedpublic final Token next(Token reusableToken) throws IOException
next
in class TokenStream
IOException
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.