public class EdgeNGramTokenFilter extends TokenFilter
Modifier and Type | Class and Description |
---|---|
static class |
EdgeNGramTokenFilter.Side
Specifies which side of the input the n-gram should be generated from
|
Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_MAX_GRAM_SIZE |
static int |
DEFAULT_MIN_GRAM_SIZE |
static EdgeNGramTokenFilter.Side |
DEFAULT_SIDE |
input
Modifier | Constructor and Description |
---|---|
protected |
EdgeNGramTokenFilter(TokenStream input) |
|
EdgeNGramTokenFilter(TokenStream input,
EdgeNGramTokenFilter.Side side,
int minGram,
int maxGram)
Creates EdgeNGramTokenFilter that can generate n-grams in the sizes of the given range
|
|
EdgeNGramTokenFilter(TokenStream input,
String sideLabel,
int minGram,
int maxGram)
Creates EdgeNGramTokenFilter that can generate n-grams in the sizes of the given range
|
Modifier and Type | Method and Description |
---|---|
Token |
next(Token reusableToken)
Returns the next token in the stream, or null at EOS.
|
close, reset
next
public static final EdgeNGramTokenFilter.Side DEFAULT_SIDE
public static final int DEFAULT_MAX_GRAM_SIZE
public static final int DEFAULT_MIN_GRAM_SIZE
protected EdgeNGramTokenFilter(TokenStream input)
public EdgeNGramTokenFilter(TokenStream input, EdgeNGramTokenFilter.Side side, int minGram, int maxGram)
input
- TokenStream holding the input to be tokenizedside
- the EdgeNGramTokenFilter.Side
from which to chop off an n-gramminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic EdgeNGramTokenFilter(TokenStream input, String sideLabel, int minGram, int maxGram)
input
- TokenStream holding the input to be tokenizedsideLabel
- the name of the EdgeNGramTokenFilter.Side
from which to chop off an n-gramminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generatepublic final Token next(Token reusableToken) throws IOException
next
in class TokenStream
reusableToken
- a Token that may or may not be used to
return; this parameter should never be null (the callee
is not required to check for null before using it, but it is a
good idea to assert that it is not null.)IOException
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.