public class DictionaryCompoundWordTokenFilter extends CompoundWordTokenFilterBase
DEFAULT_MAX_SUBWORD_SIZE, DEFAULT_MIN_SUBWORD_SIZE, DEFAULT_MIN_WORD_SIZE, dictionary, maxSubwordSize, minSubwordSize, minWordSize, onlyLongestMatch, tokens
input
Constructor and Description |
---|
DictionaryCompoundWordTokenFilter(TokenStream input,
java.util.Set dictionary) |
DictionaryCompoundWordTokenFilter(TokenStream input,
java.util.Set dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch) |
DictionaryCompoundWordTokenFilter(TokenStream input,
java.lang.String[] dictionary) |
DictionaryCompoundWordTokenFilter(TokenStream input,
java.lang.String[] dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch) |
Modifier and Type | Method and Description |
---|---|
protected void |
decomposeInternal(Token token) |
addAllLowerCase, createToken, decompose, makeDictionary, makeLowerCaseCopy, next
close, reset
next
public DictionaryCompoundWordTokenFilter(TokenStream input, java.lang.String[] dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)
input
- the token stream to processdictionary
- the word dictionary to match againstminWordSize
- only words longer than this get processedminSubwordSize
- only subwords longer than this get to the output streammaxSubwordSize
- only subwords shorter than this get to the output streamonlyLongestMatch
- Add only the longest matching subword to the streampublic DictionaryCompoundWordTokenFilter(TokenStream input, java.lang.String[] dictionary)
input
- the token stream to processdictionary
- the word dictionary to match againstpublic DictionaryCompoundWordTokenFilter(TokenStream input, java.util.Set dictionary)
input
- the token stream to processdictionary
- the word dictionary to match against. If this is a CharArraySet
it must have set ignoreCase=false and only contain
lower case strings.public DictionaryCompoundWordTokenFilter(TokenStream input, java.util.Set dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)
input
- the token stream to processdictionary
- the word dictionary to match against. If this is a CharArraySet
it must have set ignoreCase=false and only contain
lower case strings.minWordSize
- only words longer than this get processedminSubwordSize
- only subwords longer than this get to the output streammaxSubwordSize
- only subwords shorter than this get to the output streamonlyLongestMatch
- Add only the longest matching subword to the streamprotected void decomposeInternal(Token token)
decomposeInternal
in class CompoundWordTokenFilterBase
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.