public class SimpleSpanFragmenter extends Object implements Fragmenter
Fragmenter
implementation which breaks text up into same-size
fragments but does not split up Spans. This is a simple sample class.Constructor and Description |
---|
SimpleSpanFragmenter(SpanScorer spanscorer) |
SimpleSpanFragmenter(SpanScorer spanscorer,
int fragmentSize) |
Modifier and Type | Method and Description |
---|---|
boolean |
isNewFragment(Token token)
Test to see if this token from the stream should be held in a new TextFragment
|
void |
start(String originalText)
Initializes the Fragmenter
|
public SimpleSpanFragmenter(SpanScorer spanscorer)
spanscorer
- SpanScorer that was used to score hitspublic SimpleSpanFragmenter(SpanScorer spanscorer, int fragmentSize)
spanscorer
- SpanScorer that was used to score hitsfragmentSize
- size in bytes of each fragmentpublic boolean isNewFragment(Token token)
Fragmenter
isNewFragment
in interface Fragmenter
public void start(String originalText)
Fragmenter
start
in interface Fragmenter
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.