List of reusableTokenStream() Examples

Examples of reusableTokenStream()

org.apache.lucene.analysis.Analyzer.reusableTokenStream()
Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. Callers that do not need to use more than one TokenStream at the same time from this analyzer should use this method for better performance.
org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()
This method uses {@link #createComponents(String,Reader)} to obtain aninstance of {@link TokenStreamComponents}. It returns the sink of the components and stores the components internally. Subsequent calls to this method will reuse the previously stored components if and only if the {@link TokenStreamComponents#reset(Reader)} method returnedtrue. Otherwise a new instance of {@link TokenStreamComponents} is created. @param fieldName the name of the field the created TokenStream is used for @param reader the reader the streams source reads from

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

          stringReader.init(stringValue);
          reader = stringReader;
        }
        
        // Tokenize field
        stream = analyzer.reusableTokenStream(field.name(), reader);
      }


      // reset the TokenStream to the first token
      stream.reset();

View Full Code Here

Examples of org.apache.lucene.analysis.Analyzer.reusableTokenStream()

          stringReader.init(stringValue);
          reader = stringReader;
        }
        
        // Tokenize field
        stream = analyzer.reusableTokenStream(field.name(), reader);
      }


      // reset the TokenStream to the first token
      stream.reset();

View Full Code Here

Examples of org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()

                Tokenizer t = new WhitespaceTokenizer(Lucene.VERSION, reader);
                return new TokenStreamComponents(t, new UniqueTokenFilter(t));
            }
        };


        TokenStream test = analyzer.reusableTokenStream("test", new StringReader("this test with test"));
        CharTermAttribute termAttribute = test.addAttribute(CharTermAttribute.class);
        assertThat(test.incrementToken(), equalTo(true));
        assertThat(termAttribute.toString(), equalTo("this"));


        assertThat(test.incrementToken(), equalTo(true));

View Full Code Here

Examples of org.apache.lucene.analysis.ReusableAnalyzerBase.reusableTokenStream()

                Tokenizer t = new WhitespaceTokenizer(Lucene.VERSION, reader);
                return new TokenStreamComponents(t, new TruncateTokenFilter(t, 3));
            }
        };


        TokenStream test = analyzer.reusableTokenStream("test", new StringReader("a bb ccc dddd eeeee"));
        CharTermAttribute termAttribute = test.addAttribute(CharTermAttribute.class);
        assertThat(test.incrementToken(), equalTo(true));
        assertThat(termAttribute.toString(), equalTo("a"));


        assertThat(test.incrementToken(), equalTo(true));

View Full Code Here

0 1 2

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.