Examples of PorterStemFilter

org.apache.lucene.analysis.PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm. Note: the input to the stemming filter must already be in lower case, so you will need to use LowerCaseFilter or LowerCaseTokenizer farther down the Tokenizer chain in order for this to work properly!
To use this filter with other analyzers, you'll want to write an Analyzer class that sets up the TokenStream chain as you want it. To use this with LowerCaseTokenizer, for example, you'd write an analyzer like this:
```
 class MyAnalyzer extends Analyzer { public final TokenStream tokenStream(String fieldName, Reader reader) { return new PorterStemFilter(new LowerCaseTokenizer(reader)); } } 
```
org.apache.lucene.analysis.en.PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm. Note: the input to the stemming filter must already be in lower case, so you will need to use LowerCaseFilter or LowerCaseTokenizer farther down the Tokenizer chain in order for this to work properly!
To use this filter with other analyzers, you'll want to write an Analyzer class that sets up the TokenStream chain as you want it. To use this with LowerCaseTokenizer, for example, you'd write an analyzer like this:
```
 class MyAnalyzer extends Analyzer { {@literal @Override}protected TokenStreamComponents createComponents(String fieldName, Reader reader) { Tokenizer source = new LowerCaseTokenizer(version, reader); return new TokenStreamComponents(source, new PorterStemFilter(source)); } } 
```
Note: This filter is aware of the {@link KeywordAttribute}. To prevent certain terms from being passed to the stemmer {@link KeywordAttribute#isKeyword()} should be set to truein a previous {@link TokenStream}.

Examples of org.apache.lucene.analysis.PorterStemFilter

    args.put("dictionary", "stemdict.txt");
    args.put("ignoreCase", "true");
    factory.init(args);
    factory.inform(loader);
    
    TokenStream ts = new PorterStemFilter(factory.create(tokenizer));
    assertTokenStreamContents(ts, new String[] { "test", "cat" });
  }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

    ResourceLoader loader = new SolrResourceLoader(null, null);
    args.put("protected", "protwords.txt");
    factory.init(args);
    factory.inform(loader);
    
    TokenStream ts = new PorterStemFilter(factory.create(tokenizer));
    assertTokenStreamContents(ts, new String[] { "dog", "cats" });
  }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

    args.put("protected", "protwords.txt");
    args.put("ignoreCase", "true");
    factory.init(args);
    factory.inform(loader);
    
    TokenStream ts = new PorterStemFilter(factory.create(tokenizer));
    assertTokenStreamContents(ts, new String[] { "dog", "cats", "Cats" });
  }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

/**
 * @version $Id: PorterStemFilterFactory.java 591158 2007-11-01 22:37:42Z hossman $
 */
public class PorterStemFilterFactory extends BaseTokenFilterFactory {
  public PorterStemFilter create(TokenStream input) {
    return new PorterStemFilter(input);
  }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

        } else if ("lovins".equalsIgnoreCase(language)) {
            return new SnowballFilter(tokenStream, new LovinsStemmer());
        } else if ("norwegian".equalsIgnoreCase(language)) {
            return new SnowballFilter(tokenStream, new NorwegianStemmer());
        } else if ("porter".equalsIgnoreCase(language)) {
            return new PorterStemFilter(tokenStream);
        } else if ("porter2".equalsIgnoreCase(language)) {
            return new SnowballFilter(tokenStream, new PorterStemmer());
        } else if ("portuguese".equalsIgnoreCase(language)) {
            return new SnowballFilter(tokenStream, new PortugueseStemmer());
        } else if ("romanian".equalsIgnoreCase(language)) {

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

    @Inject public PorterStemTokenFilterFactory(Index index, @IndexSettings Settings indexSettings, @Assisted String name, @Assisted Settings settings) {
        super(index, indexSettings, name, settings);
    }


    @Override public TokenStream create(TokenStream tokenStream) {
        return new PorterStemFilter(tokenStream);
    }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

        @Override
    public final TokenStream tokenStream(String fieldName, Reader reader) {
            // Apply stop words and porter stemmer using a lower-case tokenizer.
            TokenStream stream = new StopFilter(new LowerCaseTokenizer(reader),
                StandardAnalyzer.STOP_WORDS);
            return new PorterStemFilter(stream);
        }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

    TokenStream result = new StandardFilter(Version.LUCENE_31, tokenizer);
    result = new LowerCaseFilter(Version.LUCENE_31, result);
    result = new ASCIIFoldingFilter(result);
    result = new AlphaNumericMaxLengthFilter(result);
    result = new StopFilter(Version.LUCENE_31, result, stopwords);
    result = new PorterStemFilter(result);
    return new TokenStreamComponents(tokenizer, result);
  }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

    tokenStream.setMaxTokenLength(maxTokenLength);
    TokenStream result = new StandardFilter(tokenStream);


    result = new LowerCaseFilter(result);
    result = new StopFilter(result, TERRIER_STOP_WORDS);
    result = new PorterStemFilter(result);


    return result;
  }

View Full Code Here

Examples of org.apache.lucene.analysis.PorterStemFilter

    if (child == null) 
      throw new IllegalArgumentException("child analyzer must not be null");
  
    return new Analyzer() {
      public TokenStream tokenStream(String fieldName, Reader reader) {
        return new PorterStemFilter(
            child.tokenStream(fieldName, reader));
//         /* PorterStemFilter and SnowballFilter have the same behaviour, 
//         but PorterStemFilter is much faster. */
//        return new org.apache.lucene.analysis.snowball.SnowballFilter(
//            child.tokenStream(fieldName, reader), "English");

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.