Examples of UniqueTokenFilter

org.apache.lucene.analysis.miscellaneous.UniqueTokenFilter
A token filter that generates unique tokens. Can remove unique tokens only on the same position increments as well.

Examples of org.apache.lucene.analysis.miscellaneous.UniqueTokenFilter

            @Override public String name() {
                return "unique";
            }


            @Override public TokenStream create(TokenStream tokenStream) {
                return new UniqueTokenFilter(tokenStream);
            }
        }));


        tokenFilterFactories.put("truncate", new PreBuiltTokenFilterFactoryFactory(new TokenFilterFactory() {
            @Override public String name() {

View Full Code Here

Examples of org.apache.lucene.analysis.miscellaneous.UniqueTokenFilter

        super(index, indexSettings, name, settings);
        this.onlyOnSamePosition = settings.getAsBoolean("only_on_same_position", false);
    }


    @Override public TokenStream create(TokenStream tokenStream) {
        return new UniqueTokenFilter(tokenStream, onlyOnSamePosition);
    }

View Full Code Here

Examples of org.apache.lucene.analysis.miscellaneous.UniqueTokenFilter

            lastTokenStreams.set(tempTokenStreams_local);
            tempTokenStreams.set(lastTokenStreams_local);
            // New ComboTokenStream to use
            lastComboTokenStream_local.setTokenStream(new ComboTokenStream(tempTokenStreams_local));
            if (deduplication)
                lastComboTokenStream_local.setTokenStream(new UniqueTokenFilter(lastComboTokenStream_local.getTokenStream(), true));
            lastComboTokenStream.set(lastComboTokenStream_local);
        }
        return lastComboTokenStream_local;
    }

View Full Code Here

Examples of org.apache.lucene.analysis.miscellaneous.UniqueTokenFilter

        this.onlyOnSamePosition = settings.getAsBoolean("only_on_same_position", false);
    }


    @Override
    public TokenStream create(TokenStream tokenStream) {
        return new UniqueTokenFilter(tokenStream, onlyOnSamePosition);
    }

View Full Code Here

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.