Namespace Lucene.Net.Analysis.Shingle

Word n-gram filters

Classes

ShingleAnalyzerWrapper

A ShingleAnalyzerWrapper wraps a ShingleFilter around another Lucene.Net.Analysis.Analyzer.

A shingle is another name for a token based n-gram.

ShingleFilter

A ShingleFilter constructs shingles (token n-grams) from a token stream. In other words, it creates combinations of tokens as a single token.

For example, the sentence "please divide this sentence into shingles" might be tokenized into shingles "please divide", "divide this", "this sentence", "sentence into", and "into shingles".

This filter handles position increments > 1 by inserting filler tokens (tokens with termtext "_"). It does not handle a position increment of 0.

ShingleFilterFactory

Factory for ShingleFilter.

<fieldType name="text_shingle" class="solr.TextField" positionIncrementGap="100">
  <analyzer>
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.ShingleFilterFactory" minShingleSize="2" maxShingleSize="2"
            outputUnigrams="true" outputUnigramsIfNoShingles="false" tokenSeparator=" " fillerToken="_"/>
  </analyzer>
</fieldType>