Class Lucene43EdgeNGramTokenizer

Old version of EdgeNGramTokenizer which doesn't handle correctly supplementary characters.

Inheritance
object
AttributeSource
TokenStream
Tokenizer
Lucene43EdgeNGramTokenizer
Implements
Inherited Members
TokenStream.Dispose()
AttributeSource.GetAttributeFactory()
AttributeSource.GetAttributeClassesEnumerator()
AttributeSource.GetAttributeImplsEnumerator()
AttributeSource.AddAttributeImpl(Attribute)
AttributeSource.AddAttribute<T>()
AttributeSource.HasAttributes
AttributeSource.HasAttribute<T>()
AttributeSource.GetAttribute<T>()
AttributeSource.ClearAttributes()
AttributeSource.CaptureState()
AttributeSource.RestoreState(AttributeSource.State)
AttributeSource.GetHashCode()
AttributeSource.ReflectWith(IAttributeReflector)
AttributeSource.CloneAttributes()
AttributeSource.CopyTo(AttributeSource)
AttributeSource.ToString()
Namespace: Lucene.Net.Analysis.NGram
Assembly: Lucene.Net.Analysis.Common.dll
Syntax
[Obsolete]
public sealed class Lucene43EdgeNGramTokenizer : Tokenizer, IDisposable

Constructors

Lucene43EdgeNGramTokenizer(LuceneVersion, AttributeFactory, TextReader, Side, int, int)

Creates Lucene43EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Declaration
[Obsolete]
public Lucene43EdgeNGramTokenizer(LuceneVersion version, AttributeSource.AttributeFactory factory, TextReader input, Lucene43EdgeNGramTokenizer.Side side, int minGram, int maxGram)
Parameters
Type Name Description
LuceneVersion version

the Lucene match version - See Lucene.Net.Util.LuceneVersion
AttributeSource.AttributeFactory factory

Lucene.Net.Util.AttributeSource.AttributeFactory to use
TextReader input

TextReader holding the input to be tokenized
Lucene43EdgeNGramTokenizer.Side side

the Lucene43EdgeNGramTokenizer.Side from which to chop off an n-gram
int minGram

the smallest n-gram to generate
int maxGram

the largest n-gram to generate

Lucene43EdgeNGramTokenizer(LuceneVersion, AttributeFactory, TextReader, int, int)

Creates Lucene43EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Declaration
public Lucene43EdgeNGramTokenizer(LuceneVersion version, AttributeSource.AttributeFactory factory, TextReader input, int minGram, int maxGram)
Parameters
Type Name Description
LuceneVersion version

the Lucene match version - See Lucene.Net.Util.LuceneVersion
AttributeSource.AttributeFactory factory

Lucene.Net.Util.AttributeSource.AttributeFactory to use
TextReader input

TextReader holding the input to be tokenized
int minGram

the smallest n-gram to generate
int maxGram

the largest n-gram to generate

Lucene43EdgeNGramTokenizer(LuceneVersion, AttributeFactory, TextReader, string, int, int)

Creates Lucene43EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Declaration
[Obsolete]
public Lucene43EdgeNGramTokenizer(LuceneVersion version, AttributeSource.AttributeFactory factory, TextReader input, string sideLabel, int minGram, int maxGram)
Parameters
Type Name Description
LuceneVersion version

the Lucene match version - See Lucene.Net.Util.LuceneVersion
AttributeSource.AttributeFactory factory

Lucene.Net.Util.AttributeSource.AttributeFactory to use
TextReader input

TextReader holding the input to be tokenized
string sideLabel

the name of the Lucene43EdgeNGramTokenizer.Side from which to chop off an n-gram
int minGram

the smallest n-gram to generate
int maxGram

the largest n-gram to generate

Lucene43EdgeNGramTokenizer(LuceneVersion, TextReader, Side, int, int)

Creates Lucene43EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Declaration
[Obsolete]
public Lucene43EdgeNGramTokenizer(LuceneVersion version, TextReader input, Lucene43EdgeNGramTokenizer.Side side, int minGram, int maxGram)
Parameters
Type Name Description
LuceneVersion version

the Lucene match version - See Lucene.Net.Util.LuceneVersion
TextReader input

TextReader holding the input to be tokenized
Lucene43EdgeNGramTokenizer.Side side

the Lucene43EdgeNGramTokenizer.Side from which to chop off an n-gram
int minGram

the smallest n-gram to generate
int maxGram

the largest n-gram to generate

Lucene43EdgeNGramTokenizer(LuceneVersion, TextReader, int, int)

Creates Lucene43EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Declaration
public Lucene43EdgeNGramTokenizer(LuceneVersion version, TextReader input, int minGram, int maxGram)
Parameters
Type Name Description
LuceneVersion version

the Lucene match version - See Lucene.Net.Util.LuceneVersion
TextReader input

TextReader holding the input to be tokenized
int minGram

the smallest n-gram to generate
int maxGram

the largest n-gram to generate

Lucene43EdgeNGramTokenizer(LuceneVersion, TextReader, string, int, int)

Creates Lucene43EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Declaration
[Obsolete]
public Lucene43EdgeNGramTokenizer(LuceneVersion version, TextReader input, string sideLabel, int minGram, int maxGram)
Parameters
Type Name Description
LuceneVersion version

the Lucene match version - See Lucene.Net.Util.LuceneVersion
TextReader input

TextReader holding the input to be tokenized
string sideLabel

the name of the Lucene43EdgeNGramTokenizer.Side from which to chop off an n-gram
int minGram

the smallest n-gram to generate
int maxGram

the largest n-gram to generate

Fields

DEFAULT_MAX_GRAM_SIZE

Old version of EdgeNGramTokenizer which doesn't handle correctly supplementary characters.

Declaration
public const int DEFAULT_MAX_GRAM_SIZE = 1
Field Value
Type Description
int

DEFAULT_MIN_GRAM_SIZE

Old version of EdgeNGramTokenizer which doesn't handle correctly supplementary characters.

Declaration
public const int DEFAULT_MIN_GRAM_SIZE = 1
Field Value
Type Description
int

DEFAULT_SIDE

Old version of EdgeNGramTokenizer which doesn't handle correctly supplementary characters.

Declaration
public const Lucene43EdgeNGramTokenizer.Side DEFAULT_SIDE = FRONT
Field Value
Type Description
Lucene43EdgeNGramTokenizer.Side

Methods

End()

This method is called by the consumer after the last token has been consumed, after Lucene.Net.Analysis.TokenStream.IncrementToken() returned false (using the new Lucene.Net.Analysis.TokenStream API). Streams implementing the old API should upgrade to use this feature.

This method can be used to perform any end-of-stream operations, such as setting the final offset of a stream. The final offset of a stream might differ from the offset of the last token eg in case one or more whitespaces followed after the last token, but a WhitespaceTokenizer was used.

Additionally any skipped positions (such as those removed by a stopfilter) can be applied to the position increment, or any adjustment of other attributes where the end-of-stream value may be important.

If you override this method, always call base.End();.
Declaration
public override void End()
Overrides
Lucene.Net.Analysis.TokenStream.End()
Exceptions
Type Condition
IOException

If an I/O error occurs

GetSide(string)

Old version of EdgeNGramTokenizer which doesn't handle correctly supplementary characters.

Declaration
public static Lucene43EdgeNGramTokenizer.Side GetSide(string sideName)
Parameters
Type Name Description
string sideName
Returns
Type Description
Lucene43EdgeNGramTokenizer.Side

IncrementToken()

Returns the next token in the stream, or null at EOS.

Declaration
public override bool IncrementToken()
Returns
Type Description
bool
Overrides
Lucene.Net.Analysis.TokenStream.IncrementToken()

Reset()

This method is called by a consumer before it begins consumption using Lucene.Net.Analysis.TokenStream.IncrementToken().

Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.

If you override this method, always call base.Reset(), otherwise some internal state will not be correctly reset (e.g., Lucene.Net.Analysis.Tokenizer will throw InvalidOperationException on further usage).
Declaration
public override void Reset()
Overrides
Lucene.Net.Analysis.Tokenizer.Reset()

Implements