Class LookaheadTokenFilter<T>

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Inheritance

object

AttributeSource

TokenStream

TokenFilter

LookaheadTokenFilter

LookaheadTokenFilter<T>

MockGraphTokenFilter

MockRandomLookaheadTokenFilter

Implements

IDisposable

Inherited Members

LookaheadTokenFilter.IncrementToken()

TokenFilter.m_input

TokenFilter.End()

TokenFilter.Dispose(bool)

TokenStream.Dispose()

AttributeSource.GetAttributeFactory()

AttributeSource.GetAttributeClassesEnumerator()

AttributeSource.GetAttributeImplsEnumerator()

AttributeSource.AddAttributeImpl(Attribute)

AttributeSource.AddAttribute<T>()

AttributeSource.HasAttributes

AttributeSource.HasAttribute<T>()

AttributeSource.GetAttribute<T>()

AttributeSource.ClearAttributes()

AttributeSource.CaptureState()

AttributeSource.RestoreState(AttributeSource.State)

AttributeSource.GetHashCode()

AttributeSource.Equals(object)

AttributeSource.ReflectAsString(bool)

AttributeSource.ReflectWith(IAttributeReflector)

AttributeSource.CloneAttributes()

AttributeSource.CopyTo(AttributeSource)

AttributeSource.ToString()

object.Equals(object, object)

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Namespace: Lucene.Net.Analysis

Assembly: Lucene.Net.TestFramework.dll

Syntax

public abstract class LookaheadTokenFilter<T> : LookaheadTokenFilter, IDisposable where T : LookaheadTokenFilter.Position

Type Parameters

Name	Description
T

Constructors

LookaheadTokenFilter(TokenStream, IRollingBufferItemFactory<T>)

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected LookaheadTokenFilter(TokenStream input, IRollingBufferItemFactory<T> itemFactory)

Parameters

Type	Name	Description
TokenStream	input
IRollingBufferItemFactory<T>	itemFactory

Fields

DEBUG

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected static readonly bool DEBUG

Field Value

Type	Description
bool

m_end

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected bool m_end

Field Value

Type	Description
bool

m_inputPos

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected int m_inputPos

Field Value

Type	Description
int

m_offsetAtt

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected readonly IOffsetAttribute m_offsetAtt

Field Value

Type	Description
IOffsetAttribute

m_outputPos

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected int m_outputPos

Field Value

Type	Description
int

m_posIncAtt

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected readonly IPositionIncrementAttribute m_posIncAtt

Field Value

Type	Description
IPositionIncrementAttribute

m_posLenAtt

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected readonly IPositionLengthAttribute m_posLenAtt

Field Value

Type	Description
IPositionLengthAttribute

m_positions

An abstract Lucene.Net.Analysis.TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.

Declaration

protected readonly RollingBuffer<T> m_positions

Field Value

Type	Description
RollingBuffer<T>

Methods

AfterPosition()

This is called when all input tokens leaving a given position have been returned. Override this and call insertToken and then set whichever token's attributes you want, if you want to inject a token starting from this position.

Declaration

protected virtual void AfterPosition()

InsertToken()

Call this only from within AfterPosition(), to insert a new token. After calling this you should set any necessary token you need.

Declaration

protected virtual void InsertToken()

NextToken()

Call this when you are done looking ahead; it will set the next token to return. Return the boolean back to the caller.

Declaration

protected virtual bool NextToken()

Returns

Type	Description
bool

PeekToken()

Returns true if there is a new token.

Declaration

protected virtual bool PeekToken()

Returns

Type	Description
bool

Reset()

This method is called by a consumer before it begins consumption using Lucene.Net.Analysis.TokenStream.IncrementToken().

Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.

If you override this method, always call base.Reset(), otherwise some internal state will not be correctly reset (e.g., Lucene.Net.Analysis.Tokenizer will throw InvalidOperationException on further usage).

Declaration

public override void Reset()

Overrides

Lucene.Net.Analysis.TokenFilter.Reset()

Remarks

NOTE: The default implementation chains the call to the input Lucene.Net.Analysis.TokenStream, so be sure to call base.Reset() when overriding this method.

Implements

IDisposable