Class TokenFilter

A TokenFilter is a TokenStream whose input is another TokenStream.

This is an abstract class; subclasses must override IncrementToken().
Inheritance
object
AttributeSource
TokenStream
TokenFilter
CachingTokenFilter
Implements
Inherited Members
Namespace: Lucene.Net.Analysis
Assembly: Lucene.Net.dll
Syntax
public abstract class TokenFilter : TokenStream, IDisposable

Constructors

TokenFilter(TokenStream)

Construct a token stream filtering the given input.

Declaration
protected TokenFilter(TokenStream input)
Parameters
Type Name Description
TokenStream input
See Also

Fields

m_input

The source of tokens for this filter.

Declaration
protected readonly TokenStream m_input
Field Value
Type Description
TokenStream
See Also

Methods

Dispose(bool)

Releases resources associated with this stream.

If you override this method, always call base.Dispose(disposing), otherwise some internal state will not be correctly reset (e.g., Tokenizer will throw InvalidOperationException on reuse).

NOTE: The default implementation chains the call to the input TokenStream, so be sure to call base.Dispose(disposing) when overriding this method.
Declaration
protected override void Dispose(bool disposing)
Parameters
Type Name Description
bool disposing
Overrides
See Also

End()

This method is called by the consumer after the last token has been consumed, after IncrementToken() returned false (using the new TokenStream API). Streams implementing the old API should upgrade to use this feature.

This method can be used to perform any end-of-stream operations, such as setting the final offset of a stream. The final offset of a stream might differ from the offset of the last token eg in case one or more whitespaces followed after the last token, but a WhitespaceTokenizer was used.

Additionally any skipped positions (such as those removed by a stopfilter) can be applied to the position increment, or any adjustment of other attributes where the end-of-stream value may be important.

NOTE: The default implementation chains the call to the input TokenStream, so be sure to call base.End() first when overriding this method.
Declaration
public override void End()
Overrides
Exceptions
Type Condition
IOException

If an I/O error occurs
See Also

Reset()

This method is called by a consumer before it begins consumption using IncrementToken().

Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.

If you override this method, always call base.Reset(), otherwise some internal state will not be correctly reset (e.g., Tokenizer will throw InvalidOperationException on further usage).
Declaration
public override void Reset()
Overrides
Remarks

NOTE: The default implementation chains the call to the input TokenStream, so be sure to call base.Reset() when overriding this method.

See Also

Implements

See Also