Class TokenFilter

A TokenFilter is a TokenStream whose input is another TokenStream.

This is an abstract class; subclasses must override IncrementToken().

Inheritance
System.Object
AttributeSource
TokenStream
TokenFilter
CachingTokenFilter
Implements
IDisposable
Inherited Members
Namespace: Lucene.Net.Analysis
Assembly: Lucene.Net.dll
Syntax
public abstract class TokenFilter : TokenStream, IDisposable

Constructors

| Improve this Doc View Source

TokenFilter(TokenStream)

Construct a token stream filtering the given input.

Declaration
protected TokenFilter(TokenStream input)
Parameters
Type Name Description
TokenStream input

Fields

| Improve this Doc View Source

m_input

The source of tokens for this filter.

Declaration
protected readonly TokenStream m_input
Field Value
Type Description
TokenStream

Methods

| Improve this Doc View Source

Dispose(Boolean)

Releases resources associated with this stream.

If you override this method, always call base.Dispose(disposing), otherwise some internal state will not be correctly reset (e.g., Tokenizer will throw on reuse).

NOTE: The default implementation chains the call to the input TokenStream, so be sure to call base.Dispose(disposing) when overriding this method.

Declaration
protected override void Dispose(bool disposing)
Parameters
Type Name Description
System.Boolean disposing
Overrides
| Improve this Doc View Source

End()

This method is called by the consumer after the last token has been consumed, after IncrementToken() returned false (using the new TokenStream API). Streams implementing the old API should upgrade to use this feature.

This method can be used to perform any end-of-stream operations, such as setting the final offset of a stream. The final offset of a stream might differ from the offset of the last token eg in case one or more whitespaces followed after the last token, but a WhitespaceTokenizer was used.

Additionally any skipped positions (such as those removed by a stopfilter) can be applied to the position increment, or any adjustment of other attributes where the end-of-stream value may be important.

NOTE: The default implementation chains the call to the input TokenStream, so be sure to call base.End() first when overriding this method.

Declaration
public override void End()
Overrides
| Improve this Doc View Source

Reset()

This method is called by a consumer before it begins consumption using IncrementToken().

Resets this stream to a clean state. Stateful implementations must implement this method so that they can be reused, just as if they had been created fresh.

If you override this method, always call base.Reset(), otherwise some internal state will not be correctly reset (e.g., Tokenizer will throw on further usage).

Declaration
public override void Reset()
Overrides
Remarks

NOTE: The default implementation chains the call to the input TokenStream, so be sure to call base.Reset() when overriding this method.

Implements

IDisposable

See Also