Class CompoundWordTokenFilterBase

Base class for decomposition token filters.

You must specify the required LuceneVersion compatibility when creating CompoundWordTokenFilterBase:

As of 3.1, CompoundWordTokenFilterBase correctly handles Unicode 4.0 supplementary characters in strings and char arrays provided as compound word dictionaries.
As of 4.4, CompoundWordTokenFilterBase doesn't update offsets.

Inheritance

System.Object

AttributeSource

TokenStream

TokenFilter

CompoundWordTokenFilterBase

DictionaryCompoundWordTokenFilter

HyphenationCompoundWordTokenFilter

Implements

System.IDisposable

Inherited Members

TokenFilter.m_input

TokenFilter.End()

TokenFilter.Dispose(Boolean)

TokenStream.Dispose()

AttributeSource.GetAttributeFactory()

AttributeSource.GetAttributeClassesEnumerator()

AttributeSource.GetAttributeImplsEnumerator()

AttributeSource.AddAttributeImpl(Attribute)

AttributeSource.AddAttribute<T>()

AttributeSource.HasAttributes

AttributeSource.HasAttribute<T>()

AttributeSource.GetAttribute<T>()

AttributeSource.ClearAttributes()

AttributeSource.CaptureState()

AttributeSource.RestoreState(AttributeSource.State)

AttributeSource.GetHashCode()

AttributeSource.Equals(Object)

AttributeSource.ReflectAsString(Boolean)

AttributeSource.ReflectWith(IAttributeReflector)

AttributeSource.CloneAttributes()

AttributeSource.CopyTo(AttributeSource)

AttributeSource.ToString()

System.Object.Equals(System.Object, System.Object)

System.Object.GetType()

System.Object.MemberwiseClone()

System.Object.ReferenceEquals(System.Object, System.Object)

Namespace: Lucene.Net.Analysis.Compound

Assembly: Lucene.Net.Analysis.Common.dll

Syntax

public abstract class CompoundWordTokenFilterBase : TokenFilter, IDisposable

Constructors

| Improve this Doc View Source

CompoundWordTokenFilterBase(LuceneVersion, TokenStream, CharArraySet)

Declaration

protected CompoundWordTokenFilterBase(LuceneVersion matchVersion, TokenStream input, CharArraySet dictionary)

Parameters

Type	Name	Description
LuceneVersion	matchVersion
TokenStream	input
CharArraySet	dictionary

| Improve this Doc View Source

CompoundWordTokenFilterBase(LuceneVersion, TokenStream, CharArraySet, Boolean)

Declaration

protected CompoundWordTokenFilterBase(LuceneVersion matchVersion, TokenStream input, CharArraySet dictionary, bool onlyLongestMatch)

Parameters

Type	Name	Description
LuceneVersion	matchVersion
TokenStream	input
CharArraySet	dictionary
System.Boolean	onlyLongestMatch

| Improve this Doc View Source

CompoundWordTokenFilterBase(LuceneVersion, TokenStream, CharArraySet, Int32, Int32, Int32, Boolean)

Declaration

protected CompoundWordTokenFilterBase(LuceneVersion matchVersion, TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, bool onlyLongestMatch)

Parameters

Type	Name	Description
LuceneVersion	matchVersion
TokenStream	input
CharArraySet	dictionary
System.Int32	minWordSize
System.Int32	minSubwordSize
System.Int32	maxSubwordSize
System.Boolean	onlyLongestMatch

Fields

| Improve this Doc View Source

DEFAULT_MAX_SUBWORD_SIZE

The default for maximal length of subwords that get propagated to the output of this filter

Declaration

public const int DEFAULT_MAX_SUBWORD_SIZE = 15

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

DEFAULT_MIN_SUBWORD_SIZE

The default for minimal length of subwords that get propagated to the output of this filter

Declaration

public const int DEFAULT_MIN_SUBWORD_SIZE = 2

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

DEFAULT_MIN_WORD_SIZE

The default for minimal word length that gets decomposed

Declaration

public const int DEFAULT_MIN_WORD_SIZE = 5

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

m_dictionary

Declaration

protected readonly CharArraySet m_dictionary

Field Value

Type	Description
CharArraySet

| Improve this Doc View Source

m_matchVersion

Declaration

protected readonly LuceneVersion m_matchVersion

Field Value

Type	Description
LuceneVersion

| Improve this Doc View Source

m_maxSubwordSize

Declaration

protected readonly int m_maxSubwordSize

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

m_minSubwordSize

Declaration

protected readonly int m_minSubwordSize

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

m_minWordSize

Declaration

protected readonly int m_minWordSize

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

m_offsetAtt

Declaration

protected readonly IOffsetAttribute m_offsetAtt

Field Value

Type	Description
IOffsetAttribute

| Improve this Doc View Source

m_onlyLongestMatch

Declaration

protected readonly bool m_onlyLongestMatch

Field Value

Type	Description
System.Boolean

| Improve this Doc View Source

m_termAtt

Declaration

protected readonly ICharTermAttribute m_termAtt

Field Value

Type	Description
ICharTermAttribute

| Improve this Doc View Source

m_tokens

Declaration

protected readonly LinkedList<CompoundWordTokenFilterBase.CompoundToken> m_tokens

Field Value

Type	Description
System.Collections.Generic.LinkedList<CompoundWordTokenFilterBase.CompoundToken>

Methods

| Improve this Doc View Source

Decompose()

Decomposes the current m_termAtt and places CompoundWordTokenFilterBase.CompoundToken instances in the m_tokens list. The original token may not be placed in the list, as it is automatically passed through this filter.

Declaration

protected abstract void Decompose()

| Improve this Doc View Source

IncrementToken()

Declaration

public override sealed bool IncrementToken()

Returns

Type	Description
System.Boolean

Overrides

TokenStream.IncrementToken()

| Improve this Doc View Source

Reset()

Declaration

public override void Reset()

Overrides

TokenFilter.Reset()

Implements

System.IDisposable