Class ICUNormalizer2Filter

Normalize token text with ICU's ICU4N.Text.Normalizer2.

Inheritance
System.Object
Lucene.Net.Util.AttributeSource
Lucene.Net.Analysis.TokenStream
Lucene.Net.Analysis.TokenFilter
ICUNormalizer2Filter
ICUFoldingFilter
Implements
System.IDisposable
Inherited Members
Lucene.Net.Analysis.TokenFilter.m_input
Lucene.Net.Analysis.TokenFilter.End()
Lucene.Net.Analysis.TokenFilter.Reset()
Lucene.Net.Analysis.TokenStream.Dispose()
Lucene.Net.Util.AttributeSource.GetAttributeFactory()
Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()
Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()
Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)
Lucene.Net.Util.AttributeSource.AddAttribute<T>()
Lucene.Net.Util.AttributeSource.HasAttributes
Lucene.Net.Util.AttributeSource.HasAttribute<T>()
Lucene.Net.Util.AttributeSource.GetAttribute<T>()
Lucene.Net.Util.AttributeSource.ClearAttributes()
Lucene.Net.Util.AttributeSource.CaptureState()
Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)
Lucene.Net.Util.AttributeSource.GetHashCode()
Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)
Lucene.Net.Util.AttributeSource.CloneAttributes()
Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)
Lucene.Net.Util.AttributeSource.ToString()
System.Object.Equals(System.Object, System.Object)
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
Namespace: Lucene.Net.Analysis.Icu
Assembly: Lucene.Net.ICU.dll
Syntax
public class ICUNormalizer2Filter : TokenFilter, IDisposable
Remarks

With this filter, you can normalize text in the following ways:

  • NFKC Normalization, Case Folding, and removing Ignorables (the default)
  • Using a standard Normalization mode (NFC, NFD, NFKC, NFKD)
  • Based on rules from a custom normalization mapping.

If you use the defaults, this filter is a simple way to standardize Unicode text in a language-independent way for search:

  • The case folding that it does can be seen as a replacement for LowerCaseFilter: For example, it handles cases such as the Greek sigma, so that "Μάϊος" and "ΜΆΪΟΣ" will match correctly.
  • The normalization will standardizes different forms of the same character in Unicode. For example, CJK full-width numbers will be standardized to their ASCII forms.
  • Ignorables such as Zero-Width Joiner and Variation Selectors are removed. These are typically modifier characters that affect display.

Constructors

| Improve this Doc View Source

ICUNormalizer2Filter(TokenStream)

Create a new ICUNormalizer2Filter that combines NFKC normalization, Case Folding, and removes Default Ignorables (NFKC_Casefold)

Declaration
public ICUNormalizer2Filter(TokenStream input)
Parameters
Type Name Description
Lucene.Net.Analysis.TokenStream input
| Improve this Doc View Source

ICUNormalizer2Filter(TokenStream, Normalizer2)

Create a new ICUNormalizer2Filter with the specified ICU4N.Text.Normalizer2

Declaration
public ICUNormalizer2Filter(TokenStream input, Normalizer2 normalizer)
Parameters
Type Name Description
Lucene.Net.Analysis.TokenStream input

stream
ICU4N.Text.Normalizer2 normalizer

normalizer to use

Methods

| Improve this Doc View Source

IncrementToken()

Declaration
public sealed override bool IncrementToken()
Returns
Type Description
System.Boolean
Overrides
Lucene.Net.Analysis.TokenStream.IncrementToken()

Implements

System.IDisposable

See Also

ICU4N.Text.Normalizer2
ICU4N.Text.FilteredNormalizer2