Class CJKBigramFilter

Forms bigrams of CJK terms that are generated from StandardTokenizer or ICUTokenizer.

CJK types are set by these tokenizers, but you can also use CJKBigramFilter(TokenStream, CJKScript) to explicitly control which of the CJK scripts are turned into bigrams.

By default, when a CJK character has no adjacent characters to form a bigram, it is output in unigram form. If you want to always output both unigrams and bigrams, set the

outputUnigrams

flag in CJKBigramFilter(TokenStream, CJKScript, Boolean). This can be used for a combined unigram+bigram approach.

In all cases, all non-CJK input is passed thru unmodified.

Inheritance

System.Object

Lucene.Net.Util.AttributeSource

Lucene.Net.Analysis.TokenStream

Lucene.Net.Analysis.TokenFilter

CJKBigramFilter

Implements

System.IDisposable

Inherited Members

Lucene.Net.Analysis.TokenFilter.m_input

Lucene.Net.Analysis.TokenFilter.End()

TokenFilter.Dispose(Boolean)

Lucene.Net.Analysis.TokenStream.Dispose()

Lucene.Net.Util.AttributeSource.GetAttributeFactory()

Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()

Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()

Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)

Lucene.Net.Util.AttributeSource.AddAttribute<T>()

Lucene.Net.Util.AttributeSource.HasAttributes

Lucene.Net.Util.AttributeSource.HasAttribute<T>()

Lucene.Net.Util.AttributeSource.GetAttribute<T>()

Lucene.Net.Util.AttributeSource.ClearAttributes()

Lucene.Net.Util.AttributeSource.CaptureState()

Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)

Lucene.Net.Util.AttributeSource.GetHashCode()

AttributeSource.Equals(Object)

AttributeSource.ReflectAsString(Boolean)

Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)

Lucene.Net.Util.AttributeSource.CloneAttributes()

Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)

Lucene.Net.Util.AttributeSource.ToString()

System.Object.Equals(System.Object, System.Object)

System.Object.GetType()

System.Object.MemberwiseClone()

System.Object.ReferenceEquals(System.Object, System.Object)

Namespace: Lucene.Net.Analysis.Cjk

Assembly: Lucene.Net.Analysis.Common.dll

Syntax

public sealed class CJKBigramFilter : TokenFilter, IDisposable

Constructors

| Improve this Doc View Source

CJKBigramFilter(TokenStream)

Calls CJKBigramFilter(TokenStream, CJKScript)

Declaration

public CJKBigramFilter(TokenStream in)

Parameters

Type	Name	Description
Lucene.Net.Analysis.TokenStream	in	Input Lucene.Net.Analysis.TokenStream

| Improve this Doc View Source

CJKBigramFilter(TokenStream, CJKScript)

Calls CJKBigramFilter(TokenStream, CJKScript, Boolean)

Declaration

public CJKBigramFilter(TokenStream in, CJKScript flags)

Parameters

Type	Name	Description
Lucene.Net.Analysis.TokenStream	in	Input Lucene.Net.Analysis.TokenStream
CJKScript	flags	OR'ed set from HAN, HIRAGANA, KATAKANA, HANGUL

| Improve this Doc View Source

CJKBigramFilter(TokenStream, CJKScript, Boolean)

Create a new CJKBigramFilter, specifying which writing systems should be bigrammed, and whether or not unigrams should also be output.

Declaration

public CJKBigramFilter(TokenStream in, CJKScript flags, bool outputUnigrams)

Parameters

Type	Name	Description
Lucene.Net.Analysis.TokenStream	in	Input Lucene.Net.Analysis.TokenStream
CJKScript	flags	OR'ed set from HAN, HIRAGANA, KATAKANA, HANGUL
System.Boolean	outputUnigrams	true if unigrams for the selected writing systems should also be output. when this is false, this is only done when there are no adjacent characters to form a bigram.

Fields

| Improve this Doc View Source

DOUBLE_TYPE

when we emit a bigram, its then marked as this type

Declaration

public const string DOUBLE_TYPE = "<DOUBLE>"

Field Value

Type	Description
System.String

| Improve this Doc View Source

SINGLE_TYPE

when we emit a unigram, its then marked as this type

Declaration

public const string SINGLE_TYPE = "<SINGLE>"

Field Value

Type	Description
System.String

Methods

| Improve this Doc View Source

IncrementToken()

Declaration

public override bool IncrementToken()

Returns

Type	Description
System.Boolean

Overrides

Lucene.Net.Analysis.TokenStream.IncrementToken()

| Improve this Doc View Source

Reset()

Declaration

public override void Reset()

Overrides

Lucene.Net.Analysis.TokenFilter.Reset()

Implements

System.IDisposable