Class Token

Analyzed token with morphological data from its dictionary.

Inheritance

System.Object

Token

Inherited Members

System.Object.Equals(System.Object)

System.Object.Equals(System.Object, System.Object)

System.Object.GetHashCode()

System.Object.GetType()

System.Object.MemberwiseClone()

System.Object.ReferenceEquals(System.Object, System.Object)

Namespace: Lucene.Net.Analysis.Ja

Assembly: Lucene.Net.Analysis.Kuromoji.dll

Syntax

public class Token

Constructors

| Improve this Doc View Source

Token(Int32, Char[], Int32, Int32, JapaneseTokenizerType, Int32, IDictionary)

Declaration

public Token(int wordId, char[] surfaceForm, int offset, int length, JapaneseTokenizerType type, int position, IDictionary dictionary)

Parameters

Type	Name	Description
System.Int32	wordId
System.Char[]	surfaceForm
System.Int32	offset
System.Int32	length
JapaneseTokenizerType	type
System.Int32	position
IDictionary	dictionary

Properties

| Improve this Doc View Source

Length

length of surfaceForm

Declaration

public virtual int Length { get; }

Property Value

Type	Description
System.Int32

| Improve this Doc View Source

Offset

offset into surfaceForm

Declaration

public virtual int Offset { get; }

Property Value

Type	Description
System.Int32

| Improve this Doc View Source

Position

Get index of this token in input text. Returns position of token.

Declaration

public virtual int Position { get; }

Property Value

Type	Description
System.Int32

| Improve this Doc View Source

PositionLength

Gets or Sets the length (in tokens) of this token. For normal tokens this is 1; for compound tokens it's > 1.

Declaration

public virtual int PositionLength { get; set; }

Property Value

Type	Description
System.Int32

| Improve this Doc View Source

SurfaceForm

surfaceForm

Declaration

public virtual char[] SurfaceForm { get; }

Property Value

Type	Description
System.Char[]

Methods

| Improve this Doc View Source

GetBaseForm()

base form or null if token is not inflected

Declaration

public virtual string GetBaseForm()

Returns

Type	Description
System.String	base form or `null` if token is not inflected

| Improve this Doc View Source

GetInflectionForm()

inflection form or null

Declaration

public virtual string GetInflectionForm()

Returns

Type	Description
System.String	inflection form or `null`

| Improve this Doc View Source

GetInflectionType()

inflection type or null

Declaration

public virtual string GetInflectionType()

Returns

Type	Description
System.String	inflection type or `null`

| Improve this Doc View Source

GetPartOfSpeech()

part of speech.

Declaration

public virtual string GetPartOfSpeech()

Returns

Type	Description
System.String	part of speech.

| Improve this Doc View Source

GetPronunciation()

pronunciation. null if token doesn't have pronunciation.

Declaration

public virtual string GetPronunciation()

Returns

Type	Description
System.String	pronunciation. `null` if token doesn't have pronunciation.

| Improve this Doc View Source

GetReading()

reading. null if token doesn't have reading.

Declaration

public virtual string GetReading()

Returns

Type	Description
System.String	reading. `null` if token doesn't have reading.

| Improve this Doc View Source

GetSurfaceFormString()

surfaceForm as a String

Declaration

public virtual string GetSurfaceFormString()

Returns

Type	Description
System.String	surfaceForm as a String

| Improve this Doc View Source

IsKnown()

Returns true if this token is known word.

Declaration

public virtual bool IsKnown()

Returns

Type	Description
System.Boolean	`true` if this token is in standard dictionary. `false` if not.

| Improve this Doc View Source

IsUnknown()

Returns true if this token is unknown word.

Declaration

public virtual bool IsUnknown()

Returns

Type	Description
System.Boolean	`true` if this token is unknown word. `false` if not.

| Improve this Doc View Source

IsUser()

Returns true if this token is defined in user dictionary.

Declaration

public virtual bool IsUser()

Returns

Type	Description
System.Boolean	`true` if this token is in user dictionary. `false` if not.

| Improve this Doc View Source

ToString()

Declaration

public override string ToString()

Returns

Type	Description
System.String

Overrides

System.Object.ToString()