Interface IStandardTokenizerInterface

Internal interface for supporting versioned grammars.

This is a Lucene.NET INTERNAL API, use at your own risk

Namespace: Lucene.Net.Analysis.Standard

Assembly: Lucene.Net.Analysis.Common.dll

Syntax

public interface IStandardTokenizerInterface

Properties

| Improve this Doc View Source

YyChar

Returns the current position.

Declaration

int YyChar { get; }

Property Value

Type	Description
System.Int32

| Improve this Doc View Source

YyLength

Returns the length of the matched text region.

Declaration

int YyLength { get; }

Property Value

Type	Description
System.Int32

Methods

| Improve this Doc View Source

GetNextToken()

Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.

Declaration

int GetNextToken()

Returns

Type	Description
System.Int32	the next token, YYEOF on end of stream

Exceptions

Type	Condition
System.IO.IOException	if any I/O-Error occurs

| Improve this Doc View Source

GetText(ICharTermAttribute)

Copies the matched text into the Lucene.Net.Analysis.TokenAttributes.ICharTermAttribute

Declaration

void GetText(ICharTermAttribute t)

Parameters

Type	Name	Description
Lucene.Net.Analysis.TokenAttributes.ICharTermAttribute	t

| Improve this Doc View Source

YyReset(TextReader)

Resets the scanner to read from a new input stream. Does not close the old reader.

All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to YYINITIAL.

Declaration

void YyReset(TextReader reader)

Parameters

Type	Name	Description
System.IO.TextReader	reader	the new input stream