Interface IStandardTokenizerInterface
Internal interface for supporting versioned grammars.
Note
This API is for internal purposes only and might change in incompatible ways in the next release.
Namespace: Lucene.Net.Analysis.Standard
Assembly: Lucene.Net.Analysis.Common.dll
Syntax
public interface IStandardTokenizerInterfaceProperties
YyChar
Returns the current position.
Declaration
int YyChar { get; }Property Value
| Type | Description |
|---|---|
| int |
YyLength
Returns the length of the matched text region.
Declaration
int YyLength { get; }Property Value
| Type | Description |
|---|---|
| int |
Methods
GetNextToken()
Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
Declaration
int GetNextToken()Returns
| Type | Description |
|---|---|
| int | the next token, YYEOF on end of stream |
Exceptions
| Type | Condition |
|---|---|
| IOException | if any I/O-Error occurs |
GetText(ICharTermAttribute)
Copies the matched text into the Lucene.Net.Analysis.TokenAttributes.ICharTermAttribute
Declaration
void GetText(ICharTermAttribute t)Parameters
| Type | Name | Description |
|---|---|---|
| ICharTermAttribute | t |
YyReset(TextReader)
Resets the scanner to read from a new input stream. Does not close the old reader.
All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set toYYINITIAL.
Declaration
void YyReset(TextReader reader)Parameters
| Type | Name | Description |
|---|---|---|
| TextReader | reader | the new input stream |