Lucene.Net  3.0.3
Lucene.Net is a .NET port of the Java Lucene Indexing Library
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Properties
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
Lucene.Net.Analysis.Standard.StandardTokenizerImpl Class Reference

This class is a scanner generated by JFlex 1.4.1 on 9/4/08 6:49 PM from the specification file /tango/mike/src/lucene.standarddigit/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex More...

Public Member Functions

int Yychar ()
 
void Yyclose ()
 Closes the input stream.
 
void Yyreset (System.IO.TextReader reader)
 Resets the scanner to read from a new input stream. Does not close the old reader.
 
int Yystate ()
 Returns the current lexical state.
 
void Yybegin (int newState)
 Enters a new lexical state
 
System.String Yytext ()
 Returns the text matched by the current regular expression.
 
char Yycharat (int pos)
 Returns the character at position pos from the matched text.
 
int Yylength ()
 Returns the length of the matched text region.
 
virtual void Yypushback (int number)
 Pushes the specified amount of characters back into the input stream.
 
virtual int GetNextToken ()
 Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
 

Public Attributes

const int YYEOF = - 1
 This character denotes the end of file
 
const int YYINITIAL = 0
 lexical states
 

Static Public Attributes

static readonly int ALPHANUM
 
static readonly int APOSTROPHE
 
static readonly int ACRONYM
 
static readonly int COMPANY
 
static readonly int EMAIL
 
static readonly int HOST
 
static readonly int NUM
 
static readonly int CJ
 
static readonly int ACRONYM_DEP
 
static readonly System.String[] TOKEN_TYPES
 

Detailed Description

This class is a scanner generated by JFlex 1.4.1 on 9/4/08 6:49 PM from the specification file /tango/mike/src/lucene.standarddigit/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex

Definition at line 43 of file StandardTokenizerImpl.cs.

Member Function Documentation

virtual int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.GetNextToken ( )
virtual

Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.

Returns
the next token
Exceptions
System.IO.IOExceptionif any I/O-Error occurs

Definition at line 517 of file StandardTokenizerImpl.cs.

void Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yybegin ( int  newState)

Enters a new lexical state

Parameters
newStatethe new lexical state

Definition at line 424 of file StandardTokenizerImpl.cs.

int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yychar ( )

Definition at line 256 of file StandardTokenizerImpl.cs.

char Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yycharat ( int  pos)

Returns the character at position pos from the matched text.

It is equivalent to yytext().charAt(pos), but faster

Parameters
posthe position of the character to fetch. A value from 0 to yylength()-1.
Returns
the character at position pos

Definition at line 449 of file StandardTokenizerImpl.cs.

void Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yyclose ( )

Closes the input stream.

Definition at line 380 of file StandardTokenizerImpl.cs.

int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yylength ( )

Returns the length of the matched text region.

Definition at line 456 of file StandardTokenizerImpl.cs.

virtual void Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yypushback ( int  number)
virtual

Pushes the specified amount of characters back into the input stream.

They will be read again by then next call of the scanning method

Parameters
numberthe number of characters to be read again. This number must not be greater than yylength()!

Definition at line 500 of file StandardTokenizerImpl.cs.

void Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yyreset ( System.IO.TextReader  reader)

Resets the scanner to read from a new input stream. Does not close the old reader.

All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL.

Parameters
readerthe new input stream

Definition at line 400 of file StandardTokenizerImpl.cs.

int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yystate ( )

Returns the current lexical state.

Definition at line 413 of file StandardTokenizerImpl.cs.

System.String Lucene.Net.Analysis.Standard.StandardTokenizerImpl.Yytext ( )

Returns the text matched by the current regular expression.

Definition at line 431 of file StandardTokenizerImpl.cs.

Member Data Documentation

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.ACRONYM
static

Definition at line 242 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.ACRONYM_DEP
static

<deprecated> this solves a bug where HOSTs that end with '.' are identified as ACRONYMs. </deprecated>

Definition at line 252 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.ALPHANUM
static

Definition at line 240 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.APOSTROPHE
static

Definition at line 241 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.CJ
static

Definition at line 247 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.COMPANY
static

Definition at line 243 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.EMAIL
static

Definition at line 244 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.HOST
static

Definition at line 245 of file StandardTokenizerImpl.cs.

readonly int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.NUM
static

Definition at line 246 of file StandardTokenizerImpl.cs.

readonly System.String [] Lucene.Net.Analysis.Standard.StandardTokenizerImpl.TOKEN_TYPES
static

Definition at line 254 of file StandardTokenizerImpl.cs.

const int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.YYEOF = - 1

This character denotes the end of file

Definition at line 47 of file StandardTokenizerImpl.cs.

const int Lucene.Net.Analysis.Standard.StandardTokenizerImpl.YYINITIAL = 0

lexical states

Definition at line 53 of file StandardTokenizerImpl.cs.


The documentation for this class was generated from the following file: