Class SentenceTokenizer

Tokenizes input text into sentences.

The output tokens can then be broken into words with WordTokenFilter

This is a Lucene.NET EXPERIMENTAL API, use at your own risk
Inheritance
System.Object
AttributeSource
TokenStream
Tokenizer
SentenceTokenizer
Implements
System.IDisposable
Inherited Members
System.Object.Equals(System.Object, System.Object)
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
Namespace: Lucene.Net.Analysis.Cn.Smart
Assembly: Lucene.Net.Analysis.SmartCn.dll
Syntax
[Obsolete("Use HMMChineseTokenizer instead")]
public sealed class SentenceTokenizer : Tokenizer, IDisposable

Constructors

| Improve this Doc View Source

SentenceTokenizer(AttributeSource.AttributeFactory, TextReader)

Declaration
public SentenceTokenizer(AttributeSource.AttributeFactory factory, TextReader reader)
Parameters
Type Name Description
AttributeSource.AttributeFactory factory
System.IO.TextReader reader
| Improve this Doc View Source

SentenceTokenizer(TextReader)

Declaration
public SentenceTokenizer(TextReader reader)
Parameters
Type Name Description
System.IO.TextReader reader

Methods

| Improve this Doc View Source

End()

Declaration
public override void End()
Overrides
| Improve this Doc View Source

IncrementToken()

Declaration
public override bool IncrementToken()
Returns
Type Description
System.Boolean
Overrides
| Improve this Doc View Source

Reset()

Declaration
public override void Reset()
Overrides

Implements

System.IDisposable