Class SentenceTokenizer

Tokenizes input text into sentences.

The output tokens can then be broken into words with WordTokenFilter

This is a Lucene.NET EXPERIMENTAL API, use at your own risk
Inheritance
System.Object
Lucene.Net.Util.AttributeSource
Lucene.Net.Analysis.TokenStream
Lucene.Net.Analysis.Tokenizer
SentenceTokenizer
Implements
System.IDisposable
Inherited Members
Lucene.Net.Analysis.Tokenizer.m_input
Lucene.Net.Analysis.TokenStream.Dispose()
Lucene.Net.Util.AttributeSource.GetAttributeFactory()
Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()
Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()
Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)
Lucene.Net.Util.AttributeSource.AddAttribute<T>()
Lucene.Net.Util.AttributeSource.HasAttributes
Lucene.Net.Util.AttributeSource.HasAttribute<T>()
Lucene.Net.Util.AttributeSource.GetAttribute<T>()
Lucene.Net.Util.AttributeSource.ClearAttributes()
Lucene.Net.Util.AttributeSource.CaptureState()
Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)
Lucene.Net.Util.AttributeSource.GetHashCode()
Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)
Lucene.Net.Util.AttributeSource.CloneAttributes()
Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)
Lucene.Net.Util.AttributeSource.ToString()
System.Object.Equals(System.Object, System.Object)
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
Namespace: Lucene.Net.Analysis.Cn.Smart
Assembly: Lucene.Net.Analysis.SmartCn.dll
Syntax
[Obsolete("Use HMMChineseTokenizer instead")]
public sealed class SentenceTokenizer : Tokenizer, IDisposable

Constructors

| Improve this Doc View Source

SentenceTokenizer(AttributeSource.AttributeFactory, TextReader)

Declaration
public SentenceTokenizer(AttributeSource.AttributeFactory factory, TextReader reader)
Parameters
Type Name Description
Lucene.Net.Util.AttributeSource.AttributeFactory factory
System.IO.TextReader reader
| Improve this Doc View Source

SentenceTokenizer(TextReader)

Declaration
public SentenceTokenizer(TextReader reader)
Parameters
Type Name Description
System.IO.TextReader reader

Methods

| Improve this Doc View Source

End()

Declaration
public override void End()
Overrides
Lucene.Net.Analysis.TokenStream.End()
| Improve this Doc View Source

IncrementToken()

Declaration
public override bool IncrementToken()
Returns
Type Description
System.Boolean
Overrides
Lucene.Net.Analysis.TokenStream.IncrementToken()
| Improve this Doc View Source

Reset()

Declaration
public override void Reset()
Overrides
Lucene.Net.Analysis.Tokenizer.Reset()

Implements

System.IDisposable