Class SentenceTokenizer

Tokenizes input text into sentences.

The output tokens can then be broken into words with WordTokenFilter

Note

This API is experimental and might change in incompatible ways in the next release.

Inheritance
System.Object
Lucene.Net.Util.AttributeSource
Lucene.Net.Analysis.TokenStream
Lucene.Net.Analysis.Tokenizer
SentenceTokenizer
Implements
System.IDisposable
Inherited Members
Lucene.Net.Analysis.Tokenizer.m_input
Lucene.Net.Analysis.TokenStream.Dispose()
Lucene.Net.Util.AttributeSource.GetAttributeFactory()
Lucene.Net.Util.AttributeSource.GetAttributeClassesEnumerator()
Lucene.Net.Util.AttributeSource.GetAttributeImplsEnumerator()
Lucene.Net.Util.AttributeSource.AddAttributeImpl(Lucene.Net.Util.Attribute)
Lucene.Net.Util.AttributeSource.AddAttribute<T>()
Lucene.Net.Util.AttributeSource.HasAttributes
Lucene.Net.Util.AttributeSource.HasAttribute<T>()
Lucene.Net.Util.AttributeSource.GetAttribute<T>()
Lucene.Net.Util.AttributeSource.ClearAttributes()
Lucene.Net.Util.AttributeSource.CaptureState()
Lucene.Net.Util.AttributeSource.RestoreState(Lucene.Net.Util.AttributeSource.State)
Lucene.Net.Util.AttributeSource.GetHashCode()
Lucene.Net.Util.AttributeSource.ReflectWith(Lucene.Net.Util.IAttributeReflector)
Lucene.Net.Util.AttributeSource.CloneAttributes()
Lucene.Net.Util.AttributeSource.CopyTo(Lucene.Net.Util.AttributeSource)
Lucene.Net.Util.AttributeSource.ToString()
System.Object.Equals(System.Object, System.Object)
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
Namespace: Lucene.Net.Analysis.Cn.Smart
Assembly: Lucene.Net.Analysis.SmartCn.dll
Syntax
[Obsolete("Use HMMChineseTokenizer instead")]
public sealed class SentenceTokenizer : Tokenizer, IDisposable

Constructors

| Improve this Doc View Source

SentenceTokenizer(AttributeSource.AttributeFactory, TextReader)

Declaration
public SentenceTokenizer(AttributeSource.AttributeFactory factory, TextReader reader)
Parameters
Type Name Description
Lucene.Net.Util.AttributeSource.AttributeFactory factory
System.IO.TextReader reader
| Improve this Doc View Source

SentenceTokenizer(TextReader)

Declaration
public SentenceTokenizer(TextReader reader)
Parameters
Type Name Description
System.IO.TextReader reader

Methods

| Improve this Doc View Source

End()

Declaration
public override void End()
Overrides
Lucene.Net.Analysis.TokenStream.End()
| Improve this Doc View Source

IncrementToken()

Declaration
public override bool IncrementToken()
Returns
Type Description
System.Boolean
Overrides
Lucene.Net.Analysis.TokenStream.IncrementToken()
| Improve this Doc View Source

Reset()

Declaration
public override void Reset()
Overrides
Lucene.Net.Analysis.Tokenizer.Reset()

Implements

System.IDisposable