Class TokenizerFactory

Abstract parent class for analysis factories that create Tokenizer instances.

Inheritance
System.Object
AbstractAnalysisFactory
TokenizerFactory
ArabicLetterTokenizerFactory
CJKTokenizerFactory
ChineseTokenizerFactory
KeywordTokenizerFactory
LetterTokenizerFactory
LowerCaseTokenizerFactory
WhitespaceTokenizerFactory
JapaneseTokenizerFactory
EdgeNGramTokenizerFactory
NGramTokenizerFactory
PathHierarchyTokenizerFactory
PatternTokenizerFactory
RussianLetterTokenizerFactory
ClassicTokenizerFactory
StandardTokenizerFactory
UAX29URLEmailTokenizerFactory
WikipediaTokenizerFactory
Inherited Members
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: Lucene.Net.Analysis.Util
Assembly: Lucene.Net.Analysis.Common.dll
Syntax
public abstract class TokenizerFactory : AbstractAnalysisFactory

Constructors

| Improve this Doc View Source

TokenizerFactory(IDictionary<String, String>)

Initialize this factory via a set of key-value pairs.

Declaration
protected TokenizerFactory(IDictionary<string, string> args)
Parameters
Type Name Description
System.Collections.Generic.IDictionary<System.String, System.String> args

Properties

| Improve this Doc View Source

AvailableTokenizers

returns a list of all available tokenizer names from the host project's referenced assemblies

Declaration
public static ICollection<string> AvailableTokenizers { get; }
Property Value
Type Description
System.Collections.Generic.ICollection<System.String>

Methods

| Improve this Doc View Source

Create(AttributeSource.AttributeFactory, TextReader)

Creates a TokenStream of the specified input using the given AttributeSource.AttributeFactory

Declaration
public abstract Tokenizer Create(AttributeSource.AttributeFactory factory, TextReader input)
Parameters
Type Name Description
AttributeSource.AttributeFactory factory
System.IO.TextReader input
Returns
Type Description
Tokenizer
| Improve this Doc View Source

Create(TextReader)

Creates a TokenStream of the specified input using the default attribute factory.

Declaration
public Tokenizer Create(TextReader input)
Parameters
Type Name Description
System.IO.TextReader input
Returns
Type Description
Tokenizer
| Improve this Doc View Source

ForName(String, IDictionary<String, String>)

looks up a tokenizer by name from the host project's referenced assemblies

Declaration
public static TokenizerFactory ForName(string name, IDictionary<string, string> args)
Parameters
Type Name Description
System.String name
System.Collections.Generic.IDictionary<System.String, System.String> args
Returns
Type Description
TokenizerFactory
| Improve this Doc View Source

LookupClass(String)

looks up a tokenizer class by name from the host project's referenced assemblies

Declaration
public static Type LookupClass(string name)
Parameters
Type Name Description
System.String name
Returns
Type Description
System.Type
| Improve this Doc View Source

ReloadTokenizers()

Reloads the factory list. Changes to the factories are visible after the method ends, all iterators (AvailableTokenizers,...) stay consistent.

NOTE: Only new factories are added, existing ones are never removed or replaced.

This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!

Declaration
public static void ReloadTokenizers()