Class FreeTextSuggester

Builds an ngram model from the text sent to Build(IInputIterator, Double) and predicts based on the last grams-1 tokens in the request sent to DoLookup(String, IEnumerable<BytesRef>, Boolean, Int32). This tries to handle the "long tail" of suggestions for when the incoming query is a never before seen query string.

Likely this suggester would only be used as a fallback, when the primary suggester fails to find any suggestions.

Note that the weight for each suggestion is unused, and the suggestions are the analyzed forms (so your analysis process should normally be very "light").

This uses the stupid backoff language model to smooth scores across ngram models; see "Large language models in machine translation" for details.

From DoLookup(String, IEnumerable<BytesRef>, Boolean, Int32), the key of each result is the ngram token; the value is * score (fixed point, cast to long). Divide by to get the score back, which ranges from 0.0 to 1.0. onlyMorePopular is unused. @lucene.experimental

Inheritance

System.Object

Lookup

FreeTextSuggester

Inherited Members

Lookup.CHARSEQUENCE_COMPARER

Lookup.Build(IDictionary)

Lookup.Load(Stream)

Lookup.Store(Stream)

Namespace: Lucene.Net.Search.Suggest.Analyzing

Assembly: Lucene.Net.Suggest.dll

Syntax

public class FreeTextSuggester : Lookup

Constructors

| Improve this Doc View Source

FreeTextSuggester(Analyzer)

Instantiate, using the provided analyzer for both indexing and lookup, using bigram model by default.

Declaration

public FreeTextSuggester(Analyzer analyzer)

Parameters

Type	Name	Description
Analyzer	analyzer

| Improve this Doc View Source

FreeTextSuggester(Analyzer, Analyzer)

Instantiate, using the provided indexing and lookup analyzers, using bigram model by default.

Declaration

public FreeTextSuggester(Analyzer indexAnalyzer, Analyzer queryAnalyzer)

Parameters

Type	Name	Description
Analyzer	indexAnalyzer
Analyzer	queryAnalyzer

| Improve this Doc View Source

FreeTextSuggester(Analyzer, Analyzer, Int32)

Instantiate, using the provided indexing and lookup analyzers, with the specified model (2 = bigram, 3 = trigram, etc.).

Declaration

public FreeTextSuggester(Analyzer indexAnalyzer, Analyzer queryAnalyzer, int grams)

Parameters

Type	Name	Description
Analyzer	indexAnalyzer
Analyzer	queryAnalyzer
System.Int32	grams

| Improve this Doc View Source

FreeTextSuggester(Analyzer, Analyzer, Int32, Byte)

Instantiate, using the provided indexing and lookup analyzers, and specified model (2 = bigram, 3 = trigram ,etc.). The separator is passed to SetTokenSeparator(String) to join multiple tokens into a single ngram token; it must be an ascii (7-bit-clean) byte. No input tokens should have this byte, otherwise is thrown.

Declaration

public FreeTextSuggester(Analyzer indexAnalyzer, Analyzer queryAnalyzer, int grams, byte separator)

Parameters

Type	Name	Description
Analyzer	indexAnalyzer
Analyzer	queryAnalyzer
System.Int32	grams
System.Byte	separator

Fields

| Improve this Doc View Source

ALPHA

The constant used for backoff smoothing; during lookup, this means that if a given trigram did not occur, and we backoff to the bigram, the overall score will be 0.4 times what the bigram model would have assigned.

Declaration

public const double ALPHA = null

Field Value

Type	Description
System.Double

| Improve this Doc View Source

CODEC_NAME

Codec name used in the header for the saved model.

Declaration

public const string CODEC_NAME = null

Field Value

Type	Description
System.String

| Improve this Doc View Source

DEFAULT_GRAMS

By default we use a bigram model.

Declaration

public const int DEFAULT_GRAMS = null

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

DEFAULT_SEPARATOR

The default character used to join multiple tokens into a single ngram token. The input tokens produced by the analyzer must not contain this character.

Declaration

public const byte DEFAULT_SEPARATOR = null

Field Value

Type	Description
System.Byte

| Improve this Doc View Source

VERSION_CURRENT

Current version of the the saved model file format.

Declaration

public const int VERSION_CURRENT = null

Field Value

Type	Description
System.Int32

| Improve this Doc View Source

VERSION_START

Initial version of the the saved model file format.

Declaration

public const int VERSION_START = null

Field Value

Type	Description
System.Int32

Properties

| Improve this Doc View Source

Count

Declaration

public override long Count { get; }

Property Value

Type	Description
System.Int64

Overrides

Lookup.Count

Methods

| Improve this Doc View Source

Build(IInputIterator)

Declaration

public override void Build(IInputIterator iterator)

Parameters

Type	Name	Description
IInputIterator	iterator

Overrides

Lookup.Build(IInputIterator)

| Improve this Doc View Source

Build(IInputIterator, Double)

Build the suggest index, using up to the specified amount of temporary RAM while building. Note that the weights for the suggestions are ignored.

Declaration

public virtual void Build(IInputIterator iterator, double ramBufferSizeMB)

Parameters

Type	Name	Description
IInputIterator	iterator
System.Double	ramBufferSizeMB

| Improve this Doc View Source

DoLookup(String, IEnumerable<BytesRef>, Boolean, Int32)

Declaration

public override IList<Lookup.LookupResult> DoLookup(string key, IEnumerable<BytesRef> contexts, bool onlyMorePopular, int num)

Parameters

Type	Name	Description
System.String	key
IEnumerable<BytesRef>	contexts
System.Boolean	onlyMorePopular
System.Int32	num

Returns

Type	Description
IList<Lookup.LookupResult>

Overrides

Lookup.DoLookup(String, IEnumerable<BytesRef>, Boolean, Int32)

| Improve this Doc View Source

DoLookup(String, IEnumerable<BytesRef>, Int32)

Retrieve suggestions.

Declaration

public virtual IList<Lookup.LookupResult> DoLookup(string key, IEnumerable<BytesRef> contexts, int num)

Parameters

Type	Name	Description
System.String	key
IEnumerable<BytesRef>	contexts
System.Int32	num

Returns

Type	Description
IList<Lookup.LookupResult>

| Improve this Doc View Source

DoLookup(String, Boolean, Int32)

Declaration

public override IList<Lookup.LookupResult> DoLookup(string key, bool onlyMorePopular, int num)

Parameters

Type	Name	Description
System.String	key
System.Boolean	onlyMorePopular
System.Int32	num

Returns

Type	Description
IList<Lookup.LookupResult>

Overrides

Lookup.DoLookup(String, Boolean, Int32)

| Improve this Doc View Source

DoLookup(String, Int32)

Lookup, without any context.

Declaration

public virtual IList<Lookup.LookupResult> DoLookup(string key, int num)

Parameters

Type	Name	Description
System.String	key
System.Int32	num

Returns

Type	Description
IList<Lookup.LookupResult>

| Improve this Doc View Source

Get(String)

Returns the weight associated with an input string, or null if it does not exist.

Declaration

public virtual object Get(string key)

Parameters

Type	Name	Description
System.String	key

Returns

Type	Description
System.Object

| Improve this Doc View Source

GetSizeInBytes()

Returns byte size of the underlying FST.

Declaration

public override long GetSizeInBytes()

Returns

Type	Description
System.Int64

Overrides

Lookup.GetSizeInBytes()

| Improve this Doc View Source

Load(DataInput)

Declaration

public override bool Load(DataInput input)

Parameters

Type	Name	Description
DataInput	input

Returns

Type	Description
System.Boolean

Overrides

Lookup.Load(DataInput)

| Improve this Doc View Source

Store(DataOutput)

Declaration

public override bool Store(DataOutput output)

Parameters

Type	Name	Description
DataOutput	output

Returns

Type	Description
System.Boolean

Overrides

Lookup.Store(DataOutput)