Class LogMergePolicy

This class implements a MergePolicy that tries to merge segments into levels of exponentially increasing size, where each level has fewer segments than the value of the merge factor. Whenever extra segments (beyond the merge factor upper bound) are encountered, all segments within the level are merged. You can get or set the merge factor using MergeFactor.

This class is abstract and requires a subclass to define the Size(SegmentCommitInfo) method which specifies how a segment's size is determined. LogDocMergePolicy is one subclass that measures size by document count in the segment. LogByteSizeMergePolicy is another subclass that measures size as the total byte size of the file(s) for the segment.

Inheritance

object

MergePolicy

LogMergePolicy

LogByteSizeMergePolicy

LogDocMergePolicy

Implements

IDisposable

Inherited Members

MergePolicy.DEFAULT_MAX_CFS_SEGMENT_SIZE

MergePolicy.m_writer

MergePolicy.m_noCFSRatio

MergePolicy.m_maxCFSSegmentSize

MergePolicy.Clone()

MergePolicy.SetIndexWriter(IndexWriter)

MergePolicy.Dispose()

MergePolicy.UseCompoundFile(SegmentInfos, SegmentCommitInfo)

MergePolicy.Size(SegmentCommitInfo)

MergePolicy.IsMerged(SegmentInfos, SegmentCommitInfo)

MergePolicy.NoCFSRatio

MergePolicy.MaxCFSSegmentSizeMB

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Namespace: Lucene.Net.Index

Assembly: Lucene.Net.dll

Syntax

public abstract class LogMergePolicy : MergePolicy, IDisposable

Constructors

LogMergePolicy()

Sole constructor. (For invocation by subclass constructors, typically implicit.)

Declaration

protected LogMergePolicy()

Fields

DEFAULT_MAX_MERGE_DOCS

Default maximum segment size. A segment of this size or larger will never be merged.

Declaration

public static readonly int DEFAULT_MAX_MERGE_DOCS

Field Value

Type	Description
int

See Also

MaxMergeDocs

DEFAULT_MERGE_FACTOR

Default merge factor, which is how many segments are merged at a time

Declaration

public static readonly int DEFAULT_MERGE_FACTOR

Field Value

Type	Description
int

DEFAULT_NO_CFS_RATIO

Default noCFSRatio. If a merge's size is >= 10% of the index, then we disable compound file for it.

Declaration

public static readonly double DEFAULT_NO_CFS_RATIO

Field Value

Type	Description
double

See Also

NoCFSRatio

LEVEL_LOG_SPAN

Defines the allowed range of log(size) for each level. A level is computed by taking the max segment log size, minus LEVEL_LOG_SPAN, and finding all segments falling within that range.

Declaration

public static readonly double LEVEL_LOG_SPAN

Field Value

Type	Description
double

m_calibrateSizeByDeletes

If true, we pro-rate a segment's size by the percentage of non-deleted documents.

Declaration

protected bool m_calibrateSizeByDeletes

Field Value

Type	Description
bool

m_maxMergeDocs

If a segment has more than this many documents then it will never be merged.

Declaration

protected int m_maxMergeDocs

Field Value

Type	Description
int

m_maxMergeSize

If the size of a segment exceeds this value then it will never be merged.

Declaration

protected long m_maxMergeSize

Field Value

Type	Description
long

m_maxMergeSizeForForcedMerge

If the size of a segment exceeds this value then it will never be merged during ForceMerge(int).

Declaration

protected long m_maxMergeSizeForForcedMerge

Field Value

Type	Description
long

m_mergeFactor

How many segments to merge at a time.

Declaration

protected int m_mergeFactor

Field Value

Type	Description
int

m_minMergeSize

Any segments whose size is smaller than this value will be rounded up to this value. This ensures that tiny segments are aggressively merged.

Declaration

protected long m_minMergeSize

Field Value

Type	Description
long

Properties

CalibrateSizeByDeletes

Gets or Sets whether the segment size should be calibrated by the number of deletes when choosing segments for merge.

Declaration

public virtual bool CalibrateSizeByDeletes { get; set; }

Property Value

Type	Description
bool

IsVerbose

Returns true if LogMergePolicy is enabled in infoStream.

Declaration

protected virtual bool IsVerbose { get; }

Property Value

Type	Description
bool

MaxMergeDocs

Determines the largest segment (measured by document count) that may be merged with other segments. Small values (e.g., less than 10,000) are best for interactive indexing, as this limits the length of pauses while indexing to a few seconds. Larger values are best for batched indexing and speedier searches.

The default value is MaxValue.

The default merge policy (LogByteSizeMergePolicy) also allows you to set this limit by net size (in MB) of the segment, using MaxMergeMB.

Declaration

public virtual int MaxMergeDocs { get; set; }

Property Value

Type	Description
int

MergeFactor

Gets or Sets the number of segments that are merged at once and also controls the total number of segments allowed to accumulate in the index.

This determines how often segment indices are merged by AddDocument(IEnumerable<IIndexableField>). With smaller values, less RAM is used while indexing, and searches are faster, but indexing speed is slower. With larger values, more RAM is used during indexing, and while searches is slower, indexing is faster. Thus larger values (> 10) are best for batch index creation, and smaller values (< 10) for indices that are interactively maintained.

Declaration

public virtual int MergeFactor { get; set; }

Property Value

Type	Description
int

Methods

Dispose(bool)

Release all resources for the policy.

Declaration

protected override void Dispose(bool disposing)

Parameters

Type	Name	Description
bool	disposing

Overrides

MergePolicy.Dispose(bool)

FindForcedDeletesMerges(SegmentInfos)

Finds merges necessary to force-merge all deletes from the index. We simply merge adjacent segments that have deletes, up to mergeFactor at a time.

Declaration

public override MergePolicy.MergeSpecification FindForcedDeletesMerges(SegmentInfos segmentInfos)

Parameters

Type	Name	Description
SegmentInfos	segmentInfos

Returns

Type	Description
MergePolicy.MergeSpecification

Overrides

MergePolicy.FindForcedDeletesMerges(SegmentInfos)

FindForcedMerges(SegmentInfos, int, IDictionary<SegmentCommitInfo, bool>)

Returns the merges necessary to merge the index down to a specified number of segments. this respects the m_maxMergeSizeForForcedMerge setting. By default, and assuming maxNumSegments=1, only one segment will be left in the index, where that segment has no deletions pending nor separate norms, and it is in compound file format if the current useCompoundFile setting is true. This method returns multiple merges (mergeFactor at a time) so the MergeScheduler in use may make use of concurrency.

Declaration

public override MergePolicy.MergeSpecification FindForcedMerges(SegmentInfos infos, int maxNumSegments, IDictionary<SegmentCommitInfo, bool> segmentsToMerge)

Parameters

Type	Name	Description
SegmentInfos	infos
int	maxNumSegments
IDictionary<SegmentCommitInfo, bool>	segmentsToMerge

Returns

Type	Description
MergePolicy.MergeSpecification

Overrides

MergePolicy.FindForcedMerges(SegmentInfos, int, IDictionary<SegmentCommitInfo, bool>)

FindMerges(MergeTrigger, SegmentInfos)

Checks if any merges are now necessary and returns a MergePolicy.MergeSpecification if so. A merge is necessary when there are more than MergeFactor segments at a given level. When multiple levels have too many segments, this method will return multiple merges, allowing the MergeScheduler to use concurrency.

Declaration

public override MergePolicy.MergeSpecification FindMerges(MergeTrigger mergeTrigger, SegmentInfos infos)

Parameters

Type	Name	Description
MergeTrigger	mergeTrigger
SegmentInfos	infos

Returns

Type	Description
MergePolicy.MergeSpecification

Overrides

MergePolicy.FindMerges(MergeTrigger, SegmentInfos)

IsMerged(SegmentInfos, int, IDictionary<SegmentCommitInfo, bool>)

Returns true if the number of segments eligible for merging is less than or equal to the specified maxNumSegments.

Declaration

protected virtual bool IsMerged(SegmentInfos infos, int maxNumSegments, IDictionary<SegmentCommitInfo, bool> segmentsToMerge)

Parameters

Type	Name	Description
SegmentInfos	infos
int	maxNumSegments
IDictionary<SegmentCommitInfo, bool>	segmentsToMerge

Returns

Type	Description
bool

Message(string)

Print a debug message to infoStream.

Declaration

protected virtual void Message(string message)

Parameters

Type	Name	Description
string	message

SizeBytes(SegmentCommitInfo)

Return the byte size of the provided SegmentCommitInfo, pro-rated by percentage of non-deleted documents if CalibrateSizeByDeletes is set.

Declaration

protected virtual long SizeBytes(SegmentCommitInfo info)

Parameters

Type	Name	Description
SegmentCommitInfo	info

Returns

Type	Description
long

SizeDocs(SegmentCommitInfo)

Return the number of documents in the provided SegmentCommitInfo, pro-rated by percentage of non-deleted documents if CalibrateSizeByDeletes is set.

Declaration

protected virtual long SizeDocs(SegmentCommitInfo info)

Parameters

Type	Name	Description
SegmentCommitInfo	info

Returns

Type	Description
long

ToString()

Returns a string that represents the current object.

Declaration

public override string ToString()

Returns

Type	Description
string	A string that represents the current object.

Overrides

object.ToString()

Implements

IDisposable