Class ConcurrentMergeScheduler

A MergeScheduler that runs each merge using a separate thread.

Specify the max number of threads that may run at once, and the maximum number of simultaneous merges with SetMaxMergesAndThreads(int, int).

If the number of merges exceeds the max number of threads then the largest merges are paused until one of the smaller merges completes.

If more than MaxMergeCount merges are requested then this class will forcefully throttle the incoming threads by pausing until one more more merges complete.

Inheritance
object
MergeScheduler
ConcurrentMergeScheduler
Inherited Members
Namespace: Lucene.Net.Index
Assembly: Lucene.Net.dll
Syntax
public class ConcurrentMergeScheduler : MergeScheduler, IConcurrentMergeScheduler, IMergeScheduler, IDisposable

Constructors

ConcurrentMergeScheduler()

Sole constructor, with all settings set to default values.

Declaration
public ConcurrentMergeScheduler()

Fields

DEFAULT_MAX_MERGE_COUNT

Default MaxMergeCount.

Declaration
public const int DEFAULT_MAX_MERGE_COUNT = 2
Field Value
Type Description
int

DEFAULT_MAX_THREAD_COUNT

Default MaxThreadCount. We default to 1: tests on spinning-magnet drives showed slower indexing performance if more than one merge thread runs at once (though on an SSD it was faster)

Declaration
public const int DEFAULT_MAX_THREAD_COUNT = 1
Field Value
Type Description
int

compareByMergeDocCount

Sorts ConcurrentMergeScheduler.MergeThreads; larger merges come first.

Declaration
protected static readonly IComparer<ConcurrentMergeScheduler.MergeThread> compareByMergeDocCount
Field Value
Type Description
IComparer<ConcurrentMergeScheduler.MergeThread>

m_dir

Directory that holds the index.

Declaration
protected Directory m_dir
Field Value
Type Description
Directory

m_mergeThreadCount

How many ConcurrentMergeScheduler.MergeThreads have kicked off (this is use to name them).

Declaration
protected int m_mergeThreadCount
Field Value
Type Description
int

m_mergeThreads

List of currently active ConcurrentMergeScheduler.MergeThreads.

Declaration
protected IList<ConcurrentMergeScheduler.MergeThread> m_mergeThreads
Field Value
Type Description
IList<ConcurrentMergeScheduler.MergeThread>

m_writer

IndexWriter that owns this instance.

Declaration
protected IndexWriter m_writer
Field Value
Type Description
IndexWriter

Properties

IsVerbose

Returns true if verbosing is enabled. This method is usually used in conjunction with Message(string), like that:

if (IsVerbose) 
{
    Message("your message");
}
Declaration
protected virtual bool IsVerbose { get; }
Property Value
Type Description
bool

MaxMergeCount

See SetMaxMergesAndThreads(int, int).

Declaration
public virtual int MaxMergeCount { get; }
Property Value
Type Description
int

MaxThreadCount

Returns maxThreadCount.

Declaration
public virtual int MaxThreadCount { get; }
Property Value
Type Description
int
See Also

MergeThreadCount

Returns the number of merge threads that are alive. Note that this number is <= m_mergeThreads size.

Declaration
protected virtual int MergeThreadCount { get; }
Property Value
Type Description
int

MergeThreadPriority

Return the priority that merge threads run at. By default the priority is 1 plus the priority of (ie, slightly higher priority than) the first thread that calls merge.

Declaration
public virtual int MergeThreadPriority { get; }
Property Value
Type Description
int

Methods

ClearSuppressExceptions()

Used for testing

Declaration
public virtual void ClearSuppressExceptions()

Clone()

A MergeScheduler that runs each merge using a separate thread.

Specify the max number of threads that may run at once, and the maximum number of simultaneous merges with SetMaxMergesAndThreads(int, int).

If the number of merges exceeds the max number of threads then the largest merges are paused until one of the smaller merges completes.

If more than MaxMergeCount merges are requested then this class will forcefully throttle the incoming threads by pausing until one more more merges complete.

Declaration
public override object Clone()
Returns
Type Description
object
Overrides

Dispose(bool)

Dispose this MergeScheduler.

Declaration
protected override void Dispose(bool disposing)
Parameters
Type Name Description
bool disposing
Overrides

DoMerge(OneMerge)

Does the actual merge, by calling Merge(OneMerge)

Declaration
protected virtual void DoMerge(MergePolicy.OneMerge merge)
Parameters
Type Name Description
MergePolicy.OneMerge merge

GetMergeThread(IndexWriter, OneMerge)

Create and return a new ConcurrentMergeScheduler.MergeThread

Declaration
protected virtual ConcurrentMergeScheduler.MergeThread GetMergeThread(IndexWriter writer, MergePolicy.OneMerge merge)
Parameters
Type Name Description
IndexWriter writer
MergePolicy.OneMerge merge
Returns
Type Description
ConcurrentMergeScheduler.MergeThread

HandleMergeException(Exception)

Called when an exception is hit in a background merge thread

Declaration
protected virtual void HandleMergeException(Exception exc)
Parameters
Type Name Description
Exception exc

Merge(IndexWriter, MergeTrigger, bool)

Run the merges provided by GetNextMerge().

Declaration
public override void Merge(IndexWriter writer, MergeTrigger trigger, bool newMergesFound)
Parameters
Type Name Description
IndexWriter writer

the IndexWriter to obtain the merges from.
MergeTrigger trigger

the MergeTrigger that caused this merge to happen
bool newMergesFound

true iff any new merges were found by the caller; otherwise false
Overrides

Message(string)

Outputs the given message - this method assumes IsVerbose was called and returned true.

Declaration
protected virtual void Message(string message)
Parameters
Type Name Description
string message

SetMaxMergesAndThreads(int, int)

Sets the maximum number of merge threads and simultaneous merges allowed.

Declaration
public virtual void SetMaxMergesAndThreads(int maxMergeCount, int maxThreadCount)
Parameters
Type Name Description
int maxMergeCount

the max # simultaneous merges that are allowed. If a merge is necessary yet we already have this many threads running, the incoming thread (that is calling add/updateDocument) will block until a merge thread has completed. Note that we will only run the smallest maxThreadCount merges at a time.
int maxThreadCount

The max # simultaneous merge threads that should be running at once. This must be <= maxMergeCount

SetMergeThreadPriority(int)

Set the base priority that merge threads run at. Note that CMS may increase priority of some merge threads beyond this base priority. It's best not to set this any higher than Highest(4)-maxThreadCount, so that CMS has room to set relative priority among threads.

Declaration
public virtual void SetMergeThreadPriority(int priority)
Parameters
Type Name Description
int priority

SetSuppressExceptions()

Used for testing

Declaration
public virtual void SetSuppressExceptions()

Sync()

Wait for any running merge threads to finish. This call is not interruptible as used by Dispose(bool).

Declaration
public virtual void Sync()

ToString()

Returns a string that represents the current object.

Declaration
public override string ToString()
Returns
Type Description
string

A string that represents the current object.
Overrides

UpdateMergeThreads()

Called whenever the running merges have changed, to pause & unpause threads. This method sorts the merge threads by their merge size in descending order and then pauses/unpauses threads from first to last -- that way, smaller merges are guaranteed to run before larger ones.

Declaration
protected virtual void UpdateMergeThreads()

Implements