Class NRTCachingDirectory

Wraps a RAMDirectory around any provided delegate directory, to be used during NRT search.

This class is likely only useful in a near-real-time context, where indexing rate is lowish but reopen rate is highish, resulting in many tiny files being written. This directory keeps such segments (as well as the segments produced by merging them, as long as they are small enough), in RAM.

This is safe to use: when your app calls Commit(), all cached files will be flushed from the cached and sync'd.

Here's a simple example usage: 
using Directory fsDir = FSDirectory.Open(new DirectoryInfo("/path/to/index"));
using NRTCachingDirectory cachedFSDir = new NRTCachingDirectory(fsDir, 5.0, 60.0);
IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_48, analyzer);
using IndexWriter writer = new IndexWriter(cachedFSDir, conf);

This will cache all newly flushed segments, all merges

whose expected segment size is <= 5 MB, unless the net cached bytes exceeds 60 MB at which point all writes will not be cached (until the net bytes falls below 60 MB).

Note

This API is experimental and might change in incompatible ways in the next release.

Inheritance
object
Directory
BaseDirectory
NRTCachingDirectory
Implements
Inherited Members
Namespace: Lucene.Net.Store
Assembly: Lucene.Net.dll
Syntax
public class NRTCachingDirectory : BaseDirectory, IDisposable

Constructors

NRTCachingDirectory(Directory, double, double)

We will cache a newly created output if 1) it's a flush or a merge and the estimated size of the merged segment is <= maxMergeSizeMB, and 2) the total cached bytes is <= maxCachedMB

Declaration
public NRTCachingDirectory(Directory @delegate, double maxMergeSizeMB, double maxCachedMB)
Parameters
Type Name Description
Directory delegate
double maxMergeSizeMB
double maxCachedMB

Properties

Delegate

Wraps a RAMDirectory around any provided delegate directory, to be used during NRT search.

This class is likely only useful in a near-real-time context, where indexing rate is lowish but reopen rate is highish, resulting in many tiny files being written. This directory keeps such segments (as well as the segments produced by merging them, as long as they are small enough), in RAM.

This is safe to use: when your app calls Commit(), all cached files will be flushed from the cached and sync'd.

Here's a simple example usage: 
using Directory fsDir = FSDirectory.Open(new DirectoryInfo("/path/to/index"));
using NRTCachingDirectory cachedFSDir = new NRTCachingDirectory(fsDir, 5.0, 60.0);
IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_48, analyzer);
using IndexWriter writer = new IndexWriter(cachedFSDir, conf);

This will cache all newly flushed segments, all merges

whose expected segment size is <= 5 MB, unless the net cached bytes exceeds 60 MB at which point all writes will not be cached (until the net bytes falls below 60 MB).

Note

This API is experimental and might change in incompatible ways in the next release.

Declaration
public virtual Directory Delegate { get; }
Property Value
Type Description
Directory

LockFactory

Get the LockFactory that this Directory instance is using for its locking implementation. Note that this may be null for Directory implementations that provide their own locking implementation.

Declaration
public override LockFactory LockFactory { get; }
Property Value
Type Description
LockFactory
Overrides

Methods

ClearLock(string)

Attempt to clear (forcefully unlock and remove) the specified lock. Only call this at a time when you are certain this lock is no longer in use.

Declaration
public override void ClearLock(string name)
Parameters
Type Name Description
string name

name of the lock to be cleared.
Overrides

CreateOutput(string, IOContext)

Creates a new, empty file in the directory with the given name. Returns a stream writing this file.

Declaration
public override IndexOutput CreateOutput(string name, IOContext context)
Parameters
Type Name Description
string name
IOContext context
Returns
Type Description
IndexOutput
Overrides

CreateSlicer(string, IOContext)

Creates an Directory.IndexInputSlicer for the given file name. Directory.IndexInputSlicer allows other Directory implementations to efficiently open one or more sliced IndexInput instances from a single file handle. The underlying file handle is kept open until the Directory.IndexInputSlicer is closed.

Throws FileNotFoundException if the file does not exist.

Note

This API is for internal purposes only and might change in incompatible ways in the next release.

Note

This API is experimental and might change in incompatible ways in the next release.

Declaration
public override Directory.IndexInputSlicer CreateSlicer(string name, IOContext context)
Parameters
Type Name Description
string name
IOContext context
Returns
Type Description
Directory.IndexInputSlicer
Overrides
Exceptions
Type Condition
IOException

if an IOException occurs

DeleteFile(string)

Removes an existing file in the directory.

Declaration
public override void DeleteFile(string name)
Parameters
Type Name Description
string name
Overrides

Dispose(bool)

Dispose this directory, which flushes any cached files to the delegate and then disposes the delegate.

Declaration
protected override void Dispose(bool disposing)
Parameters
Type Name Description
bool disposing
Overrides

DoCacheWrite(string, IOContext)

Subclass can override this to customize logic; return true if this file should be written to the RAMDirectory.

Declaration
protected virtual bool DoCacheWrite(string name, IOContext context)
Parameters
Type Name Description
string name
IOContext context
Returns
Type Description
bool

FileExists(string)

Returns true iff a file with the given name exists.

Declaration
[Obsolete("this method will be removed in 5.0")]
public override bool FileExists(string name)
Parameters
Type Name Description
string name
Returns
Type Description
bool
Overrides

FileLength(string)

Returns the length of a file in the directory. this method follows the following contract:

Declaration
public override long FileLength(string name)
Parameters
Type Name Description
string name

the name of the file for which to return the length.
Returns
Type Description
long
Overrides
Exceptions
Type Condition
IOException

if there was an IO error while retrieving the file's length.

GetLockID()

Return a string identifier that uniquely differentiates this Directory instance from other Directory instances. This ID should be the same if two Directory instances (even in different AppDomains and/or on different machines) are considered "the same index". This is how locking "scopes" to the right index.

Declaration
public override string GetLockID()
Returns
Type Description
string
Overrides

GetSizeInBytes()

Returns how many bytes are being used by the RAMDirectory cache

Declaration
public virtual long GetSizeInBytes()
Returns
Type Description
long

ListAll()

Returns an array of strings, one for each file in the directory.

Declaration
public override string[] ListAll()
Returns
Type Description
string[]
Overrides
Exceptions
Type Condition
DirectoryNotFoundException

if the directory is not prepared for any write operations (such as CreateOutput(string, IOContext)).
IOException

in case of other IO errors

ListCachedFiles()

Wraps a RAMDirectory around any provided delegate directory, to be used during NRT search.

This class is likely only useful in a near-real-time context, where indexing rate is lowish but reopen rate is highish, resulting in many tiny files being written. This directory keeps such segments (as well as the segments produced by merging them, as long as they are small enough), in RAM.

This is safe to use: when your app calls Commit(), all cached files will be flushed from the cached and sync'd.

Here's a simple example usage: 
using Directory fsDir = FSDirectory.Open(new DirectoryInfo("/path/to/index"));
using NRTCachingDirectory cachedFSDir = new NRTCachingDirectory(fsDir, 5.0, 60.0);
IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_48, analyzer);
using IndexWriter writer = new IndexWriter(cachedFSDir, conf);

This will cache all newly flushed segments, all merges

whose expected segment size is <= 5 MB, unless the net cached bytes exceeds 60 MB at which point all writes will not be cached (until the net bytes falls below 60 MB).

Note

This API is experimental and might change in incompatible ways in the next release.

Declaration
public virtual string[] ListCachedFiles()
Returns
Type Description
string[]

MakeLock(string)

Construct a Lock.

Declaration
public override Lock MakeLock(string name)
Parameters
Type Name Description
string name

the name of the lock file
Returns
Type Description
Lock
Overrides

OpenInput(string, IOContext)

Returns a stream reading an existing file, with the specified read buffer size. The particular Directory implementation may ignore the buffer size. Currently the only Directory implementations that respect this parameter are FSDirectory and CompoundFileDirectory.

Throws FileNotFoundException if the file does not exist.
Declaration
public override IndexInput OpenInput(string name, IOContext context)
Parameters
Type Name Description
string name
IOContext context
Returns
Type Description
IndexInput
Overrides

SetLockFactory(LockFactory)

Set the LockFactory that this Directory instance should use for its locking implementation. Each * instance of LockFactory should only be used for one directory (ie, do not share a single instance across multiple Directories).

Declaration
public override void SetLockFactory(LockFactory lockFactory)
Parameters
Type Name Description
LockFactory lockFactory

instance of LockFactory.
Overrides

Sync(ICollection<string>)

Ensure that any writes to these files are moved to stable storage. Lucene uses this to properly commit changes to the index, to prevent a machine/OS crash from corrupting the index.

NOTE: Clients may call this method for same files over and over again, so some impls might optimize for that. For other impls the operation can be a noop, for various reasons.

Declaration
public override void Sync(ICollection<string> fileNames)
Parameters
Type Name Description
ICollection<string> fileNames
Overrides

ToString()

Returns a string that represents the current object.

Declaration
public override string ToString()
Returns
Type Description
string

A string that represents the current object.
Overrides

Implements