Show / Hide Table of Contents

    Class PackedInt32s

    Simplistic compression for array of unsigned long values. Each value is >= 0 and <= a specified maximum value. The values are stored as packed ints, with each value consuming a fixed number of bits.

    NOTE: This was PackedInts in Lucene.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Inheritance
    System.Object
    PackedInt32s
    Inherited Members
    System.Object.Equals(System.Object)
    System.Object.Equals(System.Object, System.Object)
    System.Object.GetHashCode()
    System.Object.GetType()
    System.Object.MemberwiseClone()
    System.Object.ReferenceEquals(System.Object, System.Object)
    System.Object.ToString()
    Namespace: Lucene.Net.Util.Packed
    Assembly: Lucene.Net.dll
    Syntax
    public class PackedInt32s

    Fields

    | Improve this Doc View Source

    CODEC_NAME

    Declaration
    public static string CODEC_NAME
    Field Value
    Type Description
    System.String
    | Improve this Doc View Source

    COMPACT

    No memory overhead at all, but the returned implementation may be slow.

    Declaration
    public static float COMPACT
    Field Value
    Type Description
    System.Single
    | Improve this Doc View Source

    DEFAULT

    At most 20% memory overhead.

    Declaration
    public static float DEFAULT
    Field Value
    Type Description
    System.Single
    | Improve this Doc View Source

    DEFAULT_BUFFER_SIZE

    Default amount of memory to use for bulk operations.

    Declaration
    public static int DEFAULT_BUFFER_SIZE
    Field Value
    Type Description
    System.Int32
    | Improve this Doc View Source

    FAST

    At most 50% memory overhead, always select a reasonably fast implementation.

    Declaration
    public static float FAST
    Field Value
    Type Description
    System.Single
    | Improve this Doc View Source

    FASTEST

    At most 700% memory overhead, always select a direct implementation.

    Declaration
    public static float FASTEST
    Field Value
    Type Description
    System.Single
    | Improve this Doc View Source

    VERSION_BYTE_ALIGNED

    Declaration
    public static int VERSION_BYTE_ALIGNED
    Field Value
    Type Description
    System.Int32
    | Improve this Doc View Source

    VERSION_CURRENT

    Declaration
    public static int VERSION_CURRENT
    Field Value
    Type Description
    System.Int32
    | Improve this Doc View Source

    VERSION_START

    Declaration
    public static int VERSION_START
    Field Value
    Type Description
    System.Int32

    Methods

    | Improve this Doc View Source

    BitsRequired(Int64)

    Returns how many bits are required to hold values up to and including maxValue.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static int BitsRequired(long maxValue)
    Parameters
    Type Name Description
    System.Int64 maxValue

    The maximum value that should be representable.

    Returns
    Type Description
    System.Int32

    The amount of bits needed to represent values from 0 to maxValue.

    | Improve this Doc View Source

    CheckVersion(Int32)

    Check the validity of a version number.

    Declaration
    public static void CheckVersion(int version)
    Parameters
    Type Name Description
    System.Int32 version
    | Improve this Doc View Source

    Copy(PackedInt32s.Reader, Int32, PackedInt32s.Mutable, Int32, Int32, Int32)

    Copy src[srcPos:srcPos+len] into dest[destPos:destPos+len] using at most mem bytes.

    Declaration
    public static void Copy(PackedInt32s.Reader src, int srcPos, PackedInt32s.Mutable dest, int destPos, int len, int mem)
    Parameters
    Type Name Description
    PackedInt32s.Reader src
    System.Int32 srcPos
    PackedInt32s.Mutable dest
    System.Int32 destPos
    System.Int32 len
    System.Int32 mem
    | Improve this Doc View Source

    FastestFormatAndBits(Int32, Int32, Single)

    Try to find the PackedInt32s.Format and number of bits per value that would restore from disk the fastest reader whose overhead is less than acceptableOverheadRatio.

    The acceptableOverheadRatio parameter makes sense for random-access PackedInt32s.Readers. In case you only plan to perform sequential access on this stream later on, you should probably use COMPACT.

    If you don't know how many values you are going to write, use valueCount = -1.

    Declaration
    public static PackedInt32s.FormatAndBits FastestFormatAndBits(int valueCount, int bitsPerValue, float acceptableOverheadRatio)
    Parameters
    Type Name Description
    System.Int32 valueCount
    System.Int32 bitsPerValue
    System.Single acceptableOverheadRatio
    Returns
    Type Description
    PackedInt32s.FormatAndBits
    | Improve this Doc View Source

    GetDecoder(PackedInt32s.Format, Int32, Int32)

    Get a PackedInt32s.IDecoder.

    Declaration
    public static PackedInt32s.IDecoder GetDecoder(PackedInt32s.Format format, int version, int bitsPerValue)
    Parameters
    Type Name Description
    PackedInt32s.Format format

    The format used to store packed System.Int32s.

    System.Int32 version

    The compatibility version.

    System.Int32 bitsPerValue

    The number of bits per value.

    Returns
    Type Description
    PackedInt32s.IDecoder

    A decoder.

    | Improve this Doc View Source

    GetDirectReader(IndexInput)

    Construct a direct PackedInt32s.Reader from an IndexInput. this method is useful to restore data from streams which have been created using GetWriter(DataOutput, Int32, Int32, Single).

    The returned reader will have very little memory overhead, but every call to Get(Int32) is likely to perform a disk seek.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Reader GetDirectReader(IndexInput in)
    Parameters
    Type Name Description
    IndexInput in

    The stream to read data from.

    Returns
    Type Description
    PackedInt32s.Reader

    A direct PackedInt32s.Reader.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error.

    | Improve this Doc View Source

    GetDirectReaderNoHeader(IndexInput, PackedInt32s.Format, Int32, Int32, Int32)

    Expert: Construct a direct PackedInt32s.Reader from a stream without reading metadata at the beginning of the stream. This method is useful to restore data from streams which have been created using GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32).

    The returned reader will have very little memory overhead, but every call to Get(Int32) is likely to perform a disk seek.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Reader GetDirectReaderNoHeader(IndexInput in, PackedInt32s.Format format, int version, int valueCount, int bitsPerValue)
    Parameters
    Type Name Description
    IndexInput in

    The stream to read data from.

    PackedInt32s.Format format

    The format used to serialize.

    System.Int32 version

    The version used to serialize the data.

    System.Int32 valueCount

    How many values the stream holds.

    System.Int32 bitsPerValue

    The number of bits per value.

    Returns
    Type Description
    PackedInt32s.Reader

    A direct PackedInt32s.Reader.

    | Improve this Doc View Source

    GetDirectReaderNoHeader(IndexInput, PackedInt32s.Header)

    Expert: Construct a direct PackedInt32s.Reader from an IndexInput without reading metadata at the beginning of the stream. this method is useful to restore data when metadata has been previously read using ReadHeader(DataInput).

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Reader GetDirectReaderNoHeader(IndexInput in, PackedInt32s.Header header)
    Parameters
    Type Name Description
    IndexInput in

    The stream to read data from, positioned at the beginning of the packed values.

    PackedInt32s.Header header

    Metadata result from ReadHeader(DataInput).

    Returns
    Type Description
    PackedInt32s.Reader

    A PackedInt32s.Reader.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error.

    See Also
    ReadHeader(DataInput)
    | Improve this Doc View Source

    GetEncoder(PackedInt32s.Format, Int32, Int32)

    Get an PackedInt32s.IEncoder.

    Declaration
    public static PackedInt32s.IEncoder GetEncoder(PackedInt32s.Format format, int version, int bitsPerValue)
    Parameters
    Type Name Description
    PackedInt32s.Format format

    The format used to store packed System.Int32s.

    System.Int32 version

    The compatibility version.

    System.Int32 bitsPerValue

    The number of bits per value.

    Returns
    Type Description
    PackedInt32s.IEncoder

    An encoder.

    | Improve this Doc View Source

    GetMutable(Int32, Int32, PackedInt32s.Format)

    Same as GetMutable(Int32, Int32, Single) with a pre-computed number of bits per value and format.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Mutable GetMutable(int valueCount, int bitsPerValue, PackedInt32s.Format format)
    Parameters
    Type Name Description
    System.Int32 valueCount
    System.Int32 bitsPerValue
    PackedInt32s.Format format
    Returns
    Type Description
    PackedInt32s.Mutable
    | Improve this Doc View Source

    GetMutable(Int32, Int32, Single)

    Create a packed integer array with the given amount of values initialized to 0. The valueCount and the bitsPerValue cannot be changed after creation. All Mutables known by this factory are kept fully in RAM.

    Positive values of acceptableOverheadRatio will trade space for speed by selecting a faster but potentially less memory-efficient implementation. An acceptableOverheadRatio of COMPACT will make sure that the most memory-efficient implementation is selected whereas FASTEST will make sure that the fastest implementation is selected.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Mutable GetMutable(int valueCount, int bitsPerValue, float acceptableOverheadRatio)
    Parameters
    Type Name Description
    System.Int32 valueCount

    The number of elements.

    System.Int32 bitsPerValue

    The number of bits available for any given value.

    System.Single acceptableOverheadRatio

    An acceptable overhead ratio per value.

    Returns
    Type Description
    PackedInt32s.Mutable

    A mutable packed integer array.

    | Improve this Doc View Source

    GetReader(DataInput)

    Restore a PackedInt32s.Reader from a stream.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Reader GetReader(DataInput in)
    Parameters
    Type Name Description
    DataInput in

    The stream to read data from.

    Returns
    Type Description
    PackedInt32s.Reader

    A PackedInt32s.Reader.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error

    | Improve this Doc View Source

    GetReaderIterator(DataInput, Int32)

    Retrieve PackedInt32s as a PackedInt32s.IReaderIterator.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.IReaderIterator GetReaderIterator(DataInput in, int mem)
    Parameters
    Type Name Description
    DataInput in

    Positioned at the beginning of a stored packed int structure.

    System.Int32 mem

    How much memory the iterator is allowed to use to read-ahead (likely to speed up iteration).

    Returns
    Type Description
    PackedInt32s.IReaderIterator

    An iterator to access the values.

    Exceptions
    Type Condition
    System.IO.IOException

    If the structure could not be retrieved.

    | Improve this Doc View Source

    GetReaderIteratorNoHeader(DataInput, PackedInt32s.Format, Int32, Int32, Int32, Int32)

    Expert: Restore a PackedInt32s.IReaderIterator from a stream without reading metadata at the beginning of the stream. This method is useful to restore data from streams which have been created using GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32).

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.IReaderIterator GetReaderIteratorNoHeader(DataInput in, PackedInt32s.Format format, int version, int valueCount, int bitsPerValue, int mem)
    Parameters
    Type Name Description
    DataInput in

    The stream to read data from, positioned at the beginning of the packed values.

    PackedInt32s.Format format

    The format used to serialize.

    System.Int32 version

    The version used to serialize the data.

    System.Int32 valueCount

    How many values the stream holds.

    System.Int32 bitsPerValue

    the number of bits per value.

    System.Int32 mem

    How much memory the iterator is allowed to use to read-ahead (likely to speed up iteration).

    Returns
    Type Description
    PackedInt32s.IReaderIterator

    A PackedInt32s.IReaderIterator.

    See Also
    GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32)
    | Improve this Doc View Source

    GetReaderNoHeader(DataInput, PackedInt32s.Format, Int32, Int32, Int32)

    Expert: Restore a PackedInt32s.Reader from a stream without reading metadata at the beginning of the stream. This method is useful to restore data from streams which have been created using GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32).

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Reader GetReaderNoHeader(DataInput in, PackedInt32s.Format format, int version, int valueCount, int bitsPerValue)
    Parameters
    Type Name Description
    DataInput in

    The stream to read data from, positioned at the beginning of the packed values.

    PackedInt32s.Format format

    The format used to serialize.

    System.Int32 version

    The version used to serialize the data.

    System.Int32 valueCount

    How many values the stream holds.

    System.Int32 bitsPerValue

    The number of bits per value.

    Returns
    Type Description
    PackedInt32s.Reader

    A PackedInt32s.Reader.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error.

    See Also
    GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32)
    | Improve this Doc View Source

    GetReaderNoHeader(DataInput, PackedInt32s.Header)

    Expert: Restore a PackedInt32s.Reader from a stream without reading metadata at the beginning of the stream. this method is useful to restore data when metadata has been previously read using ReadHeader(DataInput).

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Reader GetReaderNoHeader(DataInput in, PackedInt32s.Header header)
    Parameters
    Type Name Description
    DataInput in

    The stream to read data from, positioned at the beginning of the packed values.

    PackedInt32s.Header header

    Metadata result from ReadHeader(DataInput).

    Returns
    Type Description
    PackedInt32s.Reader

    A PackedInt32s.Reader.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error.

    See Also
    ReadHeader(DataInput)
    | Improve this Doc View Source

    GetWriter(DataOutput, Int32, Int32, Single)

    Create a packed integer array writer for the given output, format, value count, and number of bits per value.

    The resulting stream will be long-aligned. this means that depending on the format which is used under the hoods, up to 63 bits will be wasted. An easy way to make sure that no space is lost is to always use a valueCount that is a multiple of 64.

    This method writes metadata to the stream, so that the resulting stream is sufficient to restore a PackedInt32s.Reader from it. You don't need to track valueCount or bitsPerValue by yourself. In case this is a problem, you should probably look at GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32).

    The acceptableOverheadRatio parameter controls how readers that will be restored from this stream trade space for speed by selecting a faster but potentially less memory-efficient implementation. An acceptableOverheadRatio of COMPACT will make sure that the most memory-efficient implementation is selected whereas FASTEST will make sure that the fastest implementation is selected. In case you are only interested in reading this stream sequentially later on, you should probably use COMPACT.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Writer GetWriter(DataOutput out, int valueCount, int bitsPerValue, float acceptableOverheadRatio)
    Parameters
    Type Name Description
    DataOutput out

    The data output.

    System.Int32 valueCount

    The number of values.

    System.Int32 bitsPerValue

    The number of bits per value.

    System.Single acceptableOverheadRatio

    An acceptable overhead ratio per value.

    Returns
    Type Description
    PackedInt32s.Writer

    A PackedInt32s.Writer.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error.

    | Improve this Doc View Source

    GetWriterNoHeader(DataOutput, PackedInt32s.Format, Int32, Int32, Int32)

    Expert: Create a packed integer array writer for the given output, format, value count, and number of bits per value.

    The resulting stream will be long-aligned. this means that depending on the format which is used, up to 63 bits will be wasted. An easy way to make sure that no space is lost is to always use a valueCount that is a multiple of 64.

    This method does not write any metadata to the stream, meaning that it is your responsibility to store it somewhere else in order to be able to recover data from the stream later on:

    • format (using Id),
    • valueCount,
    • bitsPerValue,
    • VERSION_CURRENT.

    It is possible to start writing values without knowing how many of them you are actually going to write. To do this, just pass -1 as valueCount. On the other hand, for any positive value of valueCount, the returned writer will make sure that you don't write more values than expected and pad the end of stream with zeros in case you have written less than valueCount when calling Finish().

    The mem parameter lets you control how much memory can be used to buffer changes in memory before flushing to disk. High values of mem are likely to improve throughput. On the other hand, if speed is not that important to you, a value of 0 will use as little memory as possible and should already offer reasonable throughput.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static PackedInt32s.Writer GetWriterNoHeader(DataOutput out, PackedInt32s.Format format, int valueCount, int bitsPerValue, int mem)
    Parameters
    Type Name Description
    DataOutput out

    The data output.

    PackedInt32s.Format format

    The format to use to serialize the values.

    System.Int32 valueCount

    The number of values.

    System.Int32 bitsPerValue

    The number of bits per value.

    System.Int32 mem

    How much memory (in bytes) can be used to speed up serialization.

    Returns
    Type Description
    PackedInt32s.Writer

    A PackedInt32s.Writer.

    See Also
    GetReaderIteratorNoHeader(DataInput, PackedInt32s.Format, Int32, Int32, Int32, Int32)
    GetReaderNoHeader(DataInput, PackedInt32s.Format, Int32, Int32, Int32)
    | Improve this Doc View Source

    MaxValue(Int32)

    Calculates the maximum unsigned long that can be expressed with the given number of bits.

    This is a Lucene.NET INTERNAL API, use at your own risk
    Declaration
    public static long MaxValue(int bitsPerValue)
    Parameters
    Type Name Description
    System.Int32 bitsPerValue

    The number of bits available for any given value.

    Returns
    Type Description
    System.Int64

    The maximum value for the given bits.

    | Improve this Doc View Source

    ReadHeader(DataInput)

    Expert: reads only the metadata from a stream. This is useful to later restore a stream or open a direct reader via GetReaderNoHeader(DataInput, PackedInt32s.Header) or GetDirectReaderNoHeader(IndexInput, PackedInt32s.Header).

    Declaration
    public static PackedInt32s.Header ReadHeader(DataInput in)
    Parameters
    Type Name Description
    DataInput in

    The stream to read data.

    Returns
    Type Description
    PackedInt32s.Header

    Packed integer metadata.

    Exceptions
    Type Condition
    System.IO.IOException

    If there is a low-level I/O error.

    See Also
    GetReaderNoHeader(DataInput, PackedInt32s.Header)
    GetDirectReaderNoHeader(IndexInput, PackedInt32s.Header)

    Extension Methods

    Number.IsNumber(Object)
    • Improve this Doc
    • View Source
    Back to top Copyright © 2019 Licensed to the Apache Software Foundation (ASF)