The TokenSources type exposes the following members.

Methods

  NameDescription
Public methodEquals
Determines whether the specified Object is equal to the current Object.
(Inherited from Object.)
Protected methodFinalize
Allows an Object to attempt to free resources and perform other cleanup operations before the Object is reclaimed by garbage collection.
(Inherited from Object.)
Public methodStatic memberGetAnyTokenStream
A convenience method that tries a number of approaches to getting a token stream. The cost of finding there are no termVectors in the index is minimal (1000 invocations still registers 0 ms). So this "lazy" (flexible?) approach to coding is probably acceptable
Public methodGetHashCode
Serves as a hash function for a particular type.
(Inherited from Object.)
Public methodStatic memberGetTokenStream(TermPositionVector)
Public methodStatic memberGetTokenStream(TermPositionVector, Boolean)
Low level api. Returns a token stream or null if no offset info available in index. This can be used to feed the highlighter with a pre-parsed token stream In my tests the speeds to recreate 1000 token streams using this method are: - with TermVector offset only data stored - 420 milliseconds - with TermVector offset AND position data stored - 271 milliseconds (nb timings for TermVector with position data are based on a tokenizer with contiguous positions - no overlaps or gaps) The cost of not using TermPositionVector to store pre-parsed content and using an analyzer to re-parse the original content: - reanalyzing the original content - 980 milliseconds The re-analyze timings will typically vary depending on - 1) The complexity of the analyzer code (timings above were using a stemmer/lowercaser/stopword combo) 2) The number of other fields (Lucene reads ALL fields off the disk when accessing just one document field - can cost dear!) 3) Use of compression on field storage - could be faster cos of compression (less disk IO) or slower (more CPU burn) depending on the content.
Public methodStatic memberGetTokenStream(IndexReader, Int32, String)
Public methodStatic memberGetTokenStream(IndexReader, Int32, String, Analyzer)
Public methodGetType
Gets the Type of the current instance.
(Inherited from Object.)
Protected methodMemberwiseClone
Creates a shallow copy of the current Object.
(Inherited from Object.)
Public methodToString
Returns a String that represents the current Object.
(Inherited from Object.)

See Also