Called when the highlighter has no more tokens for the current fragment - the scorer returns the weighting it has derived for the most recent fragment, typically based on the tokens passed to getTokenScore().

Called for each token in the current fragment

called when a new fragment is started for consideration