TokenStreamColumn (Lucene 10.5.1 core API)
Description: declaration: package: org.apache.lucene.document.column, class: TokenStreamColumn

Package org.apache.lucene.document.column

# Class TokenStreamColumn

java.lang.Object
org.apache.lucene.document.column.Column
org.apache.lucene.document.column.TokenStreamColumn

public abstract class TokenStreamColumn extends Column
A `Column` that provides caller-supplied `TokenStream`s for term inversion. This is the columnar analogue of setting a custom `TokenStream` on a `Field`: the per-doc tokens are fed straight to the inverter without going through an `Analyzer`.

Token-stream columns are inverted-index-only: the field type must declare `indexOptions() != NONE` and `tokenized() == true`, and must not also set doc values, points, be stored, or declare vectors. To both store a value and invert a custom token stream under one field name, add a separate stored `BinaryColumn` (or `LongColumn`) with the same name in the same batch — each column carries a distinct indexing aspect.

WARNING: This API is experimental and might change in incompatible ways in the next release.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.document.column.Column

  `Column.Density`

- ## Constructor Summary

  Constructors
  Modifier
  Constructor
  Description
  `protected`
  `TokenStreamColumn(String name, IndexableFieldType fieldType, Column.Density density)`
  Creates a TokenStreamColumn with the given field name, type, and density.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `abstract ObjectTupleCursor<TokenStream>`
  `tuples()`
  Returns a fresh tuple cursor starting at the beginning of the batch.

  ### Methods inherited from class org.apache.lucene.document.column.Column

  `density, fieldType, name`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### TokenStreamColumn

    protected TokenStreamColumn(String name, IndexableFieldType fieldType, Column.Density density)
    Creates a TokenStreamColumn with the given field name, type, and density.
    Throws:
      `IllegalArgumentException` - if `fieldType` is not indexed or not tokenized

- ## Method Details

  - ### tuples

    public abstract ObjectTupleCursor<TokenStream> tuples()
    Returns a fresh tuple cursor starting at the beginning of the batch.