TokenPatternFeatureGenerator (Apache OpenNLP Tools 1.9.5 API)
Date: 2026-06-18

opennlp.tools.util.featuregen

## Class TokenPatternFeatureGenerator

- java.lang.Object
- - opennlp.tools.util.featuregen.TokenPatternFeatureGenerator

- All Implemented Interfaces:
    AdaptiveFeatureGenerator

      public class TokenPatternFeatureGenerator
      extends Object
      implements AdaptiveFeatureGenerator

  Partitions tokens into sub-tokens based on character classes and generates class features for each of the sub-tokens and combinations of those sub-tokens.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `TokenPatternFeatureGenerator()` Initializes a new instance.
    `TokenPatternFeatureGenerator(Tokenizer supportTokenizer)` Initializes a new instance.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `createFeatures(List<String> feats, String[] toks, int index, String[] preds)` Adds the appropriate features for the token at the specified index with the specified array of previous outcomes to the specified list of features.

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

    - ### Methods inherited from interface opennlp.tools.util.featuregen.AdaptiveFeatureGenerator

      `clearAdaptiveData, updateAdaptiveData`

- - ### Constructor Detail

    - #### TokenPatternFeatureGenerator

          public TokenPatternFeatureGenerator()

      Initializes a new instance. For tokinization the `SimpleTokenizer` is used.

    - #### TokenPatternFeatureGenerator

          public TokenPatternFeatureGenerator(Tokenizer supportTokenizer)

      Initializes a new instance.
      Parameters:
        `supportTokenizer` -

  - ### Method Detail

    - #### createFeatures

          public void createFeatures(List<String> feats,
                                     String[] toks,
                                     int index,
                                     String[] preds)

      Description copied from interface: `AdaptiveFeatureGenerator`
      Adds the appropriate features for the token at the specified index with the specified array of previous outcomes to the specified list of features.
      Specified by:
        `createFeatures` in interface `AdaptiveFeatureGenerator`
      Parameters:
        `feats` - The list of features to be added to.
        `toks` - The tokens of the sentence or other text unit being processed.
        `index` - The index of the token which is currently being processed.
        `preds` - The outcomes for the tokens prior to the specified index.

Copyright © 2026 The Apache Software Foundation. All rights reserved.