Uses of Interface opennlp.tools.tokenize.Tokenizer (Apache OpenNLP :: API 3.0.0-M4 API)
Description: use: package: opennlp.tools.tokenize, interface: Tokenizer

# Uses of Interface
opennlp.tools.tokenize.Tokenizer

Packages that use Tokenizer
Package
Description
opennlp.tools.tokenize

- ## Uses of Tokenizer in opennlp.tools.tokenize

  Classes in opennlp.tools.tokenize that implement Tokenizer
  Modifier and Type
  Class
  Description
  `class`
  `BertTokenizer`
  A `Tokenizer` implementation of the full BERT tokenization pipeline: basic tokenization (text normalization) followed by wordpiece tokenization.
  `class`
  `WhitespaceTokenizer`
  A basic `Tokenizer` implementation which performs tokenization using white spaces.
  `class`
  `WordpieceTokenizer`
  A `Tokenizer` implementation which performs tokenization using word pieces.