# Document Categorizer

## Classifying

The OpenNLP Document Categorizer can classify text into pre-defined categories. It supports multiple classification backends: Maximum Entropy (default), Support Vector Machines (SVM) via the `opennlp-ml-libsvm` module, and ONNX-based deep learning models via the `opennlp-dl` module. For someone interested in Gross Margin, the sample text given below could be classified as GMDecrease

    Major acquisitions that have a lower gross margin than the existing network
    also had a negative impact on the overall gross margin, but it should improve
    following the implementation of its integration strategies.

and the text below could be classified as GMIncrease

    The upward movement of gross margin resulted from amounts pursuant to
    adjustments to obligations towards dealers.

To be able to classify a text, the document categorizer needs a model. The classifications are requirements-specific and hence there is no pre-built model for document categorizer under OpenNLP project.

### Document Categorizer Tool

Note that ONNX model support is not available through the command line tool. The models that can be trained using the tool are OpenNLP models. ONNX models are trained through deep learning frameworks and then utilized by OpenNLP.

The easiest way to try out the document categorizer is the command line tool. The tool is only intended for demonstration and testing. The following command shows how to use the document categorizer tool.

    $ opennlp Doccat model

The input is read from standard input and output is written to standard output, unless they are redirected or piped. As with most components in OpenNLP, document categorizer expects input which is segmented into sentences.

### Document Categorizer API

To perform classification you will need a maxent model - these are encapsulated in the DoccatModel class of OpenNLP tools - or an ONNX model trained for document classification.

Using an OpenNLP model, first you need to grab the bytes from the serialized model on an InputStream:

    InputStream is = ...
    DoccatModel m = new DoccatModel(is);

With the DoccatModel in hand we are just about there:

    String inputText = ...
    String[] textTokens = inputText.split(" "); // split by whitespace
    DocumentCategorizerME myCategorizer = new DocumentCategorizerME(m);
    double[] outcomes = myCategorizer.categorize(textTokens);
    String category = myCategorizer.getBestCategory(outcomes);

#### Using an SVM Model

OpenNLP provides SVM-based document categorization through the `DocumentCategorizerSVM` class in the `opennlp-ml-libsvm` module. This implementation uses the zlibsvm library, an object-oriented Java binding for LIBSVM.

SVM text classification supports configurable term weighting strategies (binary, TF, TF-IDF, log-normalized TF), feature selection strategies (information gain, chi-square, term frequency, document frequency), feature scaling, and all SVM parameters (kernel type, cost, gamma, etc.).

To use SVM-based categorization, first add the dependency:

    <dependency>
        <groupId>org.apache.opennlp</groupId>
        <artifactId>opennlp-ml-libsvm</artifactId>
        <version>CURRENT_OPENNLP_VERSION</version>
    </dependency>

Training an SVM model from `DocumentSample` data:

    // Configure the SVM categorizer
    SvmDoccatConfiguration config = new SvmDoccatConfiguration.Builder()
        .setSvmConfiguration(new SvmConfigurationImpl.Builder()
            .setKernelType(KernelType.LINEAR)
            .setProbability(true)
            .build())
        .setTermWeightingStrategy(TermWeightingStrategy.TF_IDF)
        .setFeatureSelectionStrategy(FeatureSelectionStrategy.INFORMATION_GAIN)
        .setMaxFeatures(1000)
        .setScaleFeatures(true)
        .setScaleRange(0.0, 1.0)
        .build();

    // Train from a DocumentSample stream with a bag-of-words feature generator
    FeatureGenerator featureGenerator = new BagOfWordsFeatureGenerator();
    SvmDoccatModel model = DocumentCategorizerSVM.train("eng", sampleStream, config, featureGenerator);

    // Serialize the model
    try (OutputStream out = new FileOutputStream("doccat-svm.bin")) {
        model.serialize(out);
    }

Classifying text with the trained model:

    // Load the model
    SvmDoccatModel model;
    try (InputStream in = new FileInputStream("doccat-svm.bin")) {
        model = SvmDoccatModel.deserialize(in);
    }

    // Classify
    DocumentCategorizer categorizer = new DocumentCategorizerSVM(model, new BagOfWordsFeatureGenerator());
    double[] outcomes = categorizer.categorize(new String[]{"tokens", "of", "text"});
    String bestCategory = categorizer.getBestCategory(outcomes);

#### Using an ONNX Model

Using an ONNX model is similar, except we will utilize the `DocumentCategorizerDL` class instead. You must provide the path to the model file and the vocabulary file to the document categorizer. (There is no need to load the model as an InputStream as in the previous example.)

    File model = new File("/path/to/model.onnx");
    File vocab = new File("/path/to/vocab.txt");
    Map<Integer, String> categories = new HashMap<>();
    String[] inputText = new String[]{"My input text is great."};
    final DocumentCategorizerDL myCategorizer = new DocumentCategorizerDL(model, vocab, categories);
    double[] outcomes = myCategorizer.categorize(inputText);
    String category = myCategorizer.getBestCategory(outcomes);

For additional examples, refer to the `DocumentCategorizerDLEval` class.

## Training

The Document Categorizer can be trained on annotated training material. The data can be in OpenNLP Document Categorizer training format. This is one document per line, containing category and text separated by a whitespace. Other formats can also be available. The following sample shows the sample from above in the required format. Here GMDecrease and GMIncrease are the categories.

    MDecrease Major acquisitions that have a lower gross margin than the existing network also \
    	 had a negative impact on the overall gross margin, but it should improve following \
    	 the implementation of its integration strategies .
    GMIncrease The upward movement of gross margin resulted from amounts pursuant to adjustments \
    	 to obligations towards dealers .

The line breaks marked with a backslash are just inserted for formatting purposes and must not be included in the training data.

### Training Tool

The following command will train the document categorizer and write the model to en-doccat.bin:

    $ opennlp DoccatTrainer -model en-doccat.bin -lang en -data en-doccat.train -encoding UTF-8

Additionally it is possible to specify the number of iterations, and the cutoff.

### Training API

So, naturally you will need some access to many pre-classified events to train your model. The class opennlp.tools.doccat.DocumentSample encapsulates a text document and its classification. DocumentSample has two constructors. Each take the text's category as one argument. The other argument can either be raw text, or an array of tokens. By default, the raw text will be split into tokens by whitespace. So, let's say your training data was contained in a text file, where the format is as described above. Then you might want to write something like this to create a collection of DocumentSamples:

    DoccatModel model = null;
    try {
      ObjectStream<String> lineStream =
    		new PlainTextByLineStream(new MarkableFileInputStreamFactory(new File("en-sentiment.train")), StandardCharsets.UTF_8);

      ObjectStream<DocumentSample> sampleStream = new DocumentSampleStream(lineStream);

      model = DocumentCategorizerME.train("eng", sampleStream,
          TrainingParameters.defaultParams(), new DoccatFactory());
    } catch (IOException e) {
      e.printStackTrace();
    }

Now might be a good time to cruise over to Hulu or something, because this could take a while if you've got a large training set. You may see a lot of output as well. Once you're done, you can pretty quickly step to classification directly, but first we'll cover serialization. Feel free to skim.

    try (OutputStream modelOut = new BufferedOutputStream(new FileOutputStream(modelFile))) {
      model.serialize(modelOut);
    }

### Training an SVM Model

The SVM-based document categorizer uses the same `DocumentSample` training format as the Maximum Entropy categorizer. Training is done through the `DocumentCategorizerSVM.train()` method with a `SvmDoccatConfiguration` that controls the SVM parameters, term weighting, feature selection, and feature scaling.

A typical SVM text classification pipeline uses a LINEAR kernel with binary or TF-IDF term weighting and information gain or chi-square feature selection:

    SvmDoccatConfiguration config = new SvmDoccatConfiguration.Builder()
        .setSvmConfiguration(new SvmConfigurationImpl.Builder()
            .setKernelType(KernelType.LINEAR)
            .setProbability(true)
            .build())
        .setTermWeightingStrategy(TermWeightingStrategy.TF_IDF)
        .setFeatureSelectionStrategy(FeatureSelectionStrategy.INFORMATION_GAIN)
        .setMaxFeatures(1000)
        .setScaleFeatures(true)
        .build();

    SvmDoccatModel model = DocumentCategorizerSVM.train("eng", sampleStream, config,
        new BagOfWordsFeatureGenerator());

The resulting `SvmDoccatModel` can be serialized and deserialized using Java serialization:

    // Save
    model.serialize(new FileOutputStream("svm-doccat.bin"));

    // Load
    SvmDoccatModel loaded = SvmDoccatModel.deserialize(new FileInputStream("svm-doccat.bin"));