org.apache.lucene.benchmark.quality (Lucene 10.5.1 benchmark API)
Description: declaration: package: org.apache.lucene.benchmark.quality

# Package org.apache.lucene.benchmark.quality

package org.apache.lucene.benchmark.quality
Search Quality Benchmarking.

This package allows to benchmark search quality of a Lucene application.

In order to use this package you should provide:

- A IndexSearcher.
- `Quality queries`.
- `Judging object`.
- `Reporting object`.

For benchmarking TREC collections with TREC QRels, take a look at the `trec package`.

Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:

         File topicsFile = new File("topics-701-850.txt");
         File qrelsFile = new File("qrels-701-850.txt");
         IndexReader ir = DirectoryReader.open(directory):
         IndexSearcher searcher = new IndexSearcher(ir);

         int maxResults = 1000;
         String docNameField = "docname";

         PrintWriter logger = new PrintWriter(System.out,true);

         // use trec utilities to read trec topics into quality queries
         TrecTopicsReader qReader = new TrecTopicsReader();
         QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile)));

         // prepare judge, with trec utilities that read from a QRels file
         Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile)));

         // validate topics & judgments match each other
         judge.validateData(qqs, logger);

         // set the parsing of quality queries into Lucene queries.
         QualityQueryParser qqParser = new SimpleQQParser("title", "body");

         // run the benchmark
         QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField);
         SubmissionReport submitLog = null;
         QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger);

         // print an average sum of the results
         QualityStats avg = QualityStats.average(stats);
         avg.log("SUMMARY",2,logger, "  ");

Some immediate ways to modify this program to your needs are:

- To run on different formats of queries and judgements provide your own `Judge` and `Quality queries`.
- Create sophisticated Lucene queries by supplying a different `Quality query parser`.

- Related Packages
  Package
  Description
  org.apache.lucene.benchmark
  Lucene Benchmarking Package
  org.apache.lucene.benchmark.quality.trec
  Utilities for Trec related quality benchmarking, feeding from Trec Topics and QRels inputs.
  org.apache.lucene.benchmark.quality.utils
  Miscellaneous utilities for search quality benchmarking: query parsing, submission reports.
  org.apache.lucene.benchmark.byTask
  Benchmarking Lucene By Tasks
  org.apache.lucene.benchmark.utils
  Benchmark Utility functions.
- Class
  Description
  Judge
  Judge if a document is relevant for a quality query.
  QualityBenchmark
  Main entry point for running a quality benchmark.
  QualityQuery
  A QualityQuery has an ID and some name-value pairs.
  QualityQueryParser
  Parse a QualityQuery into a Lucene query.
  QualityStats
  Results of quality benchmark run for a single query or for a set of queries.
  QualityStats.RecallPoint
  A certain rank in which a relevant doc was found.