prc

Utilities for precision-recall curves
README

prc implements a few functions to compute
Precision-Recall
curves, and their AUC (area under curve). The implementation is
checked against scikit-learn.

Installation

Latest release

opam install prc

Latest commit

opam pin add -y prc https://github.com/pveber/prc.git

Motivation

Precision-Recall curves are a useful representation of the performance
of binary classification methods in the case there are many negative
items. Problem is they are difficult to estimate from a finite
sample. As an illustration, compare the following graph obtained in
a particular case where the true precision-recall curve can be
computed analytically (so-called binormal model):

The red curve shows the "true" precision-recall curve while the grey
ones are the empirical estimated obtained from samples of
size 1000. We see that there is a lot of variability, particularly in
the low-recall region.

As a consequence, displaying a precision-recall curve can be
misleading, and it is safer to report an estimate of its area under
curve (AUC) along with a confidence interval. This is what this
library provides.

Quick test

Demo code is available in the demo directory. To run it, simply
launch an interpreter via a dune utop demo command, and then call
functions in Prc_demo. For instance to get an overview of the
sampling distribution of several estimators under the binormal model,
just type:

# Prc_demo.estimator_sampling_distribution ~sample_size:1000 ();;
``` This yields:

![demo](img/estimators_sampling_distribution.png)

where each boxplot represents the distribution of an estimator
(resp. binormal, trapezoidal and average precision) for a binormal
model when drawing finite samples of size 1000.

Install
Published
03 Nov 2021
Sources
prc-0.1.0.tbz
sha256=898322ef7f722e37f332362a24c1ad5f1bf8f8bb396cdb549fb44d59fc4f769a
sha512=c85fc65530195913b20c7b0e1763938f3d45187ca5510a37514f5dec5b4b903a9e40b7621ce8509cf023f1ccb1045fc09b362d716b2e6340cbd5960919a65efc
Dependencies
sklearn
with-test
ppx_deriving
with-test
dune
>= "2.4"
core_kernel
>= "v0.12.0"
alcotest
with-test
Reverse Dependencies