Legend:
Library
Module
Module type
Parameter
Class
Class type
Library
Module
Module type
Parameter
Class
Class type
package bench
Benchmarking toolbox, based on the Criterion library by Bryan O'Sullivan
Easy Benchmarking
The following function suffices for most uses of this library. To benchmark the (unit -> unit) functions f,g,and h, the following will run the benchmark and print a comparison:
Bench.bench ["f",f; "g",g; "h",h] The number of iterations to run per sample and number of samples to take will be computed automatically.
Internal result format
module Bootstrap : sig ... endtype results = {desc : string;times : float array;mean : Bootstrap.estimate;stdev : Bootstrap.estimate;ov : float;
}The results of running a test
Detailed benchmarking functions
val bench_arg : (string * ('a -> 'b) * 'a) list -> results listbench fs benchmarks the named functions with argument given in fs
let res = Bench.bench_arg ["div", ( /. ) 3., 4.;
"mul", ( *. ) 3., 0.25] in
Bench.run_outputs resval bench_args : ('a -> 'b) -> (string * 'a) list -> results listBenchmark one function with many arguments.
let () = Bench.run_outputs (Bench.bench_args List.rev [
"10", Array.to_list (Array.create 10 0);
"100", Array.to_list (Array.create 100 0);
"1000", Array.to_list (Array.create 1000 0);
])val bench_funs : (string * ('a -> 'b)) list -> 'a -> results listBenchmark many functions with a fixed parameter. let l = Array.to_list (Array.init 1000 (fun _ -> Random.float 20.)) in let res = Bench.bench_funs
"listsort", List.sort compare;
"arrsort", (fun l -> let a = Array.of_list l in Array.sort compare a; Array.to_list a);
l in Bench.run_outputs res
val bench_n : (string * (int -> 'a)) list -> results listThis function benchmarks functions that take a single parameter of how many repetitions to run. This is slightly more accurate benchmark than others, as the function isn't wrapped by an extension function that calls it many times.
val bench_throughput : (int -> 'a) -> int list -> results listBenchmark a function for throughput. The function's argument is the block size it will work over, and the block sizes to test are given in a list. The results are in terms of throughput - the time taken divided by the size of the block.
val summarize : float -> results list -> unitThe function that summarizes the results by comparing the results linearly. The first parameter is the alpha (type I error rate) for the "same time" test, the second is the list of results from the tests.
Configuration API
type config = {mutable verbose : bool;(*bench will print much more progress information if this is true. Default=true.
*)mutable samples : int;(*The minimum number of samples to measure Default=1000
*)mutable gc_between_tests : bool;(*Whether or not to call
*)Gc.compactbetween tests Default=falsemutable resamples : int;(*The number of resamples to use when computing confidence intervals Default=1000
*)mutable confidence_interval : float;(*How big a confidence interval to estimate for mean and stdev Default: 95% (0.95)
*)mutable output : (results list -> unit) list;(*Output functions to use. Default:
*)summarize 0.05
}The following global configuration parameters are available for bench
val config : config