package windtrap

  1. Overview
  2. Docs
One library for all your OCaml tests

Install

dune-project
 Dependency

Authors

Maintainers

Sources

windtrap-0.2.0.tbz
sha256=2e61a86f8c97502c1f8a593c28e255d591a44b1533c6eed8cd0c847afa3d772a
sha512=060d0a47d926d420e0f96f4912c2690d148d6196bcbb701b2463a4fb9ffade5c4ff4d98d703f4b80b2b8ad1bed848c68f1d613e3ba745f9cd5e7d313e049bdc3

doc/windtrap.runtime/Windtrap_runtime/Verdicts/index.html

Module Windtrap_runtime.VerdictsSource

Mutation verdicts and the .mutants file that carries them.

Mutation verdicts: what a run made of each mutant, and the .mutants file that carries the verdicts of one executable to the merge.

The mutation loop writes one verdict file for each test executable. windtrap mutants loads the files of a project and combines them with merge, under which a mutant that is killed anywhere is killed. Nothing here runs when the module loads or when the process exits.

Verdicts

A verdict is one of five cases, and never a boolean or an exit code. A failure to supervise the child that arms a mutant is none of the five, and a caller must record no verdict for it.

Sourcetype reaching_test = string list

The type for a test that evaluated a mutant, by its path: the names of its groups, the outermost first, and then its own, as ["arithmetic"; "adds"].

Sourcetype verdict =
  1. | Killed
    (*

    A reaching test failed, or the child that armed the mutant crashed or hung.

    *)
  2. | Survived of {
    1. first : reaching_test;
    2. others : reaching_test list;
    }
    (*

    Every reaching test passed, and the armed site was evaluated. first and others are the reaching tests, sorted and without duplicates, and first is the first.

    *)
  3. | Not_evaluated
    (*

    Every reaching test passed, and the armed site was not evaluated. The dry run evaluated the site and the child did not, as when the dry run cached the result of the site in a table or a forced lazy value.

    *)
  4. | Outside_tests
    (*

    No test reached the site, and the dry run evaluated it outside every test, at module initialization or in a fixture release. The loop forks no child for the mutant.

    *)
  5. | Unreached
    (*

    No test reached the site, and the dry run did not evaluate it outside every test either. The loop forks no child for the mutant.

    *)

The type for verdicts. A survivor always names a reaching test, because a mutant that no test reached is Outside_tests or Unreached.

Sourceval survived : reaching_test list -> verdict

survived ts is the Survived verdict whose reaching tests are ts, sorted and without duplicates. Raises Invalid_argument if ts is empty.

Collections

Sourcetype record = {
  1. id : Mutate.id;
    (*

    The identifier of the mutant.

    *)
  2. before : string;
    (*

    The source text of the original expression.

    *)
  3. after : string;
    (*

    The source text of the armed expression.

    *)
  4. verdict : verdict;
    (*

    What the run made of the mutant.

    *)
}

The type for the record of one mutant. A record is self-describing: it carries the two renderings that a report draws the mutant with, so a report needs no catalogue and outlives the executable that produced it.

Sourceval record_of_mutant : Mutate.mutant -> verdict -> record

record_of_mutant m v is the record of m with the verdict v. m.dismissed is dropped.

Sourcetype t

The type for verdict collections: finite maps from Mutate.id to record, ordered by Mutate.compare_id. They are immutable.

Sourceval empty : t

empty is the collection with no record.

Sourceval add : t -> record -> t

add t r is t with r recorded. A Survived verdict is sorted on its way in. When t already holds a record under r.id, the smaller (before, after) pair in lexicographic order is kept, and the two verdicts combine into the verdict of a mutant that one executable saw one way and another the other way. It is Killed if either is. Otherwise it is Not_evaluated if either is, then Survived with the reaching tests of both if either is, then Outside_tests if either is, and Unreached if both are.

add checks nothing of r.id. It accepts an empty file, a line below 1, a negative col and a rewrite outside Mutate.rewrites, and load then refuses the file that save writes for the collection.

Sourceval records : t -> record list

records t is the records of t, ordered by Mutate.compare_id.

Sourceval merge : t -> t -> t

merge a b is the union of a and b, where two records that share an identifier are combined as add does. merge is commutative, associative and idempotent, empty is its unit, and an Unreached verdict leaves the verdict it is combined with unchanged, so merging any number of files in any order gives one answer.

Verdict files

An executable has one verdict file, at output_file.

The first line of a file is the magic line windtrap-mutants-v3, which carries the version of the format. A file with another first line is refused, never converted, and nothing is promised from one version to the next.

Sourcetype error = Instr.error =
  1. | Unknown_format of {
    1. path : string;
    2. header : string;
    }
  2. | Unreadable of {
    1. path : string;
    2. reason : string;
    }
  3. | Corrupt of {
    1. path : string;
    2. reason : string;
    }

The type for the errors of reading a verdict file, which is Instr.error, where each case is described.

Sourceval pp_error : Format.formatter -> error -> unit

pp_error ppf e is Instr.pp_error format ppf e.

Sourcetype identity = Instr.identity = {
  1. exe : string;
  2. digest : string;
}

The type for the identity of the writer of a verdict file, which is Instr.identity.

Sourceval format : Instr.format

format is the constants of the .mutants format: the magic line above, the data directory mutants, the extension mutants and the kind verdict.

Sourceval writer_identity : exe:string -> identity option

writer_identity ~exe is the identity to record for the executable at exe, whose exe is Instr.exe_identity ~exe, or None when exe cannot be read.

Sourceval output_file : exe:string -> string

output_file ~exe is Instr.output_file format ~exe, the one verdict file of the executable at exe. An executable that is renamed or moved leaves its previous file behind.

Sourceval load : string -> (t * identity option, error) result

load path is Ok (t, identity) when the file at path is a verdict file, where identity is the recorded writer, if any. The file is read with Instr.read_file, and every error names path. Otherwise it is the error of the read, Error (Unknown_format _) for another first line, or Error (Corrupt _) for data that is truncated or invalid:

  • a negative count, or a count larger than the input,
  • an empty file name, a line below 1 or a negative column,
  • a rewrite outside Mutate.rewrites, or an unknown verdict,
  • a survivor with no reaching test, or a second record for one identifier,
  • a malformed identity line, or bytes after the last record.

Nothing is repaired, and a file with one malformed record is not read at all.

Sourceval save : ?identity:identity -> string -> t -> unit

save ?identity path t writes t to path in the format of a verdict file, with Instr.write_file. Records are ordered by Mutate.compare_id and reaching tests are sorted, so equal collections give equal files. identity is recorded after the magic line when it is given. A merged collection has no single writer and is written without one.

load path is then Ok (t, identity) for every t whose identifiers load accepts, and Error (Corrupt _) for any other (see add). A collection built from the mutants of Mutate.catalogue round-trips, unless a source file was registered under the name "".

Raises Invalid_argument before anything is written if identity.exe is empty or if identity.digest is not 32 lowercase hexadecimal digits, and Sys_error if the file cannot be written.