package autofonce_m4

v0.8 ( 2023-03-13 )

• Promotion (promote and run --auto-promote):

  • Fix --no-comment for promotion that was not used
  • Add argument --no-exit to promote, to promote everything except the exit code (so that the test will still fail with exitcode)
  • Don't promote failing AT_CHECK with run-if-fail or run-if-pass

• Configuration file:

  • New config option 'project.run_from' that specifies where _autofonce/ should be created: build for build directory (default), source for source directory or config for configuration file directory

• Command Arguments and Behavior:

  • Support for range of test ids like 1-10 or 10-
  • Exit with non-zero status on failed tests
  • If testsuite location is a directory instead of a file, scan all directoies and sub-directories for files *.at containing tests, with copying enabled (AT_COPY_ALL([true])).
  • New option -o LOGFILE to specify the log file
  • New command autofonce config to print the currently read config
  • New options -T <FILE.at>, -E <env.sh> and -I <path> to describe a testsuite directly from the command-line
  • Option -k KEYWORDS: if a keyword starts is negated with -, then tests matching that keyword will not be executed (same as --not)

• Improved version of _autofonce/results.log:

  • Include all files from failed checks and honor captured files
  • New option captured_files in autofonce.toml to include some project files

• Standard Macro Language:

  • Implements AT_FAIL_IF([shell-cond])
  • To conform with namespace rules for autoconf, all macros are supported with both a AT_ and AF_ prefix, and autofonce will promote autofonce specific macros with the AF_ prefix, while using the AT_ prefix for autoconf compatible macros

• Macro language extensions (default is AF_, but AT_ is accepted:

  • Skip macro definitions with AC_DEFUN/m4_define. This is currently mostly done to allow the definition of: AC_DEFUN([AF_ENV], [$1]). Print a warning for discarding the macro definition, except for AF_ENV.

  • Add AF_ENV([env]) as a way to define a specific environment for the test (such as environment variables).

    • At toplevel, declarations by AF_ENV are added in the autofonce_env.sh file of every test.
    • At test level, declarations are added directly in all following check scripts
  • Add AF_COPY(files) to copy files from the test source directory to the test run directory
  • Add AF_LINK(files), same as AF_COPY, but symlink instead of copy
  • Add AF_COPY_ALL([true|false]). If true, used before tests, make tests copy all non .at files in their directory as if AF_COPY had been used. If false, disable copying files for following tests.
  • Add AF_LINK_ALL(), same as AF_COPY_ALL, but symlink instead of copy
  • Add AF_SUBST(variables) to replace occurrences of these env variables in the stdout/stderr. A special case is AUTOFONCE that replaces ${AUTOFONCE_RUN_DIR}/${TEST_ID}, ${AUTOFONCE_BUILD_DIR} and ${AUTOFONCE_SOURCE_DIR}.
• Add a testsuite in test/testsuite.at

v0.7 ( 2023-02-09 )

• Add option --auto-promote MAX to autofonce run to iterate run/promote until all changes have been promoted or a maximal number of MAX iterations
• Add option --failures REASON to select tests matching a specific reason like exitcode, stdout or stderr
• Add config option build_dir_candidates (default to [ "_build" ]) to automatically try to find build anchors from outside the build dir using alternative candidates

v0.6 ( 2023-02-05 )

• Replace autofonce.env by a configuration file:

  • Configuration can be stored in autofonce.toml (typically not committed in sources) or .autofonce (committed in sources)
  • autofonce will lookup autofonce.toml first, and .autofonce if not found, and fail if none is found
  • autofonce init will create a autofonce.toml in the local directory.
  • A autofonce.toml file can be renamed to .autofonce and committed.

  • The configuration provides:

    • [testsuites] section: aliases for testsuites, with the testsuite file, the path for included files, and the env name
    • [envs] section: possibility to define different envs (included in tests scripts) for different testsuites
    • source_anchors and build_anchors are used to locate the project source dir and the project build dir in the upper directories

  • autofonce defines several variables at the beginning of test scripts envs:

    • AUTOFONCE_TESTSUITE: name of the testsuite
    • AUTOFONCE_RUN_DIR: directory where tests are run (containing `_autofonce/)
    • AUTOFONCE_SOURCE_DIR: source directory of the project (root/top dir)
    • AUTOFONCE_BUILD_DIR: build directory of the project

• The run-dir (the directory that will contain the _autofonce/ subdir) is determined by:

  • the directory containing the file autofonce.toml, or
  • the build directory as defined by the anchors in the .autofonce file

v0.5 ( 2023-02-04 )

• Add subcommand autofonce promote to update tests files with current results. This command can be applied using standard filters. By default, display a diff of changes to be performed. Use --apply to actually perform the tests.
• Add selector --failed to run only previously failed tests
• Scripts are run with AUTOFONCE_SUITE_DIR and AUTOFONCE_SUITE_FILE env variables (to locate and identify the testsuite.at file)
• Rename _autotest/ dir to _autofonce/ dir
• Simplify default gnucobol.env to use atconfig and atlocal: slower but probably more portable

v0.4 ( 2023-02-02 )

• Improve m4 parsing
• display nchecks performed
• allow if/then/else within tests
• min-edition set to 4.10

v0.3 ( 2023-01-28 )

• add autofonce new to create new tests

v0.2 ( 2023-01-28 )

• Improved documentation

v0.1 ( 2023-01-28 )

• Initial commit