[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: unit tests best practice

From: Mike Gran
Subject: Re: unit tests best practice
Date: Thu, 3 Jan 2019 10:44:51 -0800
User-agent: Mutt/1.10.1 (2018-07-13)

On Thu, Jan 03, 2019 at 05:53:33PM +0100, Catonano wrote:
> Wat's the best practice to instrument a Guile based project for running
> unit tests ?
> guile-git has this fragment in the file
> SCM_LOG_DRIVER =                                \
>   $(top_builddir)/pre-inst-env                  \
>   $(GUILE) --no-auto-compile -e main            \
>       $(top_srcdir)/build-aux/test-driver.scm
> as you can see, what Automake produces calls a "main" procedure in a
> "test-driver.scm" script in the "build-aux" folder
> The effect is that you can call
> make check
> in the root folder and have the tests run
> Instead, in the file produced by guile-hall wired projects you
> ave this fragment
> AM_TESTS_ENVIRONMENT = abs_top_srcdir=\"$(abs_top_srcdir)\"
> SCM_LOG_COMPILER = $(top_builddir)/test-env $(GUILE)
> AM_SCM_LOG_FLAGS = --no-auto-compile -L \"$(top_srcdir)\"
> so there's a SCM_LOG_COMPILER instead of a SCM_LOG_DRIVER
> What is this LOG_COMPILER doing, exactly ?

OK, this is a muddle, but, it goes like this.

LOG_DRIVER and LOG_COMPILER come from Automake.  If you use them,
Automake will generate "make check" rules for all the test scripts in

When you use LOG_COMPILER, Automake considers a test to pass if it
returns an exit status of 0, perhaps by calling '(exit 0)' in Guile.
Exit code 1 = failure. 77 = skip.  99 = fatal eror.  So your TESTS
scripts as run by the command line in LOG_COMPILER need to return
proper exit codes.  Automake will also automatically gather anything
your tests print into .log files and .trs (test result files).

If you use LOG_DRIVER, Automake expects your scripts to manually
create its .log and .trs files.  You use LOG_DRIVER if the simple
exit-code-based pass/fail criteria isn't good enough.  Look
in the Automake docs under 'Custom Test Drivers' for info.

> This is what the test-env file contains
> #!/bin/sh
> \"@abs_top_builddir@/pre-inst-env\" \"address@hidden"
> exit $?
> what is this ?
> What does this do ?

Typically these pre-inst-env scripts set up the GUILE_LOAD_PATH for
when you are running the tests in the development tree before
installing them.  (There is no standard pre-inst-env script; there are
a few different versions.)  GUILE_LOAD_PATH works better with absolute
paths.  It may also set compiled paths, environment variables, or

> Alex admits he goes full cargo culting when dealing with the Autotools, in
> fact there's this comment line at the beginning of the section producing
> the Autotools related files in guile-hall
> ;;;; Full on cargo cult!
> So, if there's any kind soul here willing to clarify/illustrate what is
> going on we could make the effort to condensate such knowledge in
> guile-hall
> Improving the experience in dealing with Guile based project would be a
> great progress
> One last thing
> test-driver uses the API provided by srfi-64 to actually run the tests and
> display the results
> Does Guile provide a standard version of such script ?
> Should Guile provide one ?

I should note that srfi-64 doesn't work out of the box with
LOG_COMPILER because it doesn't return the proper exit status on
failure, either 0 = pass, 1 = fail, 77 on skip, 99 on hard error.  It
isn't that complicated to add that functionality in, and that would be
a great project for someone to do.

For the one project were I tried to make a full test suite
(guile-ncurses), I went with lots of tiny scripts and used the
LOG_COMPILER exit code strategy.  For that I make a small test library
here that converts #t and #f from a procedure into exit codes.


Mike Gran

reply via email to

[Prev in Thread] Current Thread [Next in Thread]