Adding a TOR system test - confusing results

[Chris Marusich]

Hi Guix,

In response to a recent email thread [1], I'd like to update our TOR
service to allow users to run TOR over a UNIX-domain socket.  I
discovered that TOR has no system tests, so I decided to start by
creating a basic system test to protect against regressions I might
accidentally introduce.

Unfortunately, the system test I've added behaves mysteriously.  It
succeeds when I think it obviously should not.  Could you please help me
understand what's going on?

Let's examine the problem.  I've attached two patches to this email.
They apply cleanly to 889e7fab3c04be98a59b880bf44fbdaa6ddf0a4e.

To begin, let's apply the first patch ("Add system tests for the TOR
service.").  Then run the test:

--8<---------------cut here---------------start------------->8---
$ make -j1 check-system TESTS=tor
[...]
# of expected passes      1
@ build-succeeded /gnu/store/zdh97qnh4b69yr23yp5js16hw9lvjhzm-tor-test.drv -
TOTAL: 1
PASS: /gnu/store/c3f0y7jbqi1qw8s63p3jbp5mrwl1bxnc-tor-test
--8<---------------cut here---------------end--------------->8---

It passed!  Great!  Now, let's make sure it also detects failures
correctly: Let's break TOR by applying the second patch ("For testing
purposes, break the TOR service.").  Then run it again:

--8<---------------cut here---------------start------------->8---
$ make -j1 check-system TESTS=tor
[...]
This is the GNU system.  Welcome.
komputilo login: exec of 
"/gnu/store/5jhlsv29n6kx76na1kkgfarznrl5kjjg-tor-0.3.3.9DOES-NOT-EXIST" failed: 
No such file or directory
QEMU runs as PID 4
connected to QEMU's monitor
read QEMU monitor prompt
connected to guest REPL
%%%% Starting test tor  (Writing full log to "tor.log")
marionette is ready
# of expected passes      1
@ build-succeeded /gnu/store/z8q9y8zivsipaq9fdj7yhhcvqgfq8ryl-tor-test.drv -
TOTAL: 1
PASS: /gnu/store/r84kh8x3d9ypqi6nl6r9xkqpk9i8jgk2-tor-test
[0] [env] address@hidden:~/guix
$ 
--8<---------------cut here---------------end--------------->8---

The test still passes, but I can't figure out why.  There is even an
error message in there, complaining that we can't exec the path to the
TOR program.  What is going on?  I've already tried rebuilding
everything from a clean slate, so that isn't the issue.

Any help would be greatly appreciated!  Thank you in advance,

Footnotes: 
[1]  https://lists.gnu.org/archive/html/help-guix/2018-07/msg00087.html

-- 
Chris

0001-Add-system-tests-for-the-TOR-service.patch
Description: Text Data

0002-For-testing-purposes-break-the-TOR-service.patch
Description: Text Data

signature.asc
Description: PGP signature