Re: 'make check-acceptance' eats lots of disk space and never cleans it up

[Cleber Rosa]

On Tue, Aug 3, 2021 at 9:47 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Tue, 3 Aug 2021 at 13:58, Cleber Rosa <crosa@redhat.com> wrote:
> >
> > On Tue, Aug 3, 2021 at 8:43 AM Peter Maydell <peter.maydell@linaro.org> 
> > wrote:
> > >
> > > It looks like 'make check-acceptance' creates directories in
> > > build/clang/tests/results which are huge and which it never
> > > cleans up. For example one of my build directories (configured
> > > just for arm targets) has over 350 'job-[timestamp]' directories,
> > > many of which are 2.5GB or more in size.
> > >
> >
> > Every "job-[timestamp]" directory is the result of an "avocado run"
> > invocation, that is, one "make check-acceptance" command.
> >
> > > I assume most of this is artefacts (disk images etc) needed to
> > > rerun the tests. That's useful to keep around so you can manually
> > > run a test. However, we should be sharing this between runs, not
> > > creating a fresh copy for every time check-acceptance is
> > > run, surely ?
> > >
> >
> > They contain results and files needed for debugging the results of
> > tests, not artefacts needed to re-run them.  Everything that is
> > shareable is in the "~/avocado/data/caches" directory.
>
> This doesn't in practice seem to be the case. Picking a subdirectory
> at random:
>
> ./build/clang/tests/results/job-2021-07-30T11.20-63bd0a6/test-results/tmp_dir4_a3m36o/091-tests_acceptance_machine_sparc64_sun4u.py_Sun4uMachine.test_sparc64_sun4u/day23
>
> This contains (among other things) a vmlinux file which I assume is
> the one we run on the guest. It looks to me like this is a directory
> where we unzipped/untarred a downloaded file with the guest image.
>
> And another:
>
> ./build/clang/tests/results/job-2021-07-30T11.20-63bd0a6/test-results/tmp_dirwowk1bzp/026-tests_acceptance_boot_linux_console.py_BootLinuxConsole.test_arm_cubieboard_initrd/
>
> This seems to contain a rootfilesystem for some test or other,
> with a boot/, lib/, usr/, etc.
>
> These all look like artefacts to me, in the sense that they're
> the same every time.
>
> I notice that all these have 'tmp_dir*' directories in the paths. Is the
> problem just that we're failing to clean up a tempdir in some situations?
>

These are all directories meant to be temporary (the name gives it
away) and meant to be cleaned up.  You actually found a bug in the
"avocado_qemu.Test" class that is *not* calling the base
"avocado.Test" class tearDown().  It's a trivial one liner fix:

---

diff --git a/tests/acceptance/avocado_qemu/__init__.py
b/tests/acceptance/avocado_qemu/__init__.py
index 2c4fef3e14..1e807e2e55 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -276,6 +276,7 @@ def tearDown(self):
        for vm in self._vms.values():
            vm.shutdown()
        self._sd = None
+        super(Test, self).tearDown()

    def fetch_asset(self, name,
                    asset_hash=None, algorithm=None,

---

> thanks
> -- PMM
>

Thanks a lot for spotting that, I'll send a fix to the ML right away.

Best regards,
- Cleber.