[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] gitlab-ci.yml: Use the whole tree as artifacts to speed up t

From: Thomas Huth
Subject: Re: [PATCH] gitlab-ci.yml: Use the whole tree as artifacts to speed up the CI
Date: Fri, 22 Jan 2021 17:25:19 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0

On 22/01/2021 11.18, Daniel P. Berrangé wrote:
On Fri, Jan 22, 2021 at 11:07:22AM +0100, Thomas Huth wrote:
Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files
as an artifact from the previous stage - which simply does not work
anymore (with the current version of meson). Due to some changed
time stamps, meson is always trying to rebuild the whole tree.

This used to work in the past didn't it ? Did something change in
meson to break this, or have we just not noticed before.

As mentioned by Philippe already, there seems to be a problem in current meson which triggers a rebuild of the whole tree in some cases. See this thread here:


So instead of trying to marry a freshly checked out source tree
with the pre-built binaries in these jobs, let's simply pass the
whole source including the submodules and the build tree as artifact
to the test jobs. That way timestamps get preserved and there is
no rebuild of the sources anymore. This saves ca. 15 - 20 minutes
of precious CI cycles in each run.

I'm a little worried we might end up hitting the artifact size
limit which is supposedly 1GB on gitlab.com.  Im guessing this
must be measuring the compressed size though, as a src checkout
with build dir  and .git dir is already way over 1GB.

I've added a "du -sh" in one of the jobs and the size of the directory was indeed 2 GB already. But I did not see any problems due to that size. So either the limit must be bigger, or it's about the compressed size. Either way, it's working at least right now.

We could maybe also simply exclude the .git directory of the source checkout, I think that's the biggest chunk there and likely not needed for the tests.

> For to ask, could we address it by using  'meson test
> --no-rebuild' perhaps ?

Interesting idea! I'll give it a try, and if it works, I'll send a patch for that instead.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]