Re: [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics

[Thomas Huth]

On 20/05/2021 13.27, Philippe Mathieu-Daudé wrote:
> +Stefan/Daniel
>
> On 5/20/21 10:02 AM, Thomas Huth wrote:
> > On 19/05/2021 20.45, Philippe Mathieu-Daudé wrote:
> > > If a runner has ccache installed, use it and display statistics
> > > at the end of the build.
> > >
> > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > > ---
> > >    .gitlab-ci.d/buildtest-template.yml | 5 +++++
> > >    1 file changed, 5 insertions(+)
> > >
> > > diff --git a/.gitlab-ci.d/buildtest-template.yml
> > > b/.gitlab-ci.d/buildtest-template.yml
> > > index f284d7a0eec..a625c697d3b 100644
> > > --- a/.gitlab-ci.d/buildtest-template.yml
> > > +++ b/.gitlab-ci.d/buildtest-template.yml
> > > @@ -6,13 +6,18 @@
> > >          then
> > >            JOBS=$(sysctl -n hw.ncpu)
> > >            MAKE=gmake
> > > +        PATH=/usr/local/libexec/ccache:$PATH
> > >            ;
> > >          else
> > >            JOBS=$(expr $(nproc) + 1)
> > >            MAKE=make
> > > +        PATH=/usr/lib/ccache:/usr/lib64/ccache:$PATH
> >
> > That does not make sense for the shared runners yet. We first need
> > something to enable the caching there - see my series "Use ccache in the
> > gitlab-CI" from April (which is currently stalled unfortunately).
>
> TL;DR: I don't think we should restrict our templates to shared runners.

I'm certainly not voting for restricting ourselves to only use shared 
runners here - but my concern is that this actually *slows* down the shared 
runners even more! (sorry, I should have elaborated on that in my previous 
mail already)

When I was experimenting with ccache in the shared runners, I saw that the 
jobs are running even slower with ccache enabled as long as the cache is not 
populated yet. You only get a speedup afterwards. So if you add this now 
without also adding the possibility to store the cache persistently, the 
shared runners will try to populate the cache each time just to throw away 
the results afterwards again. Thus all the shared runners only get slower 
without any real benefit here.

Thus we either need to get ccache working properly for the shared runners 
first, or you have to think of a different way of enabling ccache for the 
non-shared runners, so that it does not affect the shared runners negatively.

 Thomas