Re: [PATCH v3] meson: Pass -j option to sphinx

[Daniel P . Berrangé]

On Sat, Apr 29, 2023 at 02:33:17PM +0200, Paolo Bonzini wrote:
> On 4/28/23 17:01, Fabiano Rosas wrote:
> > Also make sure our plugins support parallelism and report it properly
> > to sphinx. Particularly, implement the merge_domaindata method in
> > DBusDomain that is used to merge in data from other subprocesses.
> > 
> > before:
> >    $ time make man html
> >    ...
> >    [1/2] Generating docs/QEMU manual with a custom command
> >    [2/2] Generating docs/QEMU man pages with a custom command
> > 
> >    real    0m43.157s
> >    user    0m42.642s
> >    sys     0m0.576s
> > 
> > after:
> >    $ time make man html
> >    ...
> >    [1/2] Generating docs/QEMU manual with a custom command
> >    [2/2] Generating docs/QEMU man pages with a custom command
> > 
> >    real    0m25.014s
> >    user    0m51.288s
> >    sys     0m2.085s
> 
> The 'nproc' fallback will potentially cause twice #CPUs processes to be
> active, since sphinx will run in parallel with everything else.
> 
> Is this result with "-j auto", and if so with which computer?  If the
> speedup is only 2x as it seems to be from the "time" above, I'd rather have
> "-j 2" only so that sphinx doesn't risk killing the machine...

Why would it kill the machine ? If there are two sphinx processes
concurrent, thus overcomitting available CPUs, the scheduler will
just end up giving them shorter timeslice OS. Given that the
sphinx parallelism seems to be very bursty when I monitored it,
I think having the high CPU counts is justified. The times when
both sphinx processes need all 8/16/whatever CPUs is relatively
unlikely to clash for prolonged periods.

What could kill the machine is if the RAM usage was excessive,
but I would thing we would see that already if it were a problem.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|