Supported Sphinx Versions (was: Re: [PATCH for-5.0? 0/3] Make docs build work with Sphinx 3)

[John Snow]

On 4/11/20 2:29 PM, Peter Maydell wrote:
> Our current docs don't build with Sphinx 3, as noted in
> https://bugs.launchpad.net/bugs/1872113 -- this is a combination of:
>  (1) we are using the sphinx-build -W option so warnings are treated
>      as errors
>  (3) a kernel-doc script bug meant it was omitting a close-paren
>      when a function parameter was a function pointer; older Sphinx
>      ignored this but Sphinx 3 parses the function declaration and
>      warns about it; and because of (1) this is fatal to the QEMU build
>  (2) Sphinx 3 makes a breaking change in how it wants C structs
>      to be marked up (moving from 'c:type:: struct Foo' to
>      'c:struct:: Foo'); our use of the old syntax provokes a
>      warning, which again because of point (1) is fatal
> 
> Patch 1 extends configure's --disable-werror to cover Sphinx as
> well as the C compiler, so that at least there is a workaround
> (which will be automatic for release builds).
> 
> Patch 2 fixes the trivial kernel-doc bug.
> 
> Patch 3 adds and uses a new --sphinx-version option to kernel-doc,
> so that our Sphinx plugin can pass the Sphinx version down and
> the script can then choose the right syntax.
> 
> I've marked this up as 'for-5.0?' because I think it would be
> nice if at least patch 1 went in. Patch 2 seems uncontroversial
> (though I guess we should forward it up to the kernel folks
> since kernel-doc is from them originally). Patch 3 is the
> expedient change, but you could argue about whether this is
> the best way to tell kernel-doc what to do.
> 
> thanks
> -- PMM
> 

I was curious about our actual version compatibility, so I did some testing.

I modified configure to prefer 'sphinx-build' over 'sphinx-build-3' so
it would use my venv version, and then;

>From my qemu build directory (~/src/qemu/bin/git):

> python3 -m venv 200
> source ./200/bin/activate.fish
> pip install sphinx==2.0.0
> ../../configure --target-list='x86_64-softmmu' --enable-debug
--enable-docs; and cat config-host.mak | grep sphinx; and make html;


Repeat the process for the major versions we believe that we support in
the abstract (1.3.x through 2.4.x):

1.3: Can't even pass the configure check with a blank document.

Exception occurred:
  File
"/home/jhuston/src/qemu/bin/git/130/lib64/python3.7/site-packages/sphinx/environment.py",
line 146, in __init__
    FileInput.__init__(self, *args, **kwds)
TypeError: __init__() got an unexpected keyword argument 'handle_io_errors'
The full traceback has been saved in /tmp/sphinx-err-owwisn63.log, if
you want to report the issue to the developers.

No idea.


1.4 - 1.4.9: Fails to build.

Warning, treated as error:
/home/jsnow/src/qemu/docs/system/images.rst:4: SEVERE: Duplicate ID:
"cmdoption-qcow2-arg-encrypt".

It doesn't seem to like the "encrypt.FOO" section names here and
considers them duplicates, cutting off at the '.'.

Not clear if there's a fix, or if it's worth caring about.


1.5 - 1.5.6: Fails to build.

Warning, treated as error:
/home/jsnow/src/qemu/docs/system/invocation.rst:544: WARNING: Malformed
option description '[enable=]PATTERN', should look like "opt", "-opt
args", "--opt args", "/opt args" or "+opt args"

... I actually don't know where this one goes wrong; that's not a valid
line number there. device-url-syntax.rst.inc isn't that long either, so
I don't know what this correlates to, unfortunately.

1.6.1 through 2.2.2 all appear to work just fine, but produce a lot of
warnings about a coming incompatibility with Docutils > 0.16.

2.3.0 - 2.4.4 work silently with Docutils 0.16.

3.0.0: Notably fails, as is the subject of this patch. :)




Conclusion:

Required: >= 1.6.1
Recommended: >= 2.3.0



We can make this a little easier by using python virtual environments as
part of our build tree; we can freeze version dependencies if we want to
get more reproducible python builds.

We might also begin "installing" the QAPI generator module into such a
virtual environment such that the include statements are written in a
more formal manner, which will assist for pylint and mypy gating, but
that's another email.


I want to send patches to:

1. Change configure to try and prefer a virtualenv version of
sphinx-build, when found

2. Change sphinx conf.py to require >= 1.6.1 so that the requirement is
documented

3. Update documentation (somewhere?) explaining our sphinx dependency
and which versions are required and why ("Because 1.5.x does not work
with our tree.")

--js