Re: [PATCH v2 14/38] qapi/common.py: Convert comments into docstrings, and elaborate

[John Snow]

On 9/25/20 1:02 PM, Cleber Rosa wrote:
> On Wed, Sep 23, 2020 at 05:18:54PM -0400, John Snow wrote:
> > On 9/23/20 3:38 PM, Cleber Rosa wrote:
> > > On Tue, Sep 22, 2020 at 05:00:37PM -0400, John Snow wrote:
> > > > As docstrings, they'll show up in documentation and IDE help.
> > > >
> > > > Signed-off-by: John Snow <jsnow@redhat.com>
> > > > ---
> > > >    scripts/qapi/common.py | 51 ++++++++++++++++++++++++++++++------------
> > > >    1 file changed, 37 insertions(+), 14 deletions(-)
> > > >
> > > > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > > > index 0ce4a107e6..730283722a 100644
> > > > --- a/scripts/qapi/common.py
> > > > +++ b/scripts/qapi/common.py
> > > > @@ -20,10 +20,18 @@
> > > >    _C_NAME_TRANS = str.maketrans('.-', '__')
> > > > -# ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
> > > > -# ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
> > > > -# ENUM24_Name -> ENUM24_NAME
> > > >    def camel_to_upper(value: str) -> str:
> > > > +    """
> > > > +    Converts CamelCase to CAMEL_CASE.
> > > > +
> > > > +    Examples:
> > > > +      ENUMName -> ENUM_NAME
> > > > +      EnumName1 -> ENUM_NAME1
> > > > +      ENUM_NAME -> ENUM_NAME
> > > > +      ENUM_NAME1 -> ENUM_NAME1
> > > > +      ENUM_Name2 -> ENUM_NAME2
> > > > +      ENUM24_Name -> ENUM24_NAME
> > > > +    """
> > > >        c_fun_str = c_name(value, False)
> > > >        if value.isupper():
> > > >            return c_fun_str
> > > > @@ -45,21 +53,33 @@ def camel_to_upper(value: str) -> str:
> > > >    def c_enum_const(type_name: str,
> > > >                     const_name: str,
> > > >                     prefix: Optional[str] = None) -> str:
> > > > +    """
> > > > +    Generate a C enumeration constant name.
> > > > +
> > > > +    :param type_name: The name of the enumeration.
> > > > +    :param const_name: The name of this constant.
> > > > +    :param prefix: Optional, prefix that overrides the type_name.
> > > > +    """
> > > >        if prefix is not None:
> > > >            type_name = prefix
> > > >        return camel_to_upper(type_name) + '_' + c_name(const_name, 
> > > > False).upper()
> > > > -# Map @name to a valid C identifier.
> > > > -# If @protect, avoid returning certain ticklish identifiers (like
> > > > -# C keywords) by prepending 'q_'.
> > > > -#
> > > > -# Used for converting 'name' from a 'name':'type' qapi definition
> > > > -# into a generated struct member, as well as converting type names
> > > > -# into substrings of a generated C function name.
> > > > -# '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
> > > > -# protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
> > > >    def c_name(name: str, protect: bool = True) -> str:
> > > > +    """
> > > > +    Map `name` to a valid C identifier.
> > > > +
> > > > +    Used for converting 'name' from a 'name':'type' qapi definition
> > > > +    into a generated struct member, as well as converting type names
> > > > +    into substrings of a generated C function name.
> > > > +
> > > > +    '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
> > > > +    protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
> > > > +
> > > > +    :param name: The name to map.
> > > > +    :param protect: If true, avoid returning certain ticklish identifiers
> > > > +                    (like C keywords) by prepending ``q_``.
> > > > +    """
> > > >        # ANSI X3J11/88-090, 3.1.1
> > > >        c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
> > > >                         'default', 'do', 'double', 'else', 'enum', 'extern',
> > > > @@ -134,9 +154,12 @@ def decrease(self, amount: int = 4) -> int:
> > > >    indent = Indentation()
> > > > -# Generate @code with @kwds interpolated.
> > > > -# Obey indent, and strip EATSPACE.
> > > >    def cgen(code: str, **kwds: object) -> str:
> > > > +    """
> > > > +    Generate `code` with `kwds` interpolated.
> > > > +
> > > > +    Obey `indent`, and strip `EATSPACE`.
> > > > +    """
> > >
> > > This probably won't help on IDEs (never checked any), but sphinx will
> > > let you do:
> > >
> > >      """
> > >      Generate `code` with `kwds` interpolated.
> > >
> > >      Obey `indent`, and strip :data:`EATSPACE`.
> > >      """
> > >
> > > I'm not sure that a maximum level of docstring "sphinxzation" is the
> > > goal here, though.
> > >
> > > Reviewed-by: Cleber Rosa <crosa@redhat.com>
> >
> > It isn't yet, but I intend to address that when I remove missing-docstring
> > from pylint exemptions. Do I need :data: if I set the default role to 'any'?
>
> That's a good question.  According to the docs "any" will do its best,
> so it's probably a good fallback.  I do still favor using the correct
> role from the start if I can help it.
>
> > I'll probably try to enable sphinx at that time (and put the docs in a
> > devel/python manual?) and worry about the formatting at that point.
> >
> > --js
>
> Nice!
>
> - Cleber.

As of v3, I started toying with this, as you can see. It is a goal of 
mine to hit full doc coverage in this package, eventually.

What I learned: you can reference data members, but only if they have a 
comment. Otherwise, they are skipped. Sphinx does not appear to offer an 
"undocumented data member" option the same way it does for "undocumented 
member".

Using the "Any" role is nice, and I prefer it. If it finds that a target 
is ambiguous (2+ references), it will throw an error and the sphinx 
build will fail. This is good enough for me: there's no reason to 
clutter the docstrings with Sphinxese if we don't have to.

I would like to try and keep these readable to humans who are just in 
emacs/vim editing code, too.

--js