Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

Thank you for your thoughtful comments, Giovanni!

On 9/2/23 5:16 AM, Giovanni Biscuolo wrote:

> > 1. We should use sourcehut or continue to improve mumi
>
> Please forgive me if I insist, but the one and _only_ benefit of using
> SourceHut is the web-UI /helper/ to prepare an email message to send,
> it's "just" a web-UI version of the "git format-patch" CLI; the rest of
> the "patch management workflow" is email **and** CLI (git am) based;
> it's documented.

I enumerated the underlying issues that both my suggestions would address:

   - QA status should be visible from a patch's page
   - It should be possible to interact with the issue through the page

I know everyone is focusing on email vs. web-forge, but I am trying to 
draw attention to the root causes of the complexity, and enumerating 
possible solutions to these.

Please notice that under my first suggestion, I was calling out points 
about how QA status shows up, and switching between tools to interact 
with issues, not specifically how to prepare/apply patches.

> Furthermore, users that are comfortable with the SourceHut web UI are
> free to use that as their personal working repo, there is no need for
> Guix to use a SourceHut remote as the official one.

Defaults matter, and I don't view it as a valid starting-point to state 
that contributors can manage intrinsic complexity on their own. I think 
that's where we're at right now, and it's why I created this thread.

> >      - QA status should be visable from a patch's page
>
> On mumi web interface, in each issue page related to a patch, there is a
> "badge" linking to the QA status for that patch, right below the issue
> title; i.e.:
>
> https://issues.guix.gnu.org/65694
>
> have a link to https://qa.guix.gnu.org/issue/65694
>
> QA (and relates services, like data.qa) is a great project that could
> greatly improve current situation when completed!

Agreed! I called out Mumi as helpful but also highlighted some areas for 
improvement:

In a web-forge, I generally have a URL I can go to and see everything 
about my patch. I think we have that with https://issues.guix.gnu.org 
with two exceptions: (1) QA is a click away, or if you're using email, 
you're not even aware that there's a QA process failing (2) If you're 
not using email, context-switching between this page and email to respond.

> > - It should be possible to interact with the issue through the page
>
> I don't exactly understand: what do you mean with "interact"?

It should be possible to perform almost all meta-activities against an 
issue through its page:

- Manage status (close, re-open)
- Manage tags
- Request reviewers
- See CI status including high-level reason(s) for failure
- Apply to parent branch
- Respond to comments

> ...and what page?  https://issues.guix.gnu.org/ or
> https://qa.guix.gnu.org/issue/65694 (or any other issue)

Right now I'd hold up https://issues.guix.gnu.org/ as our best example.

> > 2. We should create scripts/sub-commands to lift contribution activities
> > into
> >      higher-order concepts:
> >      - Prepare a new submission
> >      - Run pre-checks on a submission
> >      - Submit a patch
> >      - Status of patch
>
> AFAIU you already use some of this "lifting" scripts od commands: can
> you please send patches so thay could possibly be included in Guix
> proper or in some section of the Cookbook?

It's not a stand-alone script that I can share unfortunately. It's 
functions integrated into a more complicated system. However, it's 
pretty straightforward to recreate: anywhere there's a decision-point, 
infer the correct decision (e.g. check dependencies on a package to 
determine which branch a commit should go in), execute the list of 
steps, give a human-friendly description of anything that went wrong, 
and offer suggestions on how to correct it.

> > For me, steps 20-23 are bothersome. There's a lot of "if" statements
> > that lead
> > to branching operations, and a lot of commands and flags to get
> > right.
>
> oh yes, CLI is a cognitive overhead sometimes, so we need better
> interfaces, some have found them

I loathe working with workflows that don't have integration points, and 
so I *love* CLI tools. But I don't like having to be the computer and 
walking through a workflow while filling all the blanks in.

> actually, point 21 "Run `./pre-inst-env ./etc/teams.scm cc-members
> <patch>` to get the CC flags for Git" is bothersome and we should find a
> way to better integrate that in "git format-patch" (so that will be
> automatically used in all the git interfaces we use)
>
> > The extra step to get a debbugs ID is annoying.
>
> have you tried mumi CLI with the new feature?

No, I've never used the mumi CLI.

> Forgive me if I insist: that forge site is _not_ SourceHut
>
> Second: each forge web site have a custom (not standard) way to manage
> pull-requests.

OK. I am not particularly attached to any one tool. I only care about 
reducing the complexity. I was responding to a message requesting 
specific suggestions, and Sourcehut has been brought up.

> Third: git have a pull-request mechanism [1] that could _easily_ be
> integrated in each and every forge, allowing projects to use
> /interoperable/ email based pull-request workflows if they want to.

Great! Let's talk about all the solutions so we find one that meets our 
defined criteria.

> IMO the
> core and most cognitive challenging steps of all "CI steps" are not if
> builds are done locally or not but (in order of importance):
>
> 1. having patches reviewed by humans, the "not automatable" part because
> Someone™ have to understand the _meaning_ of the patch and verify it
> conforms to the coding standards of the project, including "changelog
> style" commit messages;

I think the complexity required to contribute and having patches 
reviewed by humans are causally and cyclically linked. I would like to 
contribute much more to Guix, and others have said the same in this 
thread. Some of these issues we're discussing are preventing people from 
doing so.

I.e., would you rather have:

1. 10 very efficient contributors
2. 1,000 moderately efficient contributors

Wouldn't it be great if we had both?

> 2. understanding why build derivation fail when it fails.
>
> This is real cognitive overhead and this cannot be automated.

I disagree with this, but that's another conversation.

> > - Steps 19-23, or the "manage patch" steps.
> >
> >     I think an insight here is that the big button on forges is actually
> > a program
> >     removing the mental overhead for you.
>
> On the "web forges" vs "email based" patch workflow management I've said
> enough in other messages in this thread, here I just want to add
> (repeat) this: please do not only consider the mental overhead of
> potential contributors for "managing patches", also consider the mental
> overhead for patch reviewers; I've read many articles from professional
> patch reviewers that perfectly explains the great advanteges of using an
> email based workflow

It's a good point. We should remain focused on the overall complexity of 
the complete workflow of getting working code into Guix.

> >     I also don't usually have to worry nearly as much about crafting a commit
> >     message. So long as the title is under a character limit, and the body is
> >     helpful, it's OK. I think what bothers me most about the GNU changelog
> >     messages is that it's the worst of both spoken language and programming
> >     languages: there's an expectation of structure, but no grammar I can
> > parse
> >     against, and it's free-form.
>
> I'm sorry that the GNU policy about commit messages bothers you (on the
> contrary it makes me happy); please consider that thai is /just/ one of
> the policies of the Guix project: code of conduct, coding standards,
> others?

I recently responded to a similar question/point in another message from 
Simon about this being similar to style standard:

The distinction I draw is that I can usually run a linter against a 
coding style.

I don't care very much what the standard for commit messages is other 
than if it has an expectation of structure, I be able to run a tool to 
tell me if it's wrong.

In other words, the overhead isn't "I don't like this standard", it's "I 
can't find a way to reliably adhere to the standard".

> it's "just" an _interface_ issue

I think I would agree with this! I don't know that there's an inherent 
complexity in the individual steps. It's the aggregate of managing the 
workflow and tools that I think makes things complicated.