Re: [RFC] QEMU Gating CI

[Peter Maydell]

On Mon, 3 Feb 2020 at 03:28, Cleber Rosa <address@hidden> wrote
>    >>> Testing process:
>    >>>
>    >>>  * I get an email which is a pull request, and I run the
>    >>>    "apply-pullreq" script, which takes the GIT URL and tag/branch name
>    >>>    to test.
>    >>>  * apply-pullreq performs the merge into a 'staging' branch
>    >>>  * apply-pullreq also performs some simple local tests:
>    >>>     * does git verify-tag like the GPG signature?
>    >>>     * are we trying to apply the pull before reopening the dev tree
>    >>>       for a new release?
>    >>>     * does the pull include commits with bad UTF8 or bogus qemu-devel
>    >>>       email addresses?
>    >>>     * submodule updates are only allowed if the --submodule-ok option
>    >>>       was specifically passed
>
> These steps could go unchanged at this point.  One minor remark is
> that the repo hosted at gitlab.com would be used instead.  The
> 'staging' branch can be protected[4] so that only authorized people
> can do it (and trigger the pipeline and its jobs).
>
>    >>>  * apply-pullreq then invokes parallel-buildtest to do the actual
>    >>>    testing
>
> This would be done by GitLab instead.  The dispatching of jobs is
> based on the tags given to jobs and machines.  IMO at least the OS
> version and architecture should be given as tags, and the machine
> needs proper setup to run a job, such as having the right packages
> installed.  It can start with a proper documentation for every type of
> OS and version (and possibly job type), and evolve into scripts
> or other type of automation.
>
> These are usuall identical or very similar to what is defined in
> "tests/docker/dockerfiles", but need to be done at the machine level
> because of the "shell" executor.
>
>    >>>  * parallel-buildtest is a trivial wrapper around GNU Parallel which
>    >>>    invokes 'mergebuild' on each of the test machines
>    >>>  * if all is OK then the user gets to do the 'git push' to push the
>    >>>    staging branch to master
>
> The central place to check for success or failure would be the
> pipeline page.  Also, there's a configurable notification system that
> should (I've not tested it throughly) send failed and/or successful
> pipeline results to the pipeline author.  IIUC, this means whoever
> pushed to the 'staging' branch that caused the pipeline to be
> triggered.
>
> Let me know if this makes sense to you, and if so, we can arrange
> a real world PoC.  FIY, I've run hundreds of jobs in an internal
> GitLab instance, and GitLab itself (server and runner) seems very
> stable.

This all sounds like the right thing and great progress. So yes,
I agree that the next step would be to get to a point where you
can give me some instructions on how to say "OK, here's my staging
branch" and run it through the new test process and look at the
results.

thanks
-- PMM