Re: [PATCH] gitlab-ci.yml: Add fuzzer tests

[Alexander Bulekov]

On 200716 1209, Thomas Huth wrote:
> So far we neither compile-tested nor run any of the new fuzzers in our CI,
> which led to some build failures of the fuzzer code in the past weeks.
> To avoid this problem, add a job to compile the fuzzer code and run some
> loops (which likely don't find any new bugs via fuzzing, but at least we
> know that the code can still be run).
> 
> A nice side-effect of this test is that the leak tests are enabled here,
> so we should now notice some of the memory leaks in our code base earlier.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Thank you for this, Thomas. I just sent a patch to add a job that runs
similar tests with the build-script that we use on oss-fuzz
Patch: <20200716163330.29141-1-alxndr@bu.edu>

I know that these jobs have a lot of overlap, but there are enough
quirks in the oss-fuzz build-script that I, personally, don't think
they are redundant.

A couple notes below, and I haven't been able to test on my own fork of
qemu on gitlab, yet due to some pipeline errors, but otherwise

Reviewed-by: Alexander Bulekov <alxndr@bu.edu>

> ---
>  .gitlab-ci.yml | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 5eeba2791b..e96f8794b9 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -161,9 +161,27 @@ build-clang:
>      IMAGE: fedora
>      CONFIGURE_ARGS: --cc=clang --cxx=clang++
>      TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
> -      ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user
> +      ppc-softmmu s390x-softmmu arm-linux-user
>      MAKE_CHECK_ARGS: check
>  
> +build-fuzzer:
> +  <<: *native_build_job_definition
> +  variables:
> +    IMAGE: fedora
> +  script:
> +    - mkdir build
> +    - cd build
> +    - ../configure --cc=clang --cxx=clang++ --enable-fuzzing
> +                   --target-list=x86_64-softmmu

https://lists.nongnu.org/archive/html/qemu-devel/2020-07/msg02310.html
When/if this gets merged, enable-fuzzing won't build with
AddressSanitizer, by default, so I would add --enable-sanitizers, just
to be safe. 

> +    - make -j"$JOBS" all check-build x86_64-softmmu/fuzz
> +    - make check
> +    - for fuzzer in i440fx-qos-fork-fuzz i440fx-qos-noreset-fuzz
> +        i440fx-qtest-reboot-fuzz virtio-scsi-flags-fuzz virtio-scsi-fuzz ; do

Any reason for these particular fuzzers? I know the virtio-net ones find
crashes pretty quickly, but I dont think that causes a non-zero exit.

> +          echo Testing ${fuzzer} ... ;
> +          x86_64-softmmu/qemu-fuzz-x86_64 --fuzz-target=${fuzzer} -runs=1000
> +            || exit 1 ;
> +      done
> +
>  build-tci:
>    <<: *native_build_job_definition
>    variables:
> -- 
> 2.18.1
>