Re: [PATCH] gitlab-ci.yml: Add fuzzer tests

[Thomas Huth]

On 16/07/2020 18.46, Alexander Bulekov wrote:
> On 200716 1209, Thomas Huth wrote:
>> So far we neither compile-tested nor run any of the new fuzzers in our CI,
>> which led to some build failures of the fuzzer code in the past weeks.
>> To avoid this problem, add a job to compile the fuzzer code and run some
>> loops (which likely don't find any new bugs via fuzzing, but at least we
>> know that the code can still be run).
>>
>> A nice side-effect of this test is that the leak tests are enabled here,
>> so we should now notice some of the memory leaks in our code base earlier.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> 
> Thank you for this, Thomas. I just sent a patch to add a job that runs
> similar tests with the build-script that we use on oss-fuzz
> Patch: <20200716163330.29141-1-alxndr@bu.edu>

Good idea! ... but it does not work quite yet, I gave it a try and ended
up here:

 https://gitlab.com/huth/qemu/-/jobs/643353500

> I know that these jobs have a lot of overlap, but there are enough
> quirks in the oss-fuzz build-script that I, personally, don't think
> they are redundant.

I think we should finally go with your approach and compile the fuzzing
test via the script. But since that seems to need some more work first,
let's go with my patch now, so that we have something for 5.1-rc1, and
then when your patch is ready, replace my "build-fuzzer" job with yours, ok?

>> +build-fuzzer:
>> +  <<: *native_build_job_definition
>> +  variables:
>> +    IMAGE: fedora
>> +  script:
>> +    - mkdir build
>> +    - cd build
>> +    - ../configure --cc=clang --cxx=clang++ --enable-fuzzing
>> +                   --target-list=x86_64-softmmu
> 
> https://lists.nongnu.org/archive/html/qemu-devel/2020-07/msg02310.html
> When/if this gets merged, enable-fuzzing won't build with
> AddressSanitizer, by default, so I would add --enable-sanitizers, just
> to be safe. 

Ok, thanks, I'll add that.

>> +    - make -j"$JOBS" all check-build x86_64-softmmu/fuzz
>> +    - make check
>> +    - for fuzzer in i440fx-qos-fork-fuzz i440fx-qos-noreset-fuzz
>> +        i440fx-qtest-reboot-fuzz virtio-scsi-flags-fuzz virtio-scsi-fuzz ; 
>> do
> 
> Any reason for these particular fuzzers? I know the virtio-net ones find
> crashes pretty quickly, but I dont think that causes a non-zero exit.

I did not include the virtio-net fuzzers because I read that warning
"May result in network traffic emitted from the  process. Run in an
isolated network environment." in the help text ... so I was not sure
whether they are really suitable for the CI on the gitlab machines?

 Thomas