Re: fuzzing poke with afl & afl++

[Jose E. Marchesi]

Hi Dan!

Sorry for the delay on replying to this.  Your fuzzing work is
impressive :)

    I've had some time during the last days and used it to run some fuzzing
    tests on poke (essentially I've let poke get bombarded by tons of
    mutated input files and watched out for "interesting" things to happen,
    e.g. crashes).
    
    If you are interested in the technical details: I've written them down
    into the `fuzz` subdirectory in the defolos/fuzzing branch.

Neat.  If you would like to maintain it and working on it further, it
would be great to have fuzz/ in the master branch.

I would move the contents of fuzz/README.rst to a chapter in HACKING.
WDYT?

    Long story short: I've let afl++ fuzz poke's script mode overnight on my
    laptop (12 fuzzing processes were running, one in deterministic mode and
    11 in random mode) for about 5 hours until poke filled up my /tmp/ and
    the fuzzer died (because it was running on the same partition to not
    murder my SSD). It found 600-something input files that made poke crash,
    but of these only 164 are distinct crashes (distinct according to afl,
    at least the failed assertions could be maybe only a handful of
    bugs). I've put them into the following categories according to the type
    of crash that occurred:
    
    assertion failed: 122
    double free:      11
    signal FPE:       2
    signal SEGV:      19
    use after free:   8
    misc:             2 (these just result in an "Aborted" message)
    
    The resulting files are attached in the archive crashes.tar.gz.

Allright, we definitely want to file bugs for these bugs.  I will look
at crashes.tar.gz and do so.

Good that we got a bugzilla ;)
    
    I have one suggestion though: could we add a flag to poke (either
    compile- or run-time), so that it will not create diagnostic files in
    /tmp/? This is not really useful when running poke a few million times
    with mostly invalid input.

Definitely yes.  Will add it.
Thanks!