Re: [Qemu-devel] Reporting Heisenbugs in qemu

[Mark Cave-Ayland]

On 07/05/13 12:38, Torbjorn Granlund wrote:

Hi Torbjorn,

> I am trying to use qemu to
>
> 1. cover more of the assembly code in GMP
> 2. check configuration logic of GMP
>
> but I am not as successful as I would like to be.
>
> The 2nd table of http://gmplib.org/devel/testsystems.html shows all
> emulated systems I am using, most of which are qemu-based.

Wow - that's a really impressive setup :)

> Unfortunately, several of the qemu-based systems experience intermittent
> but common segfaults:
>
> 1. Linux mips64eb 2.6.32-5-5kc-malta #1 Sun Sep 23 12:29:36 UTC 2012 mips64 
> GNU/Linux
> 2. Linux mips64el 2.6.32-5-5kc-malta #1 Fri Feb 15 21:38:11 UTC 2013 mips64 
> GNU/Linux
> 3. Linux kick.gmplib.org 2.6.18-6-sparc32 #1 Sat Dec 27 09:13:12 UTC 2008 sparc 
> GNU/Linux
>
> An example of a failure is:
>
> gmp/tests/cxx/t-ops2.cc: In function 'void checkz()':
> gmp/tests/cxx/t-ops2.cc:86: internal compiler error: Segmentation fault
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See<URL:http://gcc.gnu.org/bugs.html>  for instructions.
> For Debian GNU/Linux specific bug reporting instructions,
> see<URL:file:///usr/share/doc/gcc-4.1/README.Bugs>.
> The bug is not reproducible, so it is likely a hardware or OS problem.
>
> (This was from the sparc32 system.)
>
> rootrem.c: In function 'mpn_rootrem_internal':
> rootrem.c:120:1: internal compiler error: Segmentation fault
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See<file:///usr/share/doc/gcc-4.6/README.Bugs>  for instructions.
> The bug is not reproducible, so it is likely a hardware or OS problem.
>
> (From the mips64eb system.)
>
> I am aware of that these systems don't exactly use the
> kernel-of-the-week.  Newer kernels I have tried cause non-boot.  (I
> don't think I've tried any newer sparc kernel, as building that would
> require a stable sparc system...)
>
> I realise that linux might have been debugged until it works on real
> hardware, but that qemu might trigger untested linux execution paths.
>
> Yesterday, I disabled GMP testing on these qemu systems, as I got tired
> of the many false alarms, and since GMP looked bad.  Is there any hope
> that these qemu systems will become stable?  Or aren't these problems
> qemu's fault?

I actually spent quite a bit of time a couple of years ago trying to 
troubleshoot a similar intermittent buildfarm compiler blowout in a Xen 
x86 guest, and in the end it turned out to be simply a lack of memory 
(both physical and virtual) assigned to the VM.

It just so happened that the code in question was tickling a gcc bug 
which caused excess memory usage during compilation, but only when 
compiling with a certain combination of flags. This may not necessarily 
be the problem here, but it's certainly worth a little experimentation 
first before looking deeper into QEMU.


HTH,

Mark.