Re: qemu-ppc64-linux-user: signal handler endianness in elfv2

[David Gibson]

On Mon, Feb 10, 2020 at 08:07:20AM -0600, Vincent Fazio wrote:
> David,
> 
> Thanks for the response.
> 
> On 2/10/20 1:08 AM, David Gibson wrote:
> > On Sat, Feb 08, 2020 at 12:57:40PM -0600, Vincent Fazio wrote:
> > > I ran into a crash scenario in ppc64-linux-user when trying to run openssl
> > > compiled with gcc & musl which emits an ELFv2 binary. The target CPU was a
> > > big endian model (e6500)
> > Uh.. so I'm pretty sure ABIv2 is little-endian specific.  If you want
> > BE binaries, I think you need to configure your toolchain differently.
> > 
> This is _generally_ the case. MUSL libc uses ELFv2 ABI for both big and
> little endian, so it's apparently assumed that gcc is set with
> --with-abi=elfv2. This is how it's currently configured with
> Buildroot.

Ah, and you did point that out in your original mail, but I missed it, sorry.

> 
> https://gitlab.com/vfazio/buildroot/commit/5ab751ca44e9da6f406876ff582b3a813056b0d4
> 
> OpenSSL recently had a similar assumption patched here: 
> https://github.com/openssl/openssl/commit/098404128383ded87ba390dd74ecd9e2ffa6f530
> 
> I imagine the most frequent test case is running ppc64le on an x86_64 host
> where the swap never even occurs due to the endianness match.
> 
> But if the fix is to just not do the swap for ELFv2 binaries because the
> function pointer is always aligned with the host's memory endianness due to
> how functions addresses are calculated in ELFv2, i don't mind submitting a
> patch.
> 
> If QEMU decides this is an unsupported case, that's probably fine because
> it's likely not in use in large scale.

Well, I'd like to get it working if we can.  I'm looking at this more
closely now.

> 
> > > Example:
> > > 
> > > vfazio@vfazio1 ~/development/buildroot/output/target :( $ gdb
> > > ~/development/qemu/ppc64-linux-user/qemu-ppc64
> > > (gdb) run -d guest_errors -cpu e6500 -E 
> > > LD_LIBRARY_PATH="/home/vfazio/development/buildroot/output/target/lib/:/home/vfazio/development/buildroot/output/target/usr/lib/"
> > > lib/libc.so bin/bash
> > > Starting program: 
> > > /home/vfazio/development/qemu/ppc64-linux-user/qemu-ppc64
> > > -d guest_errors -cpu e6500 -E 
> > > LD_LIBRARY_PATH="/home/vfazio/development/buildroot/output/target/lib/:/home/vfazio/development/buildroot/output/target/usr/lib/"
> > > lib/libc.so bin/bash
> > > [Thread debugging using libthread_db enabled]
> > > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> > > [New Thread 0x7ffff7ff9700 (LWP 27186)]
> > > invalid/unsupported opcode: 1f - 0c - 1a - 00 (7c005698) 0000000010028e58 > > > 0
> > > 
> > > Thread 1 "qemu-ppc64" received signal SIGSEGV, Segmentation fault.
> > > 0x00000000600a9257 in ldl_he_p (ptr=0x4c2c061000000000) at
> > > /home/vfazio/development/qemu/include/qemu/bswap.h:351
> > > 351        __builtin_memcpy(&r, ptr, sizeof(r));
> > > 
> > > (gdb) !readelf -h bin/bash | grep abi
> > >    Flags:                             0x2, abiv2
> > > (gdb) !readelf -h lib/libc.so | grep abi
> > >    Flags:                             0x2, abiv2
> > > <<<<<
> > > 
> > > openssl does "probing" for PPC features (Altivec, Crypto, etc) by trying 
> > > to
> > > execute possibly illegal instructions and handling them... hence that
> > > SIGILL. The problem is when it comes to jumping to the signal handler
> > > 
> > > #0  0x00000000600a9257 in ldl_he_p (ptr=0x4c2c061000000000) at
> > > /home/vfazio/development/qemu/include/qemu/bswap.h:351
> > > #1  0x00000000600a92fe in ldl_be_p (ptr=0x4c2c061000000000) at
> > > /home/vfazio/development/qemu/include/qemu/bswap.h:449
> > > #2  0x00000000600c0790 in translator_ldl_swap (env=0x62923150,
> > > pc=5488768711647035392, do_swap=false) at
> > > /home/vfazio/development/qemu/include/exec/translator.h:201
> > > #3  0x000000006011c1ab in ppc_tr_translate_insn (dcbase=0x7fffffffd250,
> > > cs=0x6291ae80) at 
> > > /home/vfazio/development/qemu/target/ppc/translate.c:7856
> > > #4  0x000000006005ae70 in translator_loop (ops=0x60805fc0 <ppc_tr_ops>,
> > > db=0x7fffffffd250, cpu=0x6291ae80, tb=0x60a5f900
> > > <static_code_gen_buffer+1681600>, max_insns=512)
> > >      at /home/vfazio/development/qemu/accel/tcg/translator.c:102
> > > <<<<<
> > > 
> > > The handler has what looks like an LE address but then swaps it since the
> > > host is LE and the target is BE
> > > 
> > > setup_rt_frame (sig=4, ka=0x628c5f80 <sigact_table+96>, info=0x62931658,
> > > set=0x7fffffffd2f8, env=0x62923150) at
> > > /home/vfazio/development/qemu/linux-user/ppc/signal.c:575
> > > 575            qemu_log_mask(LOG_GUEST_ERROR, "sa_handler NIP to "
> > > TARGET_FMT_lx "\n", ka->_sa_handler);
> > > (gdb) p/x ka->_sa_handler
> > > $11 = 0x10062c4c
> > > 579            env->nip = tswapl((target_ulong) ka->_sa_handler);
> > > (gdb) p/x env->nip
> > > $12 = 0x4c2c061000000000
> > > <<<<<
> > > 
> > > the memcpy later faults because that's obviously not a valid address from
> > > which to grab the PPC instructions that need to be translated.
> > > 
> > > I'm not familiar with the ELFv2 ABI or PPC assembly in general, so I'm not
> > > sure what to expect here. Typically ELFv2 was reserved for ppc64le, but 
> > > musl
> > > uses ELFv2 for all ppc64 targets. This likely wouldn't be an issue in
> > > ppc64LE since x86_64 is LE as well and no swap would take place.
> > > 
> > > Is the signal handler address tied to the endianess of the host? I noticed
> > > there was no swapping in elfload.c so wasn't sure...removing the swap gets
> > > me a little further in program execution but i do eventually crash (but 
> > > that
> > > may not be related to this specific issue).
> > > 
> > > Thanks
> > > 

-- 
David Gibson                    | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
                                | _way_ _around_!
http://www.ozlabs.org/~dgibson

signature.asc
Description: PGP signature