Re: [Bug 1847232] [NEW] qemu TCG in s390x mode issue with calculating HASH

[David Hildenbrand]

On 14.10.19 12:22, David Hildenbrand wrote:
> On 14.10.19 11:53, David Hildenbrand wrote:
> > On 08.10.19 16:11, Ivan Warren wrote:
> > > On 10/8/2019 3:35 PM, David Hildenbrand wrote:
> > > > On 08.10.19 14:11, Cornelia Huck wrote:
> > > > > On Tue, 08 Oct 2019 11:19:25 -0000
> > > > > Ivan Warren via <address@hidden> wrote:
> > > > >
> > > > > > Public bug reported:
> > > > > >
> > > > > > When using go on s390x on Debian x64 (buster) (host) and debian s390x
> > > > > > (sid) (guest) I run into the following problem :
> > > > > >
> > > > > > The following occurs while trying to build a custom project :
> > > > > >
> > > > > > go: github.com/FactomProject/basen@v0.0.0-20150613233007-fe3947df716e:
> > > > > > Get
> > > > > > https://proxy.golang.org/github.com/%21factom%21project/basen/@v/v0.0.0-20150613233007-fe3947df716e.mod:
> > > > > > local error: tls: bad record MAC
> > > > > >
> > > > > > Doing a git bisect I find that this problem only occurs on and after
> > > > > > commit 08ef92d556c584c7faf594ff3af46df456276e1b
> > > > > >
> > > > > > Before that commit, all works fine. Past this commit, build always
> > > > > > fails.
> > > > > What version are you using? Current master?
> > > > >
> > > > > Can you please share your command line?
> > > > >
> > > > > > Without any proof, It looks like a hash calculation bug related to using
> > > > > > z/Arch vector facilities...
> > > > > Not an unreasonable guess, cc:ing David in case he has seen that before.
> > > > Can you reproduce with "-cpu qemu,vx=off" added to the QEMU command
> > > > line? Could be some fallout from vector instruction support. Currently
> > > > ill, will have a look when I'm feeling better.
> > >
> > > Reposted with a reply all... (sorry for the duplicates)
> > >
> > > So it does !
> > >
> > >
> > > My qemu command line is now (forget the odd funny networking things..)
> > >
> > > qemu-system-s390x \
> > >         -drive
> > > file=DEB002.IMG.NEW,discard=unmap,cache=writeback,id=drive-0,if=none \
> > >         -device virtio-scsi-ccw,id=virtio-scsi-0 \
> > >         -device scsi-hd,id=scsi-hd-0,drive=drive-0 \
> > >         -m 8G \
> > >         -net nic,macaddr=52:54:00:00:00:02 \
> > >         -net tap,ifname=taparm,script=no \
> > >         -nographic -accel tcg,thread=multi \
> > >         -monitor unix:ms,server,nowait \
> > >         -cpu qemu,vx=off \  ##### THAT WAS ADDED as instructed - without it
> > > everything goes kaput !
> > >         -smp 12
> > >
> > > And using the latest bleeding edge qemu from github, my build works (the
> > > problem goes away).
> > >
> > > So the z/Arch vector instructions may have a glitch is a venue to
> > > consider.. Probably one that couldn't be screened through conventional
> > > methods.
> > >
> > > I'm not that versed into z/Arch vector instruction, but if there
> > > anything I can help with, I will !
> >
> > I'll have to reproduce, can you outline the steps needed to trigger
> > this? (never had to build a go project before #luckyme ( ;) )). It looks
> > like https://github.com/FactomProject/basen is getting pulled in from
> > some other project?
>
> I just tried with Fedora 31 Nightly using "go get"
>
> [root@f31 ~]# go get -v -d github.com/FactomProject/factom
> github.com/FactomProject/factom (download)
> github.com/FactomProject/btcutil (download)
> github.com/FactomProject/ed25519 (download)
> github.com/FactomProject/go-bip32 (download)
> github.com/FactomProject/btcutilecc (download)
> package golang.org/x/crypto/ripemd160: unrecognized import path 
> "golang.org/x/crypto/ripemd160" (https fetch: Get 
> https://golang.org/x/crypto/ripemd160?go-get=1: local error: tls: bad record MAC)
> github.com/FactomProject/go-bip39 (download)
> package golang.org/x/crypto/pbkdf2: unrecognized import path 
> "golang.org/x/crypto/pbkdf2" (https fetch: Get 
> https://golang.org/x/crypto/pbkdf2?go-get=1: local error: tls: bad record MAC)
> github.com/FactomProject/go-bip44 (download)
> github.com/FactomProject/netki-go-partner-client (download)
> github.com/FactomProject/go-simplejson (download)
>
> With vx=off:
>
> [root@f31 ~]# go get -v -d github.com/FactomProject/factom
> github.com/FactomProject/factom (download)
> github.com/FactomProject/btcutil (download)
> github.com/FactomProject/ed25519 (download)
> github.com/FactomProject/go-bip32 (download)
> github.com/FactomProject/basen (download)
> github.com/FactomProject/btcutilecc (download)
> get "golang.org/x/crypto/ripemd160": found meta tag get.metaImport{Prefix:"golang.org/x/crypto", 
> VCS:"git", RepoRoot:"https://go.googlesource.com/crypto"} at 
> //golang.org/x/crypto/ripemd160?go-get=1
> get "golang.org/x/crypto/ripemd160": verifying non-authoritative meta tag
> golang.org/x/crypto (download)
> github.com/FactomProject/go-bip39 (download)
> github.com/FactomProject/go-bip44 (download)
> github.com/FactomProject/netki-go-partner-client (download)
> github.com/FactomProject/go-simplejson (download)
>
>
> That should be sufficient to identify the instruction. Might take some time, 
> though. E.g.,
> the HASH calculation in the kernel works just fine.

By now I am pretty sure the code that gets triggered is

src/vendor/golang.org/x/crypto/poly1305/sum_s390x.s

in the go repository.

I started writing unit tests for all involved vector instructions, no 
luck so far. Could also be some side-effect/BUG of another instruction.

Will let you know once I know more :)

--

Thanks,

David / dhildenb