Re: check-tcg errors (build-user, build-user-plugins) again

[Claudio Fontana]

On 12/2/20 1:52 PM, Alex Bennée wrote:
> 
> Claudio Fontana <cfontana@suse.de> writes:
> 
>> On 12/2/20 12:16 PM, Alex Bennée wrote:
>>>
>>> Claudio Fontana <cfontana@suse.de> writes:
>>>
>>>> Hi Alex and all,
>>>>
>>>> when trying to use check-tcg (master), I am getting often these errors:
>>>>
>>>> $ ../configure --disable-system --disable-tools
>>>>
>>>> $ make -j12 check-tcg
>>>>
>>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found 
>>>> for group "claudio" in /etc/subgid 
>>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found 
>>>> for group "claudio" in /etc/subgid 
>>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found 
>>>> for group "claudio" in /etc/subgid 
>>>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11...
>>>> Trying to pull 
>>>> registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross...
>>>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian10...
>>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found 
>>>> for group "claudio" in /etc/subgid 
>>>>
>>>> [...]
>>>>   TEST    linux-test on x86_64
>>>> timeout: failed to run command 
>>>> ‘/home/claudio/git/qemu/build/qemu-x86_64’timeout: : No such file or 
>>>> directoryfailed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’
>>>>
>>>> [...]
>>>>
>>>>
>>>> Is there some pre-configuration on the host necessary to be able to
>>>> run check-tcg?
>>>
>>> There shouldn't be but those errors remind me of some of the tweaks I
>>> had to make to me Gentoo system when using podman (instead of docker).
>>> In the end I think I just ended up adding the lines:
>>>   
>>>   alex:100000:65536
>>>
>>> to /etc/subgid and /etc/subgid-
>>>
>>> Marc-André may have some better pointers as he added podman support to
>>> the builder scripts.
>>
>>
>> I did that and things seem a bit better, but still a lot of errors:
>>
>>
>> 63      ../sysdeps/x86_64/start.S: No such file or directory.
>>
>> Error: error creating build container: The following failures happened while 
>> trying to pull image specified by "debian:bullseye-slim" based on search 
>> registries in /etc/containers/registries.conf:
>> * "localhost/debian:bullseye-slim": Error initializing source 
>> docker://localhost/debian:bullseye-slim: error pinging docker registry 
>> localhost: Get https://localhost/v2/: dial tcp [::1]:443: connect: 
>> connection refused
>> * "docker.io/library/debian:bullseye-slim": Error committing the finished 
>> image: error adding layer with blob 
>> "sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": 
>> Error processing tar file(exit status 1): there might not be enough IDs 
>> available in the namespace (requested 0:42 for /etc/gshadow): lchown 
>> /etc/gshadow: invalid argument
>> Traceback (most recent call last):
>>   File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 
>> 709, in <module>
>>     sys.exit(main())
>>   File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 
>> 705, in main
>>     return args.cmdobj.run(args, argv)
>>   File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 
>> 501, in run
>>     extra_files_cksum=cksum)
>>   File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 
>> 354, in build_image
>>     quiet=quiet)
>>   File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 
>> 244, in _do_check
>>     return subprocess.check_call(self._command + cmd, **kwargs)
>>   File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
>>     raise CalledProcessError(retcode, cmd)
>>
>>
>> [...]
>> Error: error pulling image 
>> "registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross": unable to 
>> pull registry.gitlab.com/qemu-project/
>>
> 
> I'm guessing this can be fixed by adding gitlab to 
> /etc/containers/registries.conf

Hi Alex, I added:

[registries.search]
registries = ["docker.io", "registry.gitlab.com"]

I get:

Storing signatures
  Error processing tar file(exit status 1): there might not be enough IDs 
available in the namespace (requested 0:42 for /etc/gshadow): lchown 
/etc/gshadow: invalid argument
Error: error pulling image 
"registry.gitlab.com/qemu-project/qemu/qemu/debian11": unable to pull 
registry.gitlab.com/qemu-project/qemu/qemu/debian11: unable to pull image: 
Error committing the finished image: error adding layer with blob 
"sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": 
Error processing tar file(exit status 1): there might not be enough IDs 
available in the namespace (requested 0:42 for /etc/gshadow): lchown 
/etc/gshadow: invalid argument

Any idea?




> 
> I'll see if I can resurrect my podman setup because it was working before
> we added the caching from gitlab.
> 
>> [...]
>>
>>
>>
>>>
>>> The main difference between the images on the registry and the local
>>> versions is most add the current user so there is a clean mapping
>>> between the container user and the host file-system. It's the last step
>>> of the build so we still use the cached layers from the registry
>>> versions.
>>>
>>>> I see these errors in gitlab also for
>>>>
>>>> build-user
>>>> build-user-plugin
>>>>
>>>> Maybe this is what Philippe mentioned before though, that this is
>>>> expected at the moment due to a temporary Meson shortcoming?
>>>
>>> That is odd - I'm not seeing anything like that on the master builds:
>>>
>>>   https://gitlab.com/qemu-project/qemu/-/jobs/883985106
>>>   https://gitlab.com/qemu-project/qemu/-/jobs/883985113
>>>
>>> AFAIK GitLab is still using Docker to build it's containers (albeit with
>>> BUILDKIT enabled).
>>
>>
>> I am running again on gitlab the master branch, maybe there is something I 
>> need to fix, but to do that I need to enable check-tcg successfully I think.
>>
>> Thanks!
>>
>> Claudio
>>
>>>   
>>>>
>>>> Ciao,
>>>>
>>>> Claudio
>>>
>>>
> 
>