bug#22533: Python bytecode reproducibility

[Ricardo Wurmus]

Ricardo Wurmus <address@hidden> writes:

> Ricardo Wurmus <address@hidden> writes:
>
>> Unfortunately, this doesn’t fix all reproducibility problems with numpy:
>>
>> --8<---------------cut here---------------start------------->8---
>> Binary files 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/distutils/__pycache__/__config__.cpython-36.pyc
>>  and 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/distutils/__pycache__/__config__.cpython-36.pyc
>>  differ
>> Binary files 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/distutils/__pycache__/exec_command.cpython-36.pyc
>>  and 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/distutils/__pycache__/exec_command.cpython-36.pyc
>>  differ
>> Binary files 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/distutils/__pycache__/system_info.cpython-36.pyc
>>  and 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/distutils/__pycache__/system_info.cpython-36.pyc
>>  differ
>> Binary files 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/__pycache__/__config__.cpython-36.pyc
>>  and 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/__pycache__/__config__.cpython-36.pyc
>>  differ
>> Binary files 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/__pycache__/version.cpython-36.pyc
>>  and 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/__pycache__/version.cpython-36.pyc
>>  differ
>> Binary files 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/testing/nose_tools/__pycache__/utils.cpython-36.pyc
>>  and 
>> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/testing/nose_tools/__pycache__/utils.cpython-36.pyc
>>  differ
>> --8<---------------cut here---------------end--------------->8---
>
> Here’s what diffoscope says:
>
> --8<---------------cut here---------------start------------->8---
> diffoscope 
> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0{-check,}/lib/python3.6/site-packages/numpy/__pycache__/version.cpython-36.pyc
> --- 
> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0-check/lib/python3.6/site-packages/numpy/__pycache__/version.cpython-36.pyc
> +++ 
> /gnu/store/kd06ql8fynlydymzhhnwk2lh0778dwcc-python-numpy-1.14.0/lib/python3.6/site-packages/numpy/__pycache__/version.cpython-36.pyc
> @@ -1,8 +1,8 @@
> -00000000: 330d 0d0a fa87 9c5a 2601 0000 e300 0000  3......Z&.......
> +00000000: 330d 0d0a c485 9c5a 2601 0000 e300 0000  3......Z&.......
>  00000010: 0000 0000 0000 0000 0001 0000 0040 0000  address@hidden
>  00000020: 0073 2000 0000 6400 5a00 6400 5a01 6400  .s ...d.Z.d.Z.d.
>  00000030: 5a02 6401 5a03 6402 5a04 6504 731c 6502  Z.d.Z.d.Z.e.s.e.
>  00000040: 5a01 6403 5300 2904 7a06 312e 3134 2e30  Z.d.S.).z.1.14.0
>  00000050: da28 3639 3134 6262 3431 6630 6662 3363  .(6914bb41f0fb3c
>  00000060: 3162 6135 3030 6261 6534 6537 6436 3731  1ba500bae4e7d671
>  00000070: 6461 3935 3336 3738 3666 544e 2905 da0d  da9536786fTN)...
> --8<---------------cut here---------------end--------------->8---
>
> In other words: this is the timestamp field of the pyc file.
>
> Maybe this can be avoided by setting DETERMINISTIC_BUILD in the
> python-build-system?

It cannot.

So, something’s still missing from my patch.  Does anyone see what might
be missing?

-- 
Ricardo