Re: Julia and Flux

[Cayetano Santos]

(reply to full mail list)

>dim. 31 mars 2024 at 15:48, Hugo Buddelmeijer <hugo@buddelmeijer.nl> wrote:

> Hi Cayetano,
>
> Disclaimer, I'm still learning Guix as well, and I'm not (yet) a Julia
> user. So here are my 2 cents, which might not be that useful.
>
> Thanks for the channel description. It is indeed possible to reproduce
> your error with those channels. It seems to be the Julia 1.9.3 in the
> guix-hpc channel that is the cause of the differences. Removing that
> channel will give you Julia 1.8 from the main guix channel, where Flux
> does seem to import.

Sure, there is no issue with main guix channel. The issue arises only with
guix-hpc channel.

> The main error message seems to be this:
>
>> Illegal inttoptr
>>  %magicptr = ptrtoint {} addrspace(10)* %242 to i64, !dbg !26
>
> And there are various issues (e.g. [1]) on github saying that this
> problem occurs because an incorrect version of LLVM is used, and that
> a specific "patched" version of LLVM is needed. This is about how far
> my investigation went.
>
> However, I've learned that using both Guix and a language's own
> package manager, can lead to problems. It might be better to create
> Guix packages for Flux and its dependencies. At least that is my plan,
> for my (Python) projects.

I see. Julia 1.8 uses llvm-julia, whereas julia 1.9 uses llvm-14-julia
in guix-hpc.

A. Out of a guix shell, when I compile something, I’m using my system
llvm, whatever it is, and this probably produces  some mess.

If you’re right, to fix the problem, it would be necessary to create a different
guix package to all my julia dependencies (and also to its
dependencies), using julia 1.9 instead of 1.8, as they currently do in
main channel.

At this point, I think it might be easier for all if julia 1.9 (and 1.10 at some
point), was merged into main guix repo, given that this problem will
arise with other packages too.

That being said, if you’re right, one will need to install a julia
package not in the guix repositories soon or later.

B.  Coming back to my original issue, I’m doing

    guix shell -C --network --emulate-fhs --link-profile julia nss-certs -- 
julia -e 'using Pkg; Pkg.add("Flux"); using Flux'

meaning that I’m using only guix provided packages and environment. This
implies julia itself, along with llvm-14-julia. The problem remains the
same, regardless of the system I’m using outside of the container environment.

> There is already a julia-zygote package in Guix itself. So perhaps you
> could already achieve what you need by using the julia from the main
> Guix channel, instead of the one from guix-hpc, because then you can
> use the packaged zygote.
>
> [1] "Zygote and Flux Illegal inttoptr"
> https://github.com/JuliaLang/julia/issues/50153
>
> Hope this was useful Cayetano; I doubt I can contribute more.
>
> Hugo

I’d prefer to use recent version  of julia, so main channel is not an
option for me.

Thanks anyway,

C.