Re: Packages grow, no longer fit on a 💾

[Efraim Flashner]

On Sat, Jan 14, 2023 at 11:07:23PM +0100, Ludovic Courtès wrote:
> Hello!
> 
> Over the course of a few years, the size of our packages has apparently
> kept growing.  Example:
> 
> --8<---------------cut here---------------start------------->8---
> $ guix time-machine --commit=v1.2.0 -- size emacs 
> store item                                                       total    self
> /gnu/store/118xpdazyylxa1rlc68h9lmh38vhxrb4-llvm-10.0.0            210.8   
> 139.3  16.2%
> /gnu/store/1qmd9achfkm1njzxf8hi86q53pmy9sxk-mesa-20.0.7            369.2   
> 131.3  15.3%
> /gnu/store/d83hc7cqgnhl1vlz1rv4ibkvaqaq2g35-emacs-27.1             859.7   
> 106.2  12.4%
> /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2            132.8    
> 53.2   6.2%
> /gnu/store/9lckq1194qcy4a7kv8bihagd58shj7yr-gtk+-3.24.20           723.7    
> 49.0   5.7%
> /gnu/store/qizplwwgqwyq6qmz1i6jlaib5kgzrgwq-icu4c-66.1             110.2    
> 38.1   4.4%
> /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31              38.4    
> 36.7   4.3%
> […]
> total: 859.7 MiB
> $ guix time-machine --commit=v1.3.0 -- size emacs 
> store item                                                       total    self
> /gnu/store/g3idjpqsp2p2d163qfzskxj4k58nrx7f-llvm-11.0.0            220.0   
> 148.6  16.9%
> /gnu/store/jf269s6clr6r57p8v5c3c1qkyra6apq2-mesa-20.2.4            389.1   
> 141.6  16.1%
> /gnu/store/7nlq2v4000pw8vgj3m4vrwz072ib58xh-emacs-27.2             880.5   
> 106.4  12.1%
> /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2            132.8    
> 53.2   6.0%
> /gnu/store/czf23hay13pp25yzrb4yq3n4gcri5r70-gtk+-3.24.24           744.3    
> 49.1   5.6%
> /gnu/store/2wqjj3mkqdvsvksndr2hpjpi7qqwi7kr-icu4c-66.1             110.2    
> 38.1   4.3%
> /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31              38.4    
> 36.7   4.2%
> […]
> total: 880.5 MiB
> $ guix time-machine --commit=v1.4.0 -- size emacs 
> store item                                                       total    self
> /gnu/store/i11jmrc90drpx9mc6cyb6s4pjny91qx6-emacs-28.2            1592.7   
> 250.6  15.7%
> /gnu/store/sdzfljm6san79pqiy42yp0nzmkr2bafc-mesa-21.3.8            411.6   
> 169.6  10.6%
> /gnu/store/6pdzpmxg5afzss6dlivq8z84sfa31x22-llvm-11.0.0            221.5   
> 149.5   9.4%
> /gnu/store/lphzb1z0r4kbb453rsvnw7msrxxzp5r7-libgccjit-10.3.0       244.8   
> 128.5   8.1%
> /gnu/store/pfas3c4lhr1jwkj2gl0yx8dg4xjjlp4r-mozjs-91.13.0          261.2    
> 65.4   4.1%
> /gnu/store/hy6abswwv4d89zp464fw52z65fkzr7h5-perl-5.34.0            147.7    
> 58.6   3.7%
> /gnu/store/zyqimpkmpffc24nwwqp46cicj8ss85y5-binutils-2.37          126.0    
> 54.4   3.4%
> /gnu/store/cnfsv9ywaacyafkqdqsv2ry8f01yr7a9-guile-3.0.7            129.1    
> 52.0   3.3%
> /gnu/store/4aq7hw017s9ihpm1rxpwfz28c80569z9-gtk+-3.24.30          1002.2    
> 48.6   3.1%
> /gnu/store/vaxkf8cbc7slgc3ikm5qr3815wih93w8-ghostscript-with-cups-9.54.0   
> 219.1    39.1   2.5%
> /gnu/store/bjycxjjkp1da53ijsa4hfdrz9mcgg55h-icu4c-69.1             110.7    
> 38.0   2.4%
> /gnu/store/ayc9r7162rphy4zjw8ch01pmyh214h82-glibc-2.33              76.6    
> 36.6   2.3%
> […]
> total: 1592.7 MiB
> --8<---------------cut here---------------end--------------->8---
> 
> I think something went wrong here.  :-)
> 
> In particular with Emacs itself (due to JIT + dependency on libgccjit,
> Binutils, etc.?) and with GTK+ (due to mozjs in polkit?).
> 
> And then MESA and LLVM have always been problematically big.
> 
> We should do better.  I mean, it’s just a text editor, isn’t it?

I've made some progress on LLVM and I think I have a working LLVM that
can be used as an input for mesa.

(ins)efraim@3900XT ~$ du -sch 
/gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/*
3.9M    /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/bin
8.0K    /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/etc
21M     /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/include
67M     /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/lib
16K     /gnu/store/36kmnxmb1h8pxw0x71wril67fdvjx7ny-llvm-11.0.0/share
92M     total

(define llvm-for-mesa
  (package/inherit llvm-11
    ;; If we can separate out the include directory we'd save another 21MB.
    (outputs (list "out"))
    (arguments
     (substitute-keyword-arguments (package-arguments llvm-11)
       ((#:configure-flags cf ''())
        ;; AMDGPU is needed by the vulkan drivers.
        `(list ,(string-append "-DLLVM_TARGETS_TO_BUILD="
                                (system->llvm-target) ";AMDGPU")
               "-DLLVM_BUILD_TOOLS=NO"
               "-DLLVM_BUILD_LLVM_DYLIB=YES"
               "-DLLVM_LINK_LLVM_DYLIB=YES"))
       ((#:phases phases)
        `(modify-phases ,phases
           (add-after 'install 'delete-static-libraries
             (lambda* (#:key outputs #:allow-other-keys)
               (for-each delete-file
                         (find-files (string-append
                                       (assoc-ref outputs "out") "/lib")
                                     "\\.a$"))))
           (replace 'install-opt-viewer
             (lambda* (#:key outputs #:allow-other-keys)
               (let ((out (assoc-ref outputs "out")))
                 (delete-file-recursively
                   (string-append out "/share/opt-viewer")))))
           (add-after 'install 'build-and-install-llvm-config
             (lambda* (#:key outputs #:allow-other-keys)
               (let ((out (assoc-ref outputs "out")))
                 (substitute*
                   "tools/llvm-config/CMakeFiles/llvm-config.dir/link.txt"
                   (("/tmp/guix-build-llvm-11.0.0.drv-0/build/lib")
                    (string-append out "/lib")))
                 (invoke "make" "llvm-config")
                 (install-file "bin/llvm-config"
                               (string-append out "/bin")))))))))))

In addition to what I have below I found that nix has a patch to make
llvm-11 (and the others) use the GNUInstallDirs, so we'd be able to move
the include directory to another output, saving another 21MB, bringing
llvm-for-mesa down to 71MB. Another possibility would be to have
llvm-for-mesa use llvm as an input, and then to use some configure-flags
to tell llvm-for-mesa to use the includes from llvm (the input).


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

signature.asc
Description: PGP signature