guix refresh -l

[Ludovic Courtès]

Jan Synáček <address@hidden> skribis:

> I debugged this and the upstream patch
> https://github.com/libarchive/libarchive/commit/b539b2e597b566fe3c4b49cb61c9eef83e5e052d
> fixes the problem. You will probably want to add it to the build process.

To determine how to handle the mass rebuild, I wanted to see how many
packages depend on libarchive.  However, ‘guix refresh -l’ would
underestimate that number because it ignores implicit dependencies, and
it turns out that libarchive is a dependency of CMake, which in turn is
an implicit dependency of many things.

So I thought (guix scripts graph) has the necessary infrastructure to
manipulate these graphs, including using the “bag” representation, so
why not use it.

Hence, commit 8fb5837 adds a generic (guix graph), 923d846 adds a couple
of helper procedures, and a51cbec rewrites ‘guix refresh -l’ in terms of
(guix graph).

Before the change:

--8<---------------cut here---------------start------------->8---
$ time guix refresh -l libarchive
Building the following 11 packages would ensure 18 dependent packages are 
rebuilt: ath9k-htc-firmware-1.4.0 totem-3.16.1 rhythmbox-3.2.1 diffoscope-34 
rdup-1.1.14 nestopia-ue-1.46.2 file-roller-3.10.0 gvfs-1.24.1 claws-mail-3.13.0 
zathura-cb-0.1.4 python-libarchive-c-2.1

real    0m13.443s
user    0m14.556s
sys     0m0.064s
--8<---------------cut here---------------end--------------->8---

After:

--8<---------------cut here---------------start------------->8---
$ time ./pre-inst-env guix refresh -l libarchive
Building the following 279 packages would ensure 620 dependent packages are 
rebuilt: python-libarchive-c-2.1 i3status-2.9 libwebsockets-1.3 vtk-6.1.0 
taskwarrior-2.4.3 synergy-1.7.4 guile-ssh-0.8.0 diffoscope-34 python-tlsh-3.4.1 
libpano13-2.9.19 slepc-complex-3.6.2 slepc-3.6.2 plink-1.07 apl-1.5 mumps-5.0.1 
mumps-metis-5.0.1 gmsh-2.8.4 dealii-8.2.1 julia-0.3.10 c-reduce-2.3.0 gcc-4.9.3 
clang-3.5.0 unionfs-fuse-0.26 unionfs-fuse-static-0.26 pflask-0.2 avrdude-6.1 
flashrom-0.9.7 oxygen-icons-4.14.2 r-servr-0.2 r-data.table-1.9.6 
r-htmlwidgets-0.5 r-dplyr-0.4.3 r-devtools-1.9.1 python2-rpy2-2.6.0 
python-rpy2-2.6.0 rsem-1.2.20 r-qtl-1.37-11 slepc-complex-openmpi-3.6.2 
mumps-openmpi-5.0.1 superlu-dist-3.3 dealii-openmpi-8.2.1 bless-1p02 
fftw-openmpi-3.3.4 guile-charting-0.2.0 accountsservice-0.6.40 
zynaddsubfx-2.5.2 non-sequencer-1.9.5-1d9bd576 emacs-pdf-tools-0.60 mupdf-1.6 
openimageio-1.5.18 ctl-1.5.2 slim-1.3.6 python2-scikit-image-0.11.3 
raincat-1.1.1.3 guile-sly-0.1 mesa-utils-8.2.0 i3-wm-4.10.3 xnee-3.19 
racket-6.2.1 sawfish-1.11 lxappearance-0.6.1 lxtask-0.1.6 pcmanfm-1.2.3 
lxrandr-0.3.0 sra-tools-2.5.4 htsjdk-1.129 ruby-atoulme-antwrap-0.7.5 
pspp-0.8.5 arandr-0.1.8 wicd-1.7.3 gourmet-0.17.4 gajim-0.16.3 patchage-1.0.0 
gpscorrelate-1.6.1.365f6e1b3f pinentry-0.9.5 xournal-0.4.8 glade-3.8.4 
lxterminal-0.2.0 gkrellm-2.3.5 geeqie-1.1 geda-gaf-1.8.2 dvdisaster-0.72.6 
azr3-1.2.3 calf-0.0.60 ir-1.3.2 gnubik-2.4.2 pcb-20140316 
gst-plugins-ugly-1.6.1 guix-0.9.0.5c36edc skribilo-0.9.2 a2ps-4.14 
emacs-w3m-1.4.483+0.20120614 orpheus-1.6 ripperx-2.8.0 emms-4.0 abcde-2.7 
mpd-mpc-0.27 mpdscribble-0.22 ncmpcpp-0.6.7 ncmpc-0.24 pidgin-otr-4.0.1 
libstdc++-doc-5.2.0 libstdc++-doc-4.9.3 manaplus-1.5.10.24 wayland-1.9.0 
fish-2.2.0 openbox-3.5.2 guile-present-0.3.0 amule-2.3.1 gmtp-1.3.9 
tuxguitar-1.2 conkeror-1.0pre1.20150730 lablgtk-2.18.3 gnubg-1.02 
inklingreader-0.8 gxmessage-3.4.3 pavucontrol-3.0 zathura-cb-0.1.4 
zathura-ps-0.2.2 zathura-pdf-poppler-0.2.5 zathura-djvu-0.2.4 vte-0.36.5 
tilda-1.3.0 gnome-keyring-3.16.0 gsegrafix-1.0.6 d-feet-0.3.10 seahorse-3.16.0 
gnome-terminal-3.16.0 gedit-3.18.1 gjs-1.44.0 xfce-4.12.0 devhelp-3.16.1 
yelp-3.16.1 shotwell-0.22.0 hexchat-2.10.1 claws-mail-3.13.0 
ibus-libpinyin-1.7.2 network-manager-applet-1.0.6 file-roller-3.10.0 
gnome-session-3.18.1.2 epiphany-3.16.3 nestopia-ue-1.46.2 gamine-1.4 sfxr-1.2.1 
transmission-2.84 gnome-mines-3.16.0 aisleriot-3.16.1 gnucash-2.6.9 
key-mon-1.17 eog-3.16.2 gnome-themes-standard-3.18.0 gnome-klotski-3.16.1 
gnome-settings-daemon-3.16.0 gnumeric-1.12.17 xboard-4.8.0 fvwm-2.6.5 hop-2.4.0 
patches-0.0.26d7dbc magit-svn-2.1.1 emacs-typo-1.1 emacs-flycheck-0.23 
emacs-butler-0.2.4 emacs-auctex-11.88.6 emacs-debbugs-0.7 
emacs-ob-ipython-20150704.8807064693 bigloo-really-sucks-4.2a.20141017 
guile-emacs-20150512.41120e0 emacs-no-x-toolkit-24.5 abiword-2.8.6 
emacs-foo-bar-patched-42 evince-3.18.1 rhythmbox-3.2.1 caribou-0.4.19 
totem-3.16.1 libchamplain-0.12.10 mutter-3.18.1 gimp-2.8.14 wesnoth-1.12.4 
gvfs-1.24.1 python2-ipython-3.2.1 python-numexpr-2.4.4 python-h5py-2.4.0 
python-biopython-1.65 python-statsmodels-0.6.1 python-scikit-image-0.11.3 
python-scikit-learn-0.16.1 python-seaborn-0.5.1 idr-2.0.0 python-ipython-3.2.1 
python2-numexpr-2.4.4 macs-2.1.0.20140616 seqmagick-0.6.1 crossmap-0.2.1 
rseqc-2.6.1 python2-statsmodels-0.6.1 miso-0.5.3 deeptools-1.5.11 
python2-warpedlmm-0.21 clipper-0.3.0 pbtranscript-tofu-2.2.3.8f5467fe6 
grit-2.0.2 couger-1.8.2 proof-general-4.2 enblend-enfuse-4.1.3 shogun-4.0.0 
xsensors-0.70 wxmaxima-15.04.0 fastcap-2.0-18Sep92 simple-scan-3.17.4 
hydra-20150407.4c0e3e4 gerbv-2.6.1 dunst-1.1.0 evas-generic-loaders-1.16.0 
terminology-0.9.1 rage-0.1.4 enlightenment-0.19.13 ath9k-htc-firmware-1.4.0 
fcitx-4.2.8.6 soprano-2.9.4 python2-pyqt-4.11.4 polkit-qt-1-0.112.0 
frescobaldi-2.18.1 keepassx-2.0-beta2 hydrogen-0.9.5.1 phonon-4.8.3 
attica-0.4.2 libdbusmenu-qt-0.9.2 brdf-explorer-17 gpsbabel-1.5.0 
librecad-2.0.6-rc alsa-modular-synth-2.1.1 jalv-1.4.6 ardour-4.4 
avidemux-2.6.10 scribus-1.5.0 python-pyqt-5.5 pumpa-0.9.1 owncloud-client-2.0.2 
lxqt-common-0.9.1 lxqt-session-0.9.0 tiled-0.13.1 bitcoin-core-0.11.0 
calibre-2.41.0 qsynth-0.4.0 bind-utils-9.10.2-P2 mariadb-10.0.20 conky-1.10.0 
powertabeditor-2.0.0-alpha8 libreoffice-5.0.0.2 evolution-data-server-3.18.2 
flexbar-2.5 express-1.5.1 solfege-3.22.2 wine-1.7.52 mplayer-1.2 
emotion-generic-players-1.16.0 cmus-2.7.1 mpd-0.19.10 strigi-0.7.8 
gst-libav-1.6.1 guile-gnunet-0.0.383eac2 audacity-2.1.0 synfigstudio-1.0.2 
retroarch-1.2.2 minetest-0.4.13 mars-0.7.5.1.c855d04409 love-0.9.2 mpv-0.13.0 
emulation-station-2.0.1 guitarix-0.34.0 alta-minimal-0.0 alta-clang-0.0 
dfc-3.0.4 alta-0.0 alta-gcc5-0.0 alta-gcc4.8-0.0 alta-gcc4.7-0.0 
libwebsockets-1.3.dbg rdup-1.1.14

real    0m0.979s
user    0m1.152s
sys     0m0.016s
--8<---------------cut here---------------end--------------->8---

Eric, I would welcome your feedback on this.  I would think the graph
helpers in (gnu packages) that were used by ‘guix refresh -l’ can now be
removed; WDYT?

There’s still room for improvement: ‘guix refresh -l’ could look at the
graph that includes origin objects and count the number of dependents of
the origin, rather than the number of dependents of one package that
uses it.  (Well, that’s the information we’re after when we want to
upgrade, but when we just want to change the package, that’s not what we
want.)

Cheers,
Ludo’.