How to transfer everything from /gnu/store from one system to another?

[Rodrigo Morales]

Table of Contents
_________________

1. The context
2. What I know
.. 1. Transfer packages by name
.. 2. Transfer paths under /gnu/store
.. 3. Transfer everything in /gnu/store from one system to another
3. The question

1 The context
=============

  I'm currently using two systems. System A (hostname: delta, ip:
  196.168.1.42) is being used as my main station. System B (hostname:
  desktop, ip: 192.168.1.34) is used for installing and reinstalling
  Guix System multiple times since I'm testing system B's hardware.

  In order not to overload Guix servers by requesting the same packages
  multiple times and to download packages faster, I'm using guix publish
  in system A and I'm serving system B with packages coming from system
  A's store.

  In one of my experiments, I installed multiple kernels in system B and
  now I want to transfer them to system A's store.

  I want to transfer them from system B to system A because I'm going to
  wipe all partitions in system B and install GuixSD again, but I don't
  want to spend resources and time again building the same kernels, so
  before wiping all partitions in system B I want to save in system A
  the kernels and related packages that I installed in system B.


2 What I know
=============

2.1 Transfer packages by name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  I know that `guix copy' can be used to transfer packages by name
  either from system A to system B or conversely using the flags
  `--from' or `--to'.

  ,----
  | rdrg@desktop ~$ guix copy --to=rdrg@192.168.1.42 maxima
  | guix copy: sending 2 store items (128 MiB) to '192.168.1.42'...
  | /gnu/store/vmbfzagscqm647m5liycnqznk6lpp85l-gnuplot-5.4.6
  | /gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0
  | rdrg@desktop ~$ echo $?
  | 0
  `----

  I can confirm that this works because in system A I had never
  installed `maxima' but after using `guix copy' in system B, the
  directory now exists in system A.

  ,----
  | rdrg@delta ~/my$ find /gnu/store -mindepth 1 -maxdepth 1 -name
'*maxima*'
  | /gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0
  | rdrg@delta ~/my$ echo $?
  | 0
  | rdrg@delta ~/my$ find
/gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0 -mindepth 1
-maxdepth 1
  | /gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0/bin
  | /gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0/share
  | /gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0/libexec
  | /gnu/store/zc9dwfim9zy0d8ccmminvzpismx4laiz-maxima-5.46.0/lib
  | rdrg@delta ~/my$ echo $?
  | 0
  `----


2.2 Transfer paths under /gnu/store
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  I also know that `guix copy' can be used to transfer paths under
  `/gnu/store'.

  ,----
  | rdrg@desktop ~$ guix copy --to=rdrg@192.168.1.42
/gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00
  | guix copy: sending 1 store item (4 MiB) to '192.168.1.42'...
  | /gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00
  | rdrg@desktop ~$ echo $?
  | 0
  `----

  Similarly, I can confirm that this works because in system A I had
  never installed `transmission' and after `guix copy' finished, the
  path that existed in system B now exists in system A.

  ,----
  | rdrg@delta ~/my$ find
/gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00 -mindepth 1
-maxdepth 1
  | /gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00/bin
  | /gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00/share
  | /gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00/etc
  | /gnu/store/wi3v2br7d4yh2mbj9ni7y6fbjbqvcmc5-transmission-3.00/lib
  | rdrg@delta ~/my$ echo $?
  | 0
  `----


2.3 Transfer everything in /gnu/store from one system to another
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  Because it is possible to transfer paths using `guix copy', I can
  iterate through all directories in `/gnu/store' using `find' in system
  B and transfer them to system A, as shown in the command below.

  ,----
  | rdrg@desktop ~$ find /gnu/store -mindepth 1 -maxdepth 1 -exec guix copy
--to=rdrg@192.168.1.42 {} +
  `----

  I executed this command. However, some errors were shown and the
  command didn't exit successfully.

  ,----
  | rdrg@desktop ~$ find /gnu/store -mindepth 1 -maxdepth 1 -exec guix copy
--to=rdrg@192.168.1.42 {} +
  | guix copy: error: path `/gnu/store/.links' is not valid
  | guix copy: sending 0 store items (0 MiB) to '192.168.1.42'...
  | guix copy: sending 0 store items (0 MiB) to '192.168.1.42'...
  | guix copy: sending 0 store items (0 MiB) to '192.168.1.42'...
  | guix copy: sending 0 store items (0 MiB) to '192.168.1.42'...
  | guix copy: error: path
`/gnu/store/gckhwapfi4m73pvlsskri4qscf1mzs55-nvidia-module-515.76.lock' is
not valid
  | guix copy: error: path
`/gnu/store/sldynyzhq1w96b05yb5qvfgyvqji1ydm-linux-libre-5.10.177.lock' is
not valid
  | guix copy: sending 296 store items (4,894 MiB) to '192.168.1.42'...
  |  21% ▕██████████████████████████████████▏

                     ▏;;; [2023/04/07 13:16:47.068026, 0] [GSSH ERROR]
Parent session is not connected: #<unknown channel (freed) 7f357f8a2780>
  | Backtrace:
  |           17 (primitive-load "/home/rdrg/.config/guix/current/bin/gu…")
  | In guix/ui.scm:
  |    2300:7 16 (run-guix . _)
  |   2263:10 15 (run-guix-command _ . _)
  | In ice-9/boot-9.scm:
  |   1752:10 14 (with-exception-handler _ _ #:unwind? _ # _)
  |   1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
  | In guix/store.scm:
  |    659:37 12 (thunk)
  |    1298:8 11 (call-with-build-handler _ _)
  |    1298:8 10 (call-with-build-handler #<procedure 7f35845fc900 at g…> …)
  | In guix/status.scm:
  |     830:4  9 (call-with-status-report _ _)
  | In guix/scripts/copy.scm:
  |     81:26  8 (_)
  | In guix/ssh.scm:
  |     530:7  7 (send-files _ _ _ #:recursive? _ #:log-port _)
  | In guix/store.scm:
  |   1764:12  6 (export-paths #<store-connection 256.99 7f358458e910> _ …)
  |   1731:22  5 (export-path #<store-connection 256.99 7f358458e910> _ # …)
  |    716:13  4 (process-stderr _ _)
  | In guix/build/utils.scm:
  |   1014:11  3 (loop _ _)
  | In unknown file:
  |            2 (put-bytevector #<unknown channel (freed) 7f357f8a2780> …)
  | In ice-9/boot-9.scm:
  |   1685:16  1 (raise-exception _ #:continuable? _)
  |   1685:16  0 (raise-exception _ #:continuable? _)
  |
  | ice-9/boot-9.scm:1685:16: In procedure raise-exception:
  | Throw to key `guile-ssh-error' with args `("write_to_channel_port"
"Parent session is not connected" #<unknown channel (freed) 7f357f8a2780>
#f)'.
  | rdrg@desktop ~$ echo $?
  | 1
  | rdrg@desktop ~$
  `----

  I also tried executing `guix copy' on each file. However, this was
  very slow, because an instance of `guix copy' need to be launch for
  each file in `/gnu/store' =even for files that had already been
  copied.

  ,----
  | find /gnu/store -mindepth 1 -maxdepth 1 -exec guix copy --to=
rdrg@192.168.1.42 {} \;
  `----

  Instead, what I tried was to get the files under `/gnu/store' that
  were unique to system B and execute `guix copy' on them, as shown in
  the code block below. The command throw

  ,----
  | rdrg@desktop ~$ find /gnu/store -mindepth 1 -maxdepth 1 | sort >
~/e/desktop.txt
  | rdrg@desktop ~$ ssh rdrg@192.168.1.42 'find /gnu/store -mindepth 1
-maxdepth 1 | sort' > ~/e/delta.txt
  | rdrg@desktop ~$ for i in $(comm -2 -3 ~/e/desktop.txt ~/e/delta.txt);
do guix copy --to=rdrg@192.168.1.42 "$i"; done
  | guix copy: sending 2 store items (49 MiB) to '192.168.1.42'...
  | /gnu/store/3hfbyphf96riviip19cqap6hnmai6q8m-gtk+-3.24.30
  | /gnu/store/0ffbcckd6mziy5q1qpbjnpsmin9zwchq-glib-schemas
  | guix copy: sending 12 store items (0 MiB) to '192.168.1.42'...
  |
/gnu/store/wkcw7xifbvdi2an2nkq5sq6d3cgmnvzm-linux-libre-6.2.9-guix.tar.xz-builder
  |
/gnu/store/i44qvynnrxs3a21ag855diprjdq5i7gh-linux-libre-deblob-6.2.9-gnu.drv
  |
/gnu/store/gkyl8v3z17mqkyblm7v37afhlp2hjilf-linux-libre-deblob-check-6.2.9-gnu.drv
  |
/gnu/store/fh4dmd3s5z8vq2p041arrznbcharrmrl-linux-libre-arm-export-__sync_icache_dcache.patch.drv
  |
/gnu/store/dixa4n5h0fv475qi0a4cpwbi1hdjbypv-linux-libre-6.2.9-guix.tar.xz.drv
  | /gnu/store/8l15dxwm0pwdyqa7mqc9b3f7b4hvl7cs-100gnu+freedo.patch.drv
  |
/gnu/store/5yg1dg009ma726yja5ql4m0nl61jwrby-linux-libre-6.2.9-guix.tar.xz-builder
  |
/gnu/store/3605dafwvk90fw4kcm62qnlaghi753rj-linux-libre-6.2.9-guix.tar.xz.drv
  | /gnu/store/2kjrxdhwm7izgnap6z6mhw1hva49ch43-linux-libre-6.2.9-builder
  | /gnu/store/x53glvwk5sb4j2fwc90mmqamswmf41nc-linux-libre-6.2.9.drv
  | /gnu/store/r56gag4i5gk9mchncg7sig1hkgx9lql7-linux-modules-builder
  | /gnu/store/0fz88rlf18j3qq1xy7kkpazd9warl4gd-linux-modules.drv
  |
  | (... some omitted lines ...)
  `----

  After the command finished executing, the following files were the
  only ones that were not copied which don't look as if they were
  important because they are empty (see proof below).

  ,----
  | rdrg@desktop ~$ find /gnu/store -mindepth 1 -maxdepth 1 | sort >
~/e/desktop.txt
  | rdrg@desktop ~$ ssh rdrg@192.168.1.42 'find /gnu/store -mindepth 1
-maxdepth 1 | sort' > ~/e/delta.txt
  | rdrg@desktop ~$ for i in $(comm -2 -3 ~/e/desktop.txt ~/e/delta.txt);
do echo "File: $i"; du -hs "$i"; sudo cat "$i"; done
  | File:
/gnu/store/1ap2jaip98r0w5fx82bkzgx2vrkdk06h-linux-libre-4.14.312.lock
  | 0
/gnu/store/1ap2jaip98r0w5fx82bkzgx2vrkdk06h-linux-libre-4.14.312.lock
  | File:
/gnu/store/29mnj31zxycznvswjydk28ivgwa6x0fs-nvidia-module-515.76.lock
  | 0
/gnu/store/29mnj31zxycznvswjydk28ivgwa6x0fs-nvidia-module-515.76.lock
  | File:
/gnu/store/gckhwapfi4m73pvlsskri4qscf1mzs55-nvidia-module-515.76.lock
  | 0
/gnu/store/gckhwapfi4m73pvlsskri4qscf1mzs55-nvidia-module-515.76.lock
  | File:
/gnu/store/sldynyzhq1w96b05yb5qvfgyvqji1ydm-linux-libre-5.10.177.lock
  | 0
/gnu/store/sldynyzhq1w96b05yb5qvfgyvqji1ydm-linux-libre-5.10.177.lock
  | File: /gnu/store/vd0ka6x5bjbgpksiwlwgy4r3v8kc9hfi-samba-4.16.8.lock
  | 0       /gnu/store/vd0ka6x5bjbgpksiwlwgy4r3v8kc9hfi-samba-4.16.8.lock
  | rdrg@desktop ~$ echo $?
  | 0
  `----


3 The question
==============

  As I showed in the previous section "Transfer everything in /gnu/store
  from one system to another", I already know how to transfer all paths
  from one system to another. However, I'm wondering if there's a
  command available in `guix <<command>>' to accomplish the same
  behavior, since I would prefer using a solution that involve fewer
  steps.