Upstreaming my Android channel

[Julien Lepiller]

Hi Guix!

I have been working on updating Android packages for some time now. I
think it's at a point I'm confident I can share and move them upstream.
My work is currently in a separate channel at
https://framagit.org/tyreunom/guix-android. I have been working on the
SDK and tools, which we already have. We currently are limited to
Android 7 versions, because later Android versions do not use the same
build system anymore.

Since then, Android uses the Soong build system, based on blueprint.
It's pretty complex, and specific. I tried to build it, but it assumes
too many things to be useful. Instead, I "reimplemented" it by
following what a Nix contributor did in soongnix:
https://github.com/danielfullmer/soongnix

I created a soong-build-system that has a few modules for cc, art and
java types of packages. It supports building most packages from the SDK
and other Android tools: aapt, appt2, adb, aidl, apksigner, dexdump,
dmtracedump, dx, etc1tool, fastboot, hprof-conv, libaapt2_jni,
split-select, zipalign.

I have an importer that is a bit different from the existing importers.
First, it needs to checkout ~30GB of git repositories (and that's with
--depth=1 for all of them!), explore the set of Android.bp files (used
by soong) and create one package for each package in these Android.bp
files. That creates packages that can be a bit smaller than expected.
Instead of printing, the importer creates two files: one for the source
definitions (many packages share the same source) and one for the
packages and their dependencies.

Since I have all of that in a channel, the importer doesn't work from
the command-line, but I have a relatively simple API that you can use
to import packages:

```
,use (android import repo)
(define manifest (get-manifest))

;; Fetch repositories that form the android distribution.
;; This is very big (> 28GB) and will take some time.  If it fails
;; because of a network issue, restart that command again until
;; everything is downloaded.
;;
;; You can also add an optional destination directory where the
;; sources are to be downloaded, e.g. on a different file-system if
;; you lack space on the file system your home directory is on.
(define root (fetch-manifest-repositories manifest))

;; Finds all Android.bp files from which to import, takes ~10s on warm
;; cache, significantly longer on cold cache and spiny disks.
(define bp-files (get-all-bp-files root))

;; Creates the shared structure: a map of modules and variables to the
;; files that define them, so we can find them, without actually parsing
;; everything.  Will take ~5mins on an HDD, could be faster on SSD.
(define bp-maps (get-bp-maps bp-files))

;; Run the import.  The last two arguments are the names of files that
;; will be generated.
(import-recursively manifest root bp-maps '("adb" "fastboot")
                    "sources.scm" "packages.scm")
```

That would import adb and fastboot

You can see the result of importing a bit more than that at
https://framagit.org/tyreunom/guix-android/-/blob/master/android/packages/android-tools.scm
and
https://framagit.org/tyreunom/guix-android/-/blob/master/android/packages/android-sources.scm

It's pretty great but missing a few things. These packages don't have a
home page, synopsis, description, ... and I don't see a way to easily
import that. Although I could manually add them, the work will have to
be done again at each update. Every month, Google releases a new
Android revision (not necessarily a major version) that modifies some
sources, and can impact the set of dependencies we need to build.

I was aiming at having an importer that was able to recreate all the
files with no human intervention. I think it's the only sane way to
take care of all those packages.

Currently I think all of android/build, android/build-system,
android/import, android/packages/android-{headers,tools,sources}.scm
and android/packages/{bison,clang}.scm can be upstreamed. The rest is
wip or broken packages.

Do you have any ideas how we could plan upstreaming all this?
Especially, how could the importer fit in the current Guix
infrastructure?

Thoughts?