[PROPOSAL] Builder, a build system integration for Emacs

[BTuin]

Table of Contents
─────────────────

1. The problem it tries to solve
2. How it works
.. 1. Build system detection
.. 2. Configure and compile
.. 3. Project only or not?
.. 4. Command formatting
.. 5. `function-modification'
.. 6. Configuration
3. Open questions
.. 1. Local configuration
.. 2. Build directory name
.. 3. Build system detection
.. 4. Tramp support
.. 5. Flexibility
.. 6. Adding a build system
.. 7. Other
4. Conclusion


Hello, I would like to propose a new package to integrate in Emacs,
"Builder".  It provides integration with build systems, to ease the
configuration and the compilation process, by automatically detecting
the build system and proposing relevant commands.

Big disclaimer: I do not know many build systems, I do not have a lot of
experience in programming, I do not know (E)lisp very well, and English
is not my first language.  Now that you are definitely sold on it, let's
have a deeper explanation!

As far as I know, Emacs does not provide a convenient way to use build
systems, like CMake, Meson, Autotools, Cargo, Dune, and so on.  I know
your can configure the compilation command with local variables, but:
• You cannot have a choice, like `cmake .. -DCMAKE_BUILD_TYPE=Release'
  or `cmake .. -DCMAKE_BUILD_TYPE=Debug'
• It requires to set it up manually
• There is no (convenient?) way to have a configuration phase and a
  compilation phase.  (like `cmake ..' followed by `make')

With Builder, you just run `builder-configure' which set up the build
system, and `builder-compile' to compile the project.
`builder-configure' allow you to easily switch between release and
debug.

You can try it with the `builder-demo/' directory found inside this git
repository: <https://gitlab.com/btuin2/builder.git>

There are more explanations in the README.org.


1 The problem it tries to solve
═══════════════════════════════

  Builder tries to solve two issues at the same time: being convenient
  and being easy to configure.  Independently they are not *that*
  difficult to solve, but together it gets really complex.  For example,
  some build systems need to run their command /outside/ the build
  directory, while others need to run it /inside/.  Also, some commands
  need to be modified according to the state of the project: Meson needs
  the flag `--reconfigure' if it was already configured.


2 How it works
══════════════

2.1 Build system detection
──────────────────────────

  The first step is the detection of the build system.  It simply checks
  in a list of files which one exists and it returns a list of build
  system identifiers.  For instance if both CMake and Meson are
  available, it returns the list `("cmake" "meson")'.

  Notice that the detection is completely separated from build systems
  addition.  This is a deliberate design choice to make it more
  versatile, for example to filter results or have a cache system (both
  not implemented).


2.2 Configure and compile
─────────────────────────

  The configuration is a first step for some build systems (Autotools,
  CMake, Meson)… Indeed they are not actually build systems, but build
  system /generator/ (not that it really matters).  `builder-configure'
  and `builder-compile' respectively provide, well, configuration and
  compilation.  They both do what you would expect by proposing relevant
  commands and running the selected one at the right place.


2.3 Project only or not?
────────────────────────

  You will quickly notice that many functions have the argument
  `variable'.  I am not really sure about this one, but it would allow
  to use a build system not located at the root of a project.  I guess
  it is useful for monorepo projects?  I did minimal testing on it and
  it seems to works.


2.4 Command formatting
──────────────────────

  Commands pass though the function `builder-format-command' before
  being run.  It uses the function `format-spec` with predefined
  specifiers, such as `%n` for the number of cores, `%b` for the build
  directory name…  For example "mycommand -proc %n –dir %b" becomes
  "mycommand -proc 4 –dir build".


2.5 `function-modification'
───────────────────────────

  A pretty neat feature is `function-modification': before running, the
   command is modified by a function.  Its power is showcased with the
   Dune build system and CMake Presets, because you can chose a specific
   target from a list, and the command is modified as needed!  Some
   details need to be investigated though: should the command pass
   through `builder-format-command' before being send to the function?
   After?  Or let the function take care of it?


2.6 Configuration
─────────────────

  The configuration of a build system is hierarchical: at toplevel, it
  affects every instruction from the build system, but you can configure
  each instruction independently.  For example, you don't need to
  specify manually that each CMake command need to run inside the
  directory, but you can still customize a specific command.

  In the following example, both "debug" and "fast" commands are
  executed inside the build directory, and "release" is executed at the
  root of the project.

  ┌────
  │ (builder-add-build-system
  │  :build-system-id "gcc"
  │  :compile '(:inside-directory t ;; Affects all instructions
  │             :instructions
  │             ((:name "debug"
  │               :command "gcc main.c -g")
  │              (:name "release"
  │               :command "gcc main.c -O2"
  │               ;; Overrides the higher level :inside-directory
  │               :inside-directory nil)
  │              (:name "fast"
  │               :command "gcc main.c -Ofast"))))
  └────


3 Open questions
════════════════

  This is a list of problems I don't have a definitive answer to.
  Suggestions are most welcome.


3.1 Local configuration
───────────────────────

  It's necessary to have a way to locally configure a project to add
  commands, specify the build directory…  But I don't really know how to
  do that.  Should it affect the whole project?  Only a specific
  directory (e.g. only `project-root/subdir')?  I know that local
  variables are a thing in Emacs, but they are not a good solution for
  this since they are actually *buffer* variables.  So changing the
  local configuration would require to reload each local variable even
  unrelated one that could have been modified on a per buffer basis.
  Also, the commands cannot be considered as safe, so opening a file
  trigger a warning each time (unless you mark the values as safe).

  It would be a good idea to discuss with the Projectile package devs
  about it, to agree on a common solution if they wish.

  I have implemented an alternative, dir-val.el.  The variables are
  stored in a tree, each node corresponding to a directory.  The
  variables are "hierarchical".  If you have a directory
  `/dirA/dirB/dirC' and you set a variable to `dirB', then `dirC' will
  also access it.  But you can also change the value only for `dirC'.

  I'm wondering if this is actually useful.  Would project variables be
  enough?


3.2 Build directory name
────────────────────────

  This is harder to solve than it seems.  I would like to have a
  mechanism to change it easily, so you can have `build-release' and
  `build-debug' directories, allowing you to conveniently switch between
  release and debug without having to recompile the whole project each
  time you switch.  BUT: some tools have specific expectations about
  this name.  `clangd' expects it to be named `build', and I'm pretty
  sure that the OCaml ecosystem expect it to be named `_build'.  Also,
  how to know which directory to use to execute a command, as
  "configure" and "compile" may have different names?  With a cache?
  With a regex to select candidates?  Local configuration only?

  I really would like to have this feature implemented.  Currently you
  can chose to modify the build directory name according to the current
  git branch (which is pretty reliable), but it would be really useful
  to have `build-release' and `build-debug' directories.


3.3 Build system detection
──────────────────────────

  Currently there are few build systems added to the list.  But there
  could be hundreds.  How well would it scale?  Especially on slow file
  systems?


3.4 Tramp support
─────────────────

  I don't use Tramp and have never used it.  I don't think it would be
  too hard to support it, but the detection could be a problem if there
  are too many build systems.


3.5 Flexibility
───────────────

  Is Builder good enough to support every build system?  To accommodate
  most use cases?  As I said I don't have a lot of experience in build
  systems, so suggestions are welcome.


3.6 Adding a build system
─────────────────────────

  A build system is added with `builder-add-build-system'.  Is it good
  enough?  I fear it is a bit too error prone because it's easy to mess
  with parenthesis and keywords location.  How to solve that?  With a
  macro?  EIEIO?


3.7 Other
─────────

  Some other are listed in the README and there are probably many other
  issues I did not think about.


4 Conclusion
════════════

  I hope you will find this package useful.  Please tell me if this is
  unclear, needs more documentation, or if you have issues.

  Currently only a few build systems are supported.  You can already use
  it to build Emacs with the build system `autotools'.
  `builder-configure' and `builder-compile' are respectively bind to
  `C-x p C' and `C-x p c'.  `builder-configure' will create a directory
  named "build" and run `../configure' from it, and `builder-compile'
  will run `make -jN' with N the number of cores on your processor (the
  result of `num-processor').

  I think this package is far from ready, but I need to know if the
  design is sound enough and future proof.

  Obviously I will need to sign the copyright assignment.

builder.el
Description: Text Data

dir-var.el
Description: Text Data

dir-var-test.el
Description: Text Data

README.org
Description: Text Data