Emacs + bear + compile_commands.json + clangd

[Matt Armstrong]

João Távora <joaotavora@gmail.com> writes:

> Hello Gerd. Out of curiosity, and speculation, is your use of 'bear'
> intended to make a compilation database of Emacs sources for use with a LSP
> server?
>
> If so, how is that going? Last time I tried to do that, the server (clangd,
> i think) was still very confused in many files.
>
> João

Hey João, I had issues with Emacs source and clangd, especially with
header files.  The issue is that many of Emacs' C header files are not
"self contained" so they can't be compiled alone.  Also, `bear` will
only generate compile commands for .c files, and clangd uses heuristics
to guess a compile command for nearby .h files.  It does a reasonable
job in some projects, but not Emacs due to the "self contained" issue.
Biggest issue: when editing an .h file clangd won't include <config.h>
for you.

I now do this:

    ./configure <whatever>
    bear --force-wrapper -- make -j$CPUS
    emacs-fixup-compile-commands.py

where emacs-fixup-compile-commands.py is attached.  Bonus to anyone
rewrites it in elisp.  ;-)

The idea is to find the .c file with the most similar name to each .h
file, then form a compile command for the .h by hacking the .c file's
command line, adding:

   -Wno-unused-macros -include config.h

...this way, when clangd "compiles" an Emacs .h file it includes
config.h first (which fixes many issues), and won't complain about
macros going unused in the file.

With this, Emacs + clangd is pretty seamless for me.

P.S. I recently had to start passing "--force-wrapper" to bear.  Some
Emacs builds steps were failing with bear's LD_PRELOAD hack.  Maybe the
Emacs modules stuff?  Or tests?  I didn't look into it, but
"--force-wrapper" fixed it.

#!/usr/bin/env python3

import copy
import json
import glob
import os
import sys


def levenshteinDistance(s1, s2):
    if len(s1) > len(s2):
        s1, s2 = s2, s1

    distances = range(len(s1) + 1)
    for i2, c2 in enumerate(s2):
        distances_ = [i2 + 1]
        for i1, c1 in enumerate(s1):
            if c1 == c2:
                distances_.append(distances[i1])
            else:
                distances_.append(
                    1 + min((distances[i1], distances[i1 + 1], distances_[-1]))
                )
        distances = distances_
    return distances[-1]


def Read():
    with open("compile_commands.json") as file:
        return json.load(file)


def Write(compile_commands):
    with open("compile_commands.json", "w") as file:
        json.dump(compile_commands, file, sort_keys=True, indent=2)


def Headers():
    # return [f for f in os.listdir('src') if f.endswith('.h')]
    return glob.glob(os.path.join(os.getcwd(), "src/*.h"))


def NeedsConfigH(header):
    """Returns True if header needs config.h to function"""
    if header.endswith("/config.h"):
        return False
    with open(header) as file:
        return "INLINE_HEADER_BEGIN" in file.read()


def FindSimilarCommand(compile_commands, header):
    header = os.path.join(os.getcwd(), header)
    bestDistance = sys.maxsize
    bestCommand = None
    for command in compile_commands:
        distance = levenshteinDistance(header, command["file"])
        if distance < bestDistance:
            bestDistance = distance
            bestCommand = command
    return bestCommand


def AdaptTemplate(command, header):
    command = copy.deepcopy(command)
    if "output" in command:
        del command["output"]
    src = os.path.relpath(command["file"], command["directory"])
    index = command["arguments"].index(src)
    new_src = os.path.relpath(header, command["directory"])
    command["arguments"][index : index + 1] = [
        "-Wno-unused-macros",
        "-include",
        "config.h",
        new_src,
    ]
    command["file"] = header
    return command


compile_commands = Read()
new_compile_commands = copy.copy(compile_commands)
for header in Headers():
    if NeedsConfigH(header):
        similar = FindSimilarCommand(compile_commands, header)
        if similar["file"] == header:
            continue
        adapted = AdaptTemplate(similar, header)
        new_compile_commands.append(adapted)
Write(new_compile_commands)