Ability to print the make database as json

I've often wanted to extract information from a large build in some way that's more reliable than grep. The GNU make (--print-data-base) option has been a very useful way to see what the complicated makefiles I was working on finally evaluated to. The only negative about it from my point of view is that it was still a makefile in the end and still had to be parsed.

So recently I tried cloning the print-data-base feature into something that delivered JSON. Now it should be trivial to get a list of targets of some particular type - even with a tool like "jq".

The result so far is lacking a lot but I am running out of free time to work on it and I thought that I might as well seek feedback now in the event that there's anyone else out there who finds this of interest.

At a high level the output looks like a straight dump of make's internal data structures but there is a tonne of detail about each variable and each file which I've left out to make this overview simple:

{
"MakefileName": {
"variables": {
"global": {

          "pkgdatadir" : {
"origin": "makefile",
          "private": false,
            "source": "Makefile",
            "line": 92,
            "assign-recursive": "$(datadir)\/make"
          }

},
"pattern-specific-variables": {},
"pattern-specific-rule-count": 0
},
"dirs": [],
"rules": [],
"files": {
"src/w32/w32os.c": {},
"/usr/include/bits/fcntl.h": {},
"dist": {},
}
}
}

The whole thing isn't indented nicely either but one can just run jq on it.

If you looked at an individual file you might see something like this:

"src/w32/w32os.c" : {
"hname": "src\/w32\/w32os.c",
"vpath": "",
"deps": [],
"stem": "",
"also_make": [],
"target-variables": {
},
"last_mtime": 0,
"mtime_before_update": 0,
"considered": 0,
"command_flags": 0,
"update_status": 1,
"command_state": "cs_not_started",
"builtin": false,
"precious": false,
"loaded": false,
"unloaded": false,
"low_resolution_time": false,
"tried_implicit": false,
"updating": false,
"updated": false,
"is_target": false,
"cmd_target": false,
"phony": false,
"intermediate": false,
"is_explicit": true,
"secondary": false,
"notintermediate": false,
"dontcare": false,
"ignore_vpath": false,
"pat_searched": false,
"no_diag": false,
"was_shuffled": false,
"snapped": false
}

HOW TO GET IT:

===============

in the feature/jprint branch on this fork:

https://github.com/tnmurphy/gmake-experimental

I've added a new commandline option:

--print-data-base-json

you can build it and then run it like this:

./make --print-data-base-json

HOW IT GENERATES OUTPUT:

===========================

The idea is that a JSON file is created in each directory where make runs - this lets the code avoid intermingling output with other bits of make that would muck up the JSON. For now the files are named with the following convention: makefile-$PID.json where $PID is the pid of the make process which created the file. Is this enough? Is it right? I am not sure - it's interesting. We seem to get 2 files for one directory sometimes - with the same targets in them - I think it might be a remake. Anyhow this method seems to keep everything separated for now.

I've also not mentioned that for the "MakefileName" in the example I actually have inserted the contents of "$(MAKEFILE_LIST)" as that seems more correct in a way.

There is an attached json produced by running on make itself - which is all the testing I've done - this is a hack and probably needs real test data and a load of tests.

Most of the code is in src/jprint.c. I had to export some things in questionable ways and that is one of many reasons why I would never expect this feature to get back into GNU make proper. I just think it can exist in a fork and potentially be of use to someone.

Regards,

Tim Murphy

From:	Tim Murphy
Subject:	Ability to print the make database as json - experimental
Date:	Thu, 28 Dec 2023 22:22:26 +0000

Ability to print the make database as json - experimental