[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH RFC 00/21] Modularize generated QAPI code
From: |
Markus Armbruster |
Subject: |
[Qemu-devel] [PATCH RFC 00/21] Modularize generated QAPI code |
Date: |
Fri, 2 Feb 2018 14:03:15 +0100 |
Our qapi-schema.json is composed of modules connected by include
directives, but the generated code is monolithic all the same: one
qapi-types.h with all the types, one qapi-visit.h with all the
visitors, and so forth. These monolithic headers get included all
over the place. In my "build everyhing" tree, adding a QAPI type
recompiles about 4500 out of 4800 objects.
Nobody would write such monolithic headers by hand. It stands to
reason that one shouldn't generate them, either.
This series' basic idea is to split up generated headers to mirror the
schema's modular structure: one header per module. That way, you can
include just what you need.
The series is RFC for a number of reasons:
* The split is implemented only for qapi-types.h. That one should
provide the biggest benefits, though.
* There's a bit of code duplication.
* I haven't re-read my patches, yet.
Even in this incomplete state, the compile-time improvements can be
massive. Before this series, any QAPI schema change recompiles some
4500 out of 4800 objects in my "build everything" tree. Afterwards,
adding a type to qapi/migration.json recompiles less than 400, adding
a QMP event recompiles less than 200, and a documentation change no
longer recompiles anything.
Related: Marc-André's 'unit' pragma proposal. That's a different way
to split off parts of the generated code, motivated by the desire to
use poisoned identifiers such as TARGET_I386. I noted in my review of
v3 that I "can either accept it, or come up with a better solution."
This is my attempt at a better solution. It's a bit more ambitious,
and thus more useful (I hope). The pragma has one theoretical
advantage, though: you can modularize the generated output in
different ways than the input. The patches using don't do that,
however.
Based-on: <address@hidden>
[PATCH v3 00/19] Clean up includes to reduce compile time
Markus Armbruster (21):
qapi: Streamline boilerplate comment generation
qapi: Generate up-to-date copyright notice
qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
qapi: Reduce use of global variables in generators some
qapi: Turn generators into modules
qapi-gen: New common driver for code and doc generators
qapi: Move parse_command_line() next to its only use
qapi: Touch generated files only when they change
qapi: Don't absolutize include file name in error messages
qapi/common: Eliminate QAPISchema.exprs
qapi: Lift error reporting from QAPISchema.__init__() to callers
qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
qapi: Record 'include' directives in parse tree
qapi: Generate in source order
qapi: Record 'include' directives in intermediate representation
qapi/types qapi/visit: Make visitors use QAPIGen more
qapi/types qapi/visit: Generate built-in stuff into separate files
qapi/common: Fix guardname() for funny filenames
qapi/types: Generate separate .h, .c for each module
Include less of qapi-types.h
qapi: Empty out qapi-schema.json
Makefile | 131 +-
Makefile.objs | 20 +-
crypto/cipherpriv.h | 2 +-
hw/block/block.c | 1 +
hw/block/hd-geometry.c | 1 +
hw/net/rocker/rocker_fp.c | 2 +-
include/block/block.h | 2 +-
include/block/dirty-bitmap.h | 2 +-
include/chardev/char.h | 1 +
include/crypto/cipher.h | 2 +-
include/crypto/hash.h | 2 +-
include/crypto/hmac.h | 2 +-
include/crypto/secret.h | 1 +
include/crypto/tlscreds.h | 1 +
include/hw/block/block.h | 2 +-
include/hw/block/fdc.h | 2 +-
include/hw/ppc/spapr_drc.h | 1 +
include/hw/qdev-properties.h | 2 +
include/io/dns-resolver.h | 1 +
include/migration/colo.h | 2 +-
include/migration/failover.h | 2 +-
include/migration/global_state.h | 1 +
include/monitor/monitor.h | 1 +
include/net/filter.h | 1 +
include/net/net.h | 2 +-
include/qapi/error.h | 2 +-
include/qapi/qmp/qobject.h | 2 +-
include/qapi/visitor.h | 2 +-
include/qemu/sockets.h | 2 +-
include/qemu/throttle.h | 2 +-
include/qom/cpu.h | 1 +
include/qom/object.h | 2 +-
include/sysemu/arch_init.h | 2 +-
include/sysemu/balloon.h | 2 +-
include/sysemu/dump.h | 2 +
include/sysemu/hostmem.h | 1 +
include/sysemu/replay.h | 2 +
include/sysemu/sysemu.h | 1 +
include/sysemu/tpm.h | 1 +
include/sysemu/watchdog.h | 2 +-
include/ui/input.h | 2 +-
migration/migration.h | 1 +
migration/ram.h | 2 +-
net/tap_int.h | 2 +-
qapi-schema.json | 3098 +-------------------
qapi/misc.json | 3090 +++++++++++++++++++
qapi/run-state.json | 10 +
replication.h | 1 +
scripts/qapi-gen.py | 95 +
scripts/qapi/__init__.py | 0
scripts/{qapi-commands.py => qapi/commands.py} | 98 +-
scripts/{qapi.py => qapi/common.py} | 290 +-
scripts/{qapi2texi.py => qapi/doc.py} | 28 +-
scripts/{qapi-event.py => qapi/events.py} | 101 +-
scripts/{qapi-introspect.py => qapi/introspect.py} | 87 +-
scripts/{qapi-types.py => qapi/types.py} | 218 +-
scripts/{qapi-visit.py => qapi/visit.py} | 208 +-
tests/Makefile.include | 56 +-
tests/qapi-schema/comments.out | 3 +-
tests/qapi-schema/doc-bad-section.out | 5 +-
tests/qapi-schema/doc-good.out | 33 +-
tests/qapi-schema/doc-good.texi | 3 +-
tests/qapi-schema/empty.out | 2 +-
tests/qapi-schema/event-case.out | 3 +-
tests/qapi-schema/ident-with-escape.out | 7 +-
tests/qapi-schema/include-no-file.err | 2 +-
tests/qapi-schema/include-relpath.out | 7 +-
tests/qapi-schema/include-repetition.out | 12 +-
tests/qapi-schema/include-simple.out | 5 +-
tests/qapi-schema/indented-expr.out | 3 +-
tests/qapi-schema/qapi-schema-test.out | 321 +-
tests/qapi-schema/test-qapi.py | 17 +-
ui/vnc.h | 1 +
73 files changed, 4040 insertions(+), 3984 deletions(-)
create mode 100644 qapi/misc.json
create mode 100755 scripts/qapi-gen.py
create mode 100644 scripts/qapi/__init__.py
rename scripts/{qapi-commands.py => qapi/commands.py} (78%)
rename scripts/{qapi.py => qapi/common.py} (93%)
rename scripts/{qapi2texi.py => qapi/doc.py} (93%)
mode change 100755 => 100644
rename scripts/{qapi-event.py => qapi/events.py} (69%)
rename scripts/{qapi-introspect.py => qapi/introspect.py} (78%)
rename scripts/{qapi-types.py => qapi/types.py} (54%)
rename scripts/{qapi-visit.py => qapi/visit.py} (66%)
--
2.13.6
- [Qemu-devel] [PATCH RFC 00/21] Modularize generated QAPI code,
Markus Armbruster <=
Re: [Qemu-devel] [PATCH RFC 06/21] qapi-gen: New common driver for code and doc generators, Marc-Andre Lureau, 2018/02/05