Re: [PATCH v2] bison: fix the parallel build

[Akim Demaille]

Hi Mingli!

> Le 14 mai 2020 à 06:35, address@hidden a écrit :
> 
> Explicitly make the BUILT_SOURCES which are the generated headers such
> as stdio.h, fcntl.h and etc to be the dependencies of the gl_LIBOBJS
> such as libbison_a-sprintf.o, libbison_a-printf.o and etc to guarantee
> the BUILT_SOURCES is generated before begin to compile
> EXTRA_lib_libbison_a_SOURCES such as fprintf.c in parallel builid,
> otherwise there may come below error:
> 
> | muscle-tab.c:(.text+0x77a): undefined reference to `rpl_sprintf'
> 
> It does the same for src_bison_OBJECTS and lib_libbison_a_OBJECTS to
> make sure BUILT_SOURCES generated before begin to compile
> src_bison_SOURCES which contains AnnotationList.c and etc.
> 
> BTW, the MOSTLYCLEANFILES also contains the generated header needs to
> be created early in the build process, so add it also in to avoid
> below error:
> 
> | ./lib/uniwidth/width.c:21:10: fatal error: uniwidth.h: No such file or 
> directory

Thank you very much for the analysis and proposed solution!

But I still do not understand how this can happen.

The generated headers that you are referring to (stdio.h, fcntl.h)
are all put into BUILT_SOURCES.  On my machine, Makefile reads:

BUILT_SOURCES = $(mfcalc_sources) $(am__append_8) $(rpcalc_sources) \
        $(simple_sources) $(ALLOCA_H) lib/configmake.h $(ERRNO_H) \
        lib/fcntl.h $(FLOAT_H) $(GETOPT_H) $(GETOPT_CDEFS_H) \
        lib/inttypes.h $(TEXTSTYLE_H) $(LIMITS_H) $(am__append_23) \
        lib/math.h lib/sched.h lib/signal.h lib/spawn.h $(STDBOOL_H) \
        $(STDDEF_H) $(STDINT_H) lib/stdio.h lib/stdlib.h lib/string.h \
        lib/sys/resource.h $(am__append_28) lib/sys/time.h \
        lib/sys/times.h lib/sys/types.h lib/sys/wait.h lib/time.h \
        lib/unistd.h $(LIBUNISTRING_UNITYPES_H) \
        $(LIBUNISTRING_UNIWIDTH_H) lib/wchar.h lib/wctype.h \
        src/parse-gram.c src/parse-gram.h src/scan-code.c \
        src/scan-gram.c src/scan-skel.c $(top_srcdir)/.version

(stdio.h and fcntl.h are in there.  And uniwidth.h is also in there,
as $(LIBUNISTRING_UNIWIDTH_H)).


And "make all" is:

all: $(BUILT_SOURCES)
        $(MAKE) $(AM_MAKEFLAGS) all-recursive

So the built sources are always built before the object files.  Thus I
do not understand how it is possible for:

1. libbison_a-sprintf.o, libbison_a-printf.o to be built before
   stdio.h

2. Likewise with lib/uniwidth/width.c wrt uniwidth.h


Your patch certainly addresses your concern, but actually it's "overkill".
For instance:

+$(gl_LIBOBJS): $(BUILT_SOURCES) $(MOSTLYCLEANFILES:%.h)

results in the fact that all the files in gl_LIBOBJS depend on all the
$(BUILT_SOURCES), which is not true and would result in more compilations
than truly needed if one of the BUILT_SOURCES is changed.



There's a question in the ticket I believe is unanswered: how exactly
is Bison built?  Where can I read the recipe?  I would like to see how
"configure" is run, and how "make" is run too.  You gave me the configure
call, but I would like to see the full sequence.
(https://github.com/akimd/bison/issues/31#issuecomment-608987970)
I found it nowhere in
https://autobuilder.yoctoproject.org/typhoon/#/builders/92/builds/2205/steps/8/logs/step1d
but I'm uneasy with this environment.



Please understand my hesitations.

1. gnulib.mk is generated via gnulib, and gnulib is a project independent
  from Bison used by many projects.  I cannot just push changes over there
  without a strong case.

2. I still don't understand why the theory does not apply to your case,
  and maybe there are more bugs lurking because we are applying workarounds
  instead of addressing the core issue: some BUILT_SOURCES were not
  generated before the actual compilations.


Thanks in advance!

        Akim