Re: [lmi] Best way to integrate PCRE

[Greg Chicares]

On 9/18/21 1:26 PM, Vadim Zeitlin wrote:
[...]
> My branch is called "use-pcre" and the corresponding
> pull request is at https://github.com/let-me-illustrate/lmi/pull/190
After working with it for a fairly short while, I have two comments.

(1) (Trivial but important) The 'pcre_regex.hpp' header should contain
'LMI_HAS_PCRE' guards, without which 'make check_physical_closure'
fails for MinGW. They're trivial to add, e.g.:

--8<--
 #include "config.hpp"
+
+#if defined LMI_HAS_PCRE

 /// Currently we always use 8-bit version of PCRE2 library and assume UTF-8
-->8--

...and then at the bottom...

--8<--
+ 
+#endif // defined LMI_HAS_PCRE
 
 #endif // pcre_regex_hpp
-->8--

but I think I should ask you to do it, because I tried, and you can
read my tale of woe in this modification that I tried adding via
'git rebase --interactive' to the first commit:

    IMPORTANT MODIFICATION: This commit doesn't match the original merge
    request. The 'pcre_regex.hpp' header has been modified, to guard its
    substance with a macro. Without this modification, the important
    'check_physical_closure' makefile target failed; that could break any
    later 'git bisect'. It's possible that 'git bisect' could fail anyway,
    as this modification had to be committed with '--no-verify'.

By the last sentence, I realized I had departed the true path.

(2) If 'test_coding_rules' must be rebuilt (e.g. because 'test_coding_rules.cpp'
has been modified) in a i686-w64-mingw32 environment, this failure occurs:

/opt/lmi/src/lmi[0]$env |grep LMI_                                       
LMI_COMPILER=gcc
LMI_TRIPLET=i686-w64-mingw32

/opt/lmi/src/lmi[0]$touch test_coding_rules.cpp

/opt/lmi/src/lmi[0]$time make $coefficiency custom_tools 2>&1 |less -S -N
make $coefficiency custom_tools 2>&1  0.36s user 0.19s system 108% cpu 0.505 
total
less -S -N  0.00s user 0.00s system 0% cpu 3.371 total
/opt/lmi/src/lmi[0]$time make $coefficiency custom_tools                 
make[1]: Entering directory '/opt/lmi/src/lmi'
g++ -o test_coding_rules alert.o alert_cli.o fenv_lmi.o main_common.o 
main_common_non_wx.o sigfpe.o unwind.o miscellany.o my_test_coding_rules.o 
test_coding_rules.o -L . -L /opt/lmi/local/gcc_x86_64-pc-linux-gnu/lib -L 
/opt/lmi/local/gcc_x86_64-pc-linux-gnu/bin  
-L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxsltwrapp 
-L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lexslt -lxslt -lxml2 -lws2_32 -lxml2 
-L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxslt -lxml2 -lws2_32 -lxml2 
-lxmlwrapp -L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxml2 -lexslt 
-L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxslt -lxml2 -lws2_32 
-L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxml2 -lws2_32   -ldw -lunwind -ldl 
-lpcre2-8   -ggdb  -fPIC -Wl,-Map,test_coding_rules.map 
/usr/bin/ld: cannot find -lws2_32
/usr/bin/ld: cannot find -lws2_32
/usr/bin/ld: cannot find -lws2_32
/usr/bin/ld: cannot find -lws2_32
collect2: error: ld returned 1 exit status
make[2]: *** [/opt/lmi/src/lmi/workhorse.make:938: test_coding_rules] Error 1
make[1]: *** [GNUmakefile:241: /opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship] 
Error 2
make[1]: Target 'test_coding_rules' not remade because of errors.
make[1]: Leaving directory '/opt/lmi/src/lmi'
make: *** [GNUmakefile:449: custom_tools] Error 2
make $coefficiency custom_tools  0.40s user 0.18s system 108% cpu 0.531 total

The cause of the problem is seen by dumping some flags, e.g. by modifying
'workhorse.make' thus (which is probably a worthwhile change that I should
make, later):

--8<--
        @printf 'ALL_LDFLAGS             = "%s"\n' "$(ALL_LDFLAGS)"
+       @printf 'REQUIRED_LDFLAGS        = "%s"\n' "$(REQUIRED_LDFLAGS)"
+       @printf 'EXTRA_LDFLAGS           = "%s"\n' "$(EXTRA_LDFLAGS)"
+       @printf 'REQUIRED_LIBS           = "%s"\n' "$(REQUIRED_LIBS)"
+       @printf 'EXTRA_LIBS              = "%s"\n' "$(EXTRA_LIBS)"
        @printf 'ALL_RCFLAGS             = "%s"\n' "$(ALL_RCFLAGS)"
-->8--

after which:

$make LMI_TRIPLET=x86_64-pc-linux-gnu show_flags 2>&1 |less
...
EXTRA_LIBS              = "-ldw -lunwind -ldl -lpcre2-8"
REQUIRED_LIBS           = "-L/opt/lmi/local/gcc_i686-w64-mingw32/lib 
-lxsltwrapp -L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lexslt -lxslt -lxml2 
-lws2_32 -lxml2 -L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxslt -lxml2 
-lws2_32 -lxml2 -lxmlwrapp -L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxml2 
-lexslt -L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxslt -lxml2 -lws2_32 
-L/opt/lmi/local/gcc_i686-w64-mingw32/lib -lxml2 -lws2_32  "

The root cause seems to be here, in 'workhorse.make':

  # The link command promiscuously mentions libxml2 for all targets.
  # Measurements show that this costs one-tenth of a second on
  # reasonable hardware, and it saves the trouble of maintaining a list
  # of which targets require which libraries.
  #
  # TODO ?? Consider refining it anyway, because it's unclean: libxml2
  # isn't actually required for all targets.
  
  REQUIRED_LIBS := \
    $(xml_libraries) \
so maybe it's time to resolve that defect once and for all.

OTOH, suppose 'test_coding_rules' actually required libxml2: then
fixing that "TODO ??" wouldn't help, because:

  xml_libraries := \
    $(shell xmlwrapp-config --libs) \
    -lexslt \
    $(shell xslt-config --libs) \
    $(shell xml2-config --libs) \

and in that hypothetical case the root problem is that this
variable is resolved before we reach the 'custom_tools' target,
so maybe we need to re-invoke 'make' for the top-level makefile,
e.g. [untested]:

 custom_tools:
        @$(MAKE) LMI_TRIPLET=$(host_triplet) test_coding_rules
+       @/bin/make -f GNUmakefile LMI_TRIPLET=$(host_triplet) test_coding_rules

Alternatively, though, wouldn't it be better just to do the following:

  make LMI_TRIPLET=x86_64-pc-linux-gnu $coefficiency check_concinnity

as discussed here:
  https://lists.nongnu.org/archive/html/lmi/2021-09/msg00004.html
with obligatory changes to lists of pastable commands, git hooks,
and the other few places identified by 'git grep -l check_concinnity'?
It was interesting to try to save that small number of OAOO changes
by a single makefile modification, but a single modification doesn't
seem to be enough. What do you think?

BTW, I'm not seeing much speed improvement yet, in these
'make check_concinnity' timings (all measured with 'test_coding_rules'
already built):

                i686  x86_64
  before PCRE:  3.93   3.41
  after  PCRE:  3.24   3.12 [here, both use an ELF binary]

but parallelism can increase speed, later, and it's well worthwhile
to get rid of boost-1.33.1 once and for all.