Re: [lmi] Best way to integrate PCRE

[Greg Chicares]

On 9/5/21 11:26 AM, Vadim Zeitlin wrote:
> On Fri, 27 Aug 2021 16:21:44 +0000 Greg Chicares <gchicares@sbcglobal.net> 
> wrote:
> 
> GC> On 8/21/21 6:37 PM, Vadim Zeitlin wrote:
> GC> > On Wed, 28 Jul 2021 19:45:30 +0000 Greg Chicares 
> <gchicares@sbcglobal.net> wrote:
> GC> > 
> GC> > GC> On 2021-07-28 18:45, Vadim Zeitlin wrote:
> GC> > GC> > On Wed, 28 Jul 2021 16:10:34 +0000 Greg Chicares 
> <gchicares@sbcglobal.net> wrote:
> [...]
> GC> I'm not as close to this as you are, so it's hard for me to say
> GC> whether one strategy or another is better. But I can speak in
> GC> terms of outcomes.
> 
>  Hello again,
> 
>  I'm sorry to return to this discussion, but after rereading this email
> carefully, I've realized that I'm very confused by it because it seems to
> directly contradict the "The solution is to withdraw msw support for those
> things." phrase from your earlier (2021-07-28 16:10:34) reply, so I need
> to understand whether we really can afford not building (a working version
> of) test_coding_rules.exe under MSW or not.

And I'm sorry I've been distracted by many other things recently.
The short answers I've tried to give so far show that I don't have
a full grasp of the implementation issues. At the risk of making
things worse again, before the next distraction begins forty-one
hours from now, let me try again. [You may want to speed-read
ahead to the magical 'make foo=bar' override solution below.]

I'd really like 'make check_concinnity' to keep working, no matter
what values `env |grep LMI_` shows. And if 'make check_concinnity'
uses a pc-linux-gnu 'test_coding_rules' binary for all architectures,
that's perfectly okay by me.

AIUI, you're trying to use PCRE for this binary without building a
PCRE library--which is unnecessary for pc-linux-gnu because we can
just install a prebuilt library from debian. For msw, however, there
doesn't seem to be a prebuilt MinGW-64 library (arch linux provides
one, but debian doesn't), and we'd rather avoid the nuisance of
building it ourselves.

I had hoped that we could build 'test_coding_rules' for pc-linux-gnu
only, and use it for all architectures. AFAICT, that would solve all
problems neatly enough.

> GC> (1) In this scenario:
> GC> 
> GC>   $ LMI_COMPILER=gcc ; LMI_TRIPLET=i686-w64-mingw32    ; . 
> /opt/lmi/src/lmi/set_toolchain.sh
> GC>   $ $make test_coding_rules.exe
> GC> 
> GC> if I get a useless binary that only says "not implemented",
> GC> that's okay.
> 
>  Yes, this is what would happen with my current changes.

So far so good. As long as we have a GNU/linux binary, no msw binary
is necessary (they'd both behave the same anyway).

> GC> (2) In a scenario where I may have done this several months ago:
> GC> 
> GC>   $ LMI_COMPILER=gcc ; LMI_TRIPLET=i686-w64-mingw32    ; . 
> /opt/lmi/src/lmi/set_toolchain.sh
> GC> 
> GC> and then, having forgotten about that, I do this:
> GC> 
> GC>   $ make $coefficiency check_concinnity 2>&1 |less -S -N
> GC> 
> GC> then I want a concinnity check, not a "not implemented" message.
> 
>  This is rather surprising. Wouldn't this be misleading, as it could result
> in a success even when there are coding rules violations? I would think
> that you would want this command to fail rather than to pretend that it
> succeeded when it didn't really perform all the checks.

Sorry I wasn't clearer. You're asking whether it would be okay for
any "not implemented" message to be suppressed, so that
'test_coding_rules' would silently fail, or perhaps not run at all.
That would not be okay, for the reasons you give.

What I had in mind instead is that the above commands would run a
full concinnity check, including a genuine 'test_coding_rules' run
that would invoke a pc-linux-gnu binary. Thus, the last line of the
'check_concinnity' makefile target would have to change, e.g. thus:

-       @-cd $(prefascicle_dir) && $(PERFORM) $(TEST_CODING_RULES) *
+       @-cd $(prefascicle_dir) && arch-independent-bindir/$(TEST_CODING_RULES) 
*

with corresponding changes elsewhere of course. [But speed-read
ahead to see something I think you'll like better.]

> GC> How could that be accomplished?
> 
>  The simplest is to just skip the invocation of $(TEST_CODING_RULES) in
> check_concinnity target in GNUmakefile, so it's not at all hard to do, but
> I'd appreciate if you could confirm that you actually want to skip the
> coding rules check in this case,

No, neither that...

> rather than give an error reminding you
> that you're not using the LMI_TRIPLET you might be thinking you use

...nor that. Instead, maintain an up-to-date 'test_coding_rules'
ELF binary for use by all architectures, rebuilding it as needed.
IOW, if that binary is out of date, and the environment contains
  LMI_TRIPLET=i686-w64-mingw32
then do something like this [here's the magic override]:
  LMI_TRIPLET=x86_64-pc-linux-gnu make custom_tools
overriding $LMI_TRIPLET only for that submake invocation, and
then use that ELF binary as above:
+       @-cd $(prefascicle_dir) && arch-independent-bindir/$(TEST_CODING_RULES) 
*

> (because, if I understand correctly, the only difference between the
> scenarios (1) and (2) is that in the first one you use this triplet
> intentionally but in the second one you only use it because you had set it
> before and forgot about setting it).

Yes...or I'm just too lazy to do type
  make LMI_TRIPLET=x86_64-pc-linux-gnu $coefficiency check_concinnity 2>&1 
|less -S -N
where I conjecture that the $LMI_TRIPLET override might just work.

To avoid confusion, though, let me actually try that:

/opt/lmi/src/lmi[0]$env |grep LMI_
LMI_COMPILER=gcc
LMI_TRIPLET=i686-w64-mingw32

/opt/lmi/src/lmi[0]$file 
/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship/test_coding_rules
/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship/test_coding_rules: ELF 64-bit LSB 
shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter 
/lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=58cde135ff486d02318c074f34276157fee0bd41, for GNU/Linux 3.2.0, 
with debug_info, not stripped

/opt/lmi/src/lmi[0]$file 
/opt/lmi/local/gcc_x86_64-pc-linux-gnu/bin/test_coding_rules 
/opt/lmi/local/gcc_x86_64-pc-linux-gnu/bin/test_coding_rules: ELF 64-bit LSB 
shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter 
/lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=58cde135ff486d02318c074f34276157fee0bd41, for GNU/Linux 3.2.0, 
with debug_info, not stripped

/opt/lmi/src/lmi[0]$rm 
/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship/test_coding_rules
/opt/lmi/src/lmi[0]$rm 
/opt/lmi/local/gcc_x86_64-pc-linux-gnu/bin/test_coding_rules 

/opt/lmi/src/lmi[0]$make LMI_TRIPLET=x86_64-pc-linux-gnu $coefficiency 
check_concinnity 2>&1 |less -S -N
make[1]: Entering directory '/opt/lmi/src/lmi'
g++ -o test_coding_rules c_regex_traits.o cpp_regex_traits.o cregex.o [and so 
on]
PERFORM= /opt/lmi/src/lmi/test_coding_rules_test.sh
Testing 'test_coding_rules'.
make[1]: Leaving directory '/opt/lmi/src/lmi'

And it all seems to work as hoped. Just to be sure it built:

/opt/lmi/src/lmi[0]$file 
/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship/test_coding_rules
/opt/lmi/gcc_x86_64-pc-linux-gnu/build/ship/test_coding_rules: ELF 64-bit LSB 
shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter 
/lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=58cde135ff486d02318c074f34276157fee0bd41, for GNU/Linux 3.2.0, 
with debug_info, not stripped
/opt/lmi/src/lmi[0]$file 
/opt/lmi/local/gcc_x86_64-pc-linux-gnu/bin/test_coding_rules
/opt/lmi/local/gcc_x86_64-pc-linux-gnu/bin/test_coding_rules: ELF 64-bit LSB 
shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter 
/lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=58cde135ff486d02318c074f34276157fee0bd41, for GNU/Linux 3.2.0, 
with debug_info, not stripped

Can we conclude now that we'd both be satisfied if I replaced my
habitual command:
  make $coefficiency check_concinnity 2>&1 |less -S -N
with this:
  make LMI_TRIPLET=x86_64-pc-linux-gnu $coefficiency check_concinnity 2>&1 
|less -S -N
instead? If that works, then perhaps it's ideal, as it would seem
to require no other change anywhere (except, I guess, in the
hooks that invoke that command).

It makes little difference to me if I have to specify an
overriding triplet, because I never type that command anyway
(I just recall it from history).

>  IOW, it's precisely because the outcome of the test doesn't depend on the
> platform and architecture that I think it should be ok to run it only under
> Linux and give an error under MSW.

Okay, I guess, as long as I can override the triplet as above,
so that I always get what I want. If I fail to override the
triplet, or if I override it incorrectly, that's my fault; but
I simply won't do that.

>  To summarize, I think the choice is between:
> 
> 0. Not changing anything in my current version, i.e. concinnity test does
>    _not_ work under MSW because there is no working test_coding_rules.exe
>    and you have to run it under Linux only.

...or, when LMI_TRIPLET=i686-w64-mingw32, you can still run it
as though the triplet were pc-linux-gnu, simply by explicitly
overriding the triplet.

> 1. Make concinnity test under MSW skip the coding rules test, but still
>    check all the rest. I think this is what you're proposing right now,
>    but I don't like this at all because I think it's really misleading.

No, never.

> 2. Make concinnity test under MSW use some other test_coding_rules.exe
>    that would come from somewhere else -- I only list this possibility
>    because I think that you might actually be proposing this

I am. (Without the '.exe' suffix.)

And this certainly does mean that anyone who tries this on an msw-only
system will experience failure. But nobody does that, not on this
continent at least.

>    rather than
>    (1), but I don't see at all how could this work, because this program
>    would need to be rebuilt whenever test_coding_rules.cpp changes, so I
>    don't think it's a viable solution. If we really want to test coding
>    rules under MSW, we need to build a functional version of the test
>    there -- apparently contradicting the previous decision to not do it.

Now I think the best way to thread this needle is simply to override
the triplet when invoking 'make check_concinnity'.

> if it's simpler for you, perhaps I could
> submit the patch in the current state, with concinnity test not working
> under MSW?

That sounds okay. But now I'll have only forty hours to test it,
which probably won't be enough because I need to prepare for my
next distraction.