Re: [lmi] Cross-compiling lmi from Linux to MSW

[Vadim Zeitlin]

On Thu, 21 Jan 2016 01:52:31 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2016-01-14 01:35, Vadim Zeitlin wrote:
GC> [...]
GC> > This was the most complicated part. The really aggravating thing is that I
GC> > could cross-compile the latest 1.60 version without too many problems: for
GC> > the future reference (and because I don't know whether this is really
GC> > documented anywhere, at least I didn't find anything else than indirect
GC> > references), let me just note that you need to do
GC> > 
GC> >   $ echo 'using gcc : mingw : i686-w64-mingw32-g++ ;' > ~/user-config.jam
GC> > 
GC> > and then you can build bjam and run
GC> > 
GC> >   $ ./tools/build/src/engine/bin.linuxx86_64/b2 \
GC> >       toolset=i686-w64-mingw32 target-os=windows filesystem regex
GC> 
GC> Could we not live with 1.49.0.1 here:
GC>   https://packages.debian.org/wheezy/libboost-all-dev
GC> if we actuallyneed to upgrade this thing?

 We could, but we'd still need to update the code using Boost.Filesystem
(because this version is new enough to use v3 of the library instead of the
ancient v2 API used in lmi) while still being stuck at a quite old version.
1.55 from Debian Jessie would be better but, as with libxml2 before, I
don't really see any reason to not use the latest version as we need to
compile it ourselves anyhow.

GC> It's a useful library, but its build system is painful.

 It definitely is, but they're probably going to switch to CMake soon and
I'm not sure if this can be counted as an improvement.

GC> > Notice that the commands above do not use "-std=c++11" for compiling for
GC> > the good reason that Boost 1.38 doesn't build with this option (there are
GC> > many, many errors due to the fact that this old version used pre-standard
GC> > support for things like rvalue references which worked differently from 
the
GC> > real C++11, so there is really no chance of fixing this). Because of this 
I
GC> > couldn't make things compile in C++11 mode for now, we absolutely have to
GC> > update Boost for this.
GC> 
GC> Yet it does seem to compile natively in msw with '-std=c++11',
GC> so isn't it really strange that the cross-compiler fails?

 Oops. I know what happened here: I used Boost 1.38 which is used in our
branch of lmi as we had to upgrade to it to use boost::math::expm1() and
log1p() 6+ years ago:

---------------------------------- >8 --------------------------------------
% git log -1 9d7067e
commit 9d7067e57b68db6a70922c7330ed00c8caa28217
Author: Vaclav Slavik <address@hidden>
Date:   Wed Dec 30 11:37:12 2009 +0100

    Upgrade Windows build system to Boost 1.38.0.

    This patch upgrades Boost used in the MSW build system from 1.33.1 to
    1.38.0. It is a prerequisite for the upcoming patch to use
    boost::math::expm1() and log1p().

    It fixes the problem with filesystem compilation (discussed here:
    http://lists.nongnu.org/archive/html/lmi/2008-06/msg00054.html) by
    tweaking compiler flags a bit. In other words, it doesn't yet change the
    way Boost libraries are built in none of the ways also mentioned in the
    above thread.
---------------------------------- >8 --------------------------------------

The patch using these functions which was submitted soon afterwards has
never been applied to lmi mainline and so it stayed at Boost 1.33. And
while Boost 1.38 is so old that it uses rvalue references with non-standard
semantics they had in the very early stages of C++11 development, Boost
1.33 is so incredibly ancient that it doesn't use rvalue references at all
and so, in virtue of its extreme age, can be compiled in C++11 mode when
1.38 can't.

 Sorry for not realizing that I was trying to building using a different
Boost version (I just looked at a wrong branch...), you're right that the
current version of Boost can be compiled in C++11 mode.

GC> And IIRC they changed the 'filesystem' design in some deep way, so
GC> "upgrading" that means a big lmi rewrite.

 Hardly. It does require patching lmi but most of the changes (all but one
or two IIRC) are pretty trivial. I wrote about the benefits of updating to
the v3 Boost.Filesystem API back in 2011:

    http://lists.nongnu.org/archive/html/lmi/2011-06/msg00033.html

and, of course, I think they're even greater today because this v3 API is
also proposed (and almost accepted) for the inclusion in the C++17 standard
as std::filesystem. So I believe it's well worth upgrading to the new API.

 Should I post the patch doing it (after, perhaps, more testing)?

GC> The only other not-all-headers boost library we use is 'regex',
GC> and that seems to be provided by gcc now that it's a standard:
GC>   i686-w64-mingw32/include/c++/regex

 Yes, 4.9 is the first g++ version to support std::regex (unfortunately,
std::filesystem is only supported in 5.x).

GC> If we switch 'regex' to the system library, then 'filesystem' is our
GC> only headache. If we sequester 'filesystem' in its 1.38 version, then
GC> we can upgrade boost easily any time, right?

 I hoped we could avoid using Boost at all with C++ but it looks like it
wouldn't be (easily) possible because of Boost.Any. Just FYI, currently we
use the following Boost headers:

---------------------------------- >8 --------------------------------------
% git grep boost/|sed -n 
'/#include/address@hidden<\([a-z_/]*\.hpp\)address@hidden@p'|sort|uniq
boost/algorithm/minmax_element.hpp
boost/any.hpp
boost/bind.hpp
boost/cast.hpp
boost/cstdint.hpp
boost/filesystem/convenience.hpp
boost/filesystem/exception.hpp
boost/filesystem/fstream.hpp
boost/filesystem/operations.hpp
boost/filesystem/path.hpp
boost/functional.hpp
boost/operators.hpp
boost/regex.hpp
boost/scoped_ptr.hpp
boost/shared_ptr.hpp
boost/static_assert.hpp
boost/type_traits/is_arithmetic.hpp
boost/type_traits/is_base_and_derived.hpp
boost/type_traits/is_base_of.hpp
boost/type_traits/is_convertible.hpp
boost/type_traits/is_enum.hpp
boost/type_traits/is_pointer.hpp
boost/type_traits/is_same.hpp
boost/type_traits/remove_reference.hpp
boost/utility/enable_if.hpp
---------------------------------- >8 --------------------------------------

GC> > We're currently using xmlwrapp 0.7.1 but there are a couple of problems
GC> > preventing it from being easily cross-compiled, in particular it requires 
a
GC> > couple of Boost library that we don't otherwise use.
GC> 
GC> /opt/lmi/third_party[0]$grep 'include.*boost' include/x*/**/* src/lib*/**/*
GC> src/libxml/pimpl_base.h:    #include <boost/pool/singleton_pool.hpp>
GC> 
GC> But that must be a headers-only library, because the msw-native lmi
GC> build has always worked and continues to work, right?

 Because it doesn't use configure.

GC> ...what boost libraries would need to be built? What am I missing here?

 When using configure in 0.7.1, it always looks for the Boost libraries
used by the tests (iostream and unit_test). Since then a --disable-tests
option was added to it to allow compiling the library even if these
libraries are not available.

GC> Now that we have the compiler working, this may be the perfect time to
GC> update the xmlsoft.org libraries. And normally I'd say that updating
GC> xmlwrapp is one of the easiest and safest upgrades we can make. I'm just
GC> worried about whatever instability is injected through boost--boost is
GC> beginning to feel like systemd.

 I think this is slightly unfair, the main problem is due to an
incompatible change in Boost.Filesystem API from v2 to v3 and it's just one
Boost library, although a pretty important one.


 To summarize, we can continue using Boost 1.33 for now because it's über
ancient enough to not have problems with C++11, so upgrading Boost is not
as urgent as I thought, sorry again for making this (honest) mistake (as
opposed to trying to trick you into updating Boost surreptitiously).

 However I still think that it would be worth it to:

(a) Replace as much as possible of Boost classes with C++11 ones now that
    we can do it (this includes scoped_ptr, shared_ptr, regex, ...).
(b) Update Boost to profit from C++11 support in the remaining libraries.
(c) Keep using boost::filesystem until we update to C++17 but at least use
    the std-compatible API for it (of course, we don't have any choice if
    we upgrade Boost anyhow).

 Regards,
VZ