[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [lmi] Building shared zlib
From: |
Greg Chicares |
Subject: |
Re: [lmi] Building shared zlib |
Date: |
Tue, 29 Aug 2017 15:09:36 +0000 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 |
On 2016-07-15 21:11, Vadim Zeitlin wrote:
> On Fri, 15 Jul 2016 21:04:16 +0000 Greg Chicares <address@hidden> wrote:
[...problems building zlib for use with libxml2, which now reappear--see
below...]
> GC> Where do we go from here?
> GC> (1) Use './configure && make' and fix up the problems manually?
> GC> (2) Try to fix the autotoolization problem?
>
> As an aside, this is how autotools undeservedly get bad reputation :-( The
> problem is that zlib has decided _not_ to use them, hence all this
> weirdness.
>
> GC> (3) Use the msw-specific makefile provided by zlib?
> GC> (4) Just use the zlib.org msw binaries?
> GC> I rate those options 2 > 4 > 1 > 3 iff (2) is easy (it's not easy for me),
> GC> or 4 > 2 > 1 > 3 otherwise. If we're stuck with an msw-specific solution,
> GC> then 4 >>> 3 because it's less work and less prone to error. I think I'll
> GC> proceed with (4) provisionally; if (2) is feasible, we can switch to it.
>
> My personal preference is still (3) for the same reasons as stated before
> (to recap: if we build some third party libraries, let's build all of them)
> and I don't see any real drawbacks to it. (2) is easy if you agree to not
> have the version information in the generated DLL, otherwise it isn't.
Before presenting the latest problems, let me ask whether there are
other, better options:
(5) Use a zlib built by wx
(6) Use zlib binaries provided by the MinGW-w64 maintainers
(7) Drop zlib and use lzma instead
The principal motivation, as explained here:
http://lists.nongnu.org/archive/html/lmi/2016-06/msg00046.html
is to use compression as (weak) encryption, and lzma would serve
that purpose just as well as zlib. Speed shouldn't be a problem:
https://tukaani.org/lzma/benchmarks.html
| Decompression In terms of speed, gzip is the winner again.
| lzma comes right behind it two to three times slower than gzip.
Because we've upgraded to libxml2-2.9.4, which was released
2016-05-23 and is still the latest:
http://xmlsoft.org/news.html
we're able to use lzma, which has been supported since 2012-01:
https://mail.gnome.org/archives/commits-list/2012-January/msg11021.html
Option (7) sounds most promising in light of all the difficulties
we've had with zlib--so promising that I think we should try it
before discussing the others. I won't delete what I had already
written about them below, lest that become useful someday, but
nobody ever has to read it if lzma works.
(5) I've seen this in wx build logs:
configure: WARNING: zlib library not found or too old, will use built-in
instead
Does that mean that wx is capable of building a zlib that actually
might work with libxml2? I see
libwxzlib-3.1-i686-w64-mingw32.a
in my latest msw cross build, but no corresponding DLL. I guess I've
asked that question before, and you replied here:
http://lists.nongnu.org/archive/html/lmi/2016-06/msg00047.html
(6) Use binaries provided by the MinGW-w64 project. I think that's
what's been repackaged here by debian:
https://packages.debian.org/stretch/all/libz-mingw-w64/filelist
and it contains both 32- and 64-bit DLLs:
/usr/i686-w64-mingw32/lib/zlib1.dll
/usr/x86_64-w64-mingw32/lib/zlib1.dll
OTOH, a freshly installed MinGW-w64 package contains static zlib
binaries, but not the DLLs that libxml2 apparently needs:
/MinGW_[0]$ls -1 **/libz*
i686-w64-mingw32/lib/libz.a
i686-w64-mingw32/lib64/libz.a
i686-w64-mingw32/lib64/libzoneoc.a
/MinGW_[0]$ls -1 **/zlib*
i686-w64-mingw32/include/zlib.h
opt/lib/python2.7/encodings/zlib_codec.py
opt/lib/python2.7/encodings/zlib_codec.pyc
opt/lib/python2.7/encodings/zlib_codec.pyo
opt/lib/python2.7/lib-dynload/zlib.pyd
licenses/zlib:
copying.txt
and, curiously, MinGW-w64 has a separate zlib package:
https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/ray_linn/64bit-libraries/zlib/
but it contains only 64-bit binaries...so maybe the debian
maintainer built the binaries himself from scratch.
[Then I thought of option (7), as described above.]
Anyway, although building zlib for use with libxml2 seemed to be
working for a while, now it's not working. Excepted from the output
of 'install_msw.sh' on a machine in the office:
cd /opt/lmi/xml-scratch/zlib-1.2.8 \
&& export PATH="/MinGW_/bin/:/usr/local/bin: [...]
[hundreds of characters redacted from $PATH]
&& LDSHARED='gcc -shared -Wl,--out-implib,libz1.dll.a' LDSHAREDLIBC=''
SHAREDLIBV='libz1.dll' ./configure --prefix=/opt/lmi/local
--eprefix=/opt/lmi/local \
&& make \
&& make install \
Checking for gcc...
Checking for shared library support...
Building shared library libz1.dll with gcc.
Checking for off64_t... Yes.
Checking for fseeko... Yes.
Checking for strerror... Yes.
Checking for unistd.h... Yes.
Checking for stdarg.h... Yes.
Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
Checking for vsnprintf() in stdio.h... Yes.
Checking for return value of vsnprintf()... Yes.
Checking for attribute(visibility) support... No.
make[1]: Entering directory '/opt/lmi/xml-scratch/zlib-1.2.8'
gcc -O3 -D_LARGEFILE64_SOURCE=1 -I. -c -o example.o test/example.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o adler32.o adler32.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o crc32.o crc32.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o deflate.o deflate.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o inffast.o inffast.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o infback.o infback.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o inflate.o inflate.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o inftrees.o inftrees.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o zutil.o zutil.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o trees.o trees.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o compress.o compress.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o uncompr.o uncompr.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o gzclose.o gzclose.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o gzlib.o gzlib.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o gzread.o gzread.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -c -o gzwrite.o gzwrite.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -I. -c -o minigzip.o test/minigzip.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/adler32.o adler32.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/crc32.o crc32.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/infback.o infback.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/deflate.o deflate.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/inffast.o inffast.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/inftrees.o inftrees.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/inflate.o inflate.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/trees.o trees.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/zutil.o zutil.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/compress.o compress.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/uncompr.o uncompr.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/gzclose.o gzclose.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/gzlib.o gzlib.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -I. -D_FILE_OFFSET_BITS=64 -c -o example64.o
test/example.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/gzread.o gzread.c
gcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DPIC -c -o objs/gzwrite.o gzwrite.c
gcc -O3 -D_LARGEFILE64_SOURCE=1 -I. -D_FILE_OFFSET_BITS=64 -c -o minigzip64.o
test/minigzip.c
ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o
inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o
gzwrite.o
gcc -O3 -D_LARGEFILE64_SOURCE=1 -o example.exe example.o -L. libz.a
gcc -O3 -D_LARGEFILE64_SOURCE=1 -o minigzip.exe minigzip.o -L. libz.a
gcc -O3 -D_LARGEFILE64_SOURCE=1 -o example64.exe example64.o -L. libz.a
gcc -O3 -D_LARGEFILE64_SOURCE=1 -o minigzip64.exe minigzip64.o -L. libz.a
gcc -shared -Wl,--out-implib,libz1.dll.a -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -o
libz1.dll adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo
inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo
gzread.lo gzwrite.lo
rm -f libz.so libz.so.1
ln -s libz1.dll libz.so
ln -s libz1.dll libz.so.1
gcc -O3 -D_LARGEFILE64_SOURCE=1 -o examplesh.exe example.o -L. libz1.dll
gcc -O3 -D_LARGEFILE64_SOURCE=1 -o minigzipsh.exe minigzip.o -L. libz1.dll
make[1]: Leaving directory '/opt/lmi/xml-scratch/zlib-1.2.8'
make[1]: Entering directory '/opt/lmi/xml-scratch/zlib-1.2.8'
cp libz.a /opt/lmi/local/lib
chmod 644 /opt/lmi/local/lib/libz.a
cp libz1.dll /opt/lmi/local/lib
chmod 755 /opt/lmi/local/lib/libz1.dll
cp zlib.3 /opt/lmi/local/share/man/man3
chmod 644 /opt/lmi/local/share/man/man3/zlib.3
cp zlib.pc /opt/lmi/local/lib/pkgconfig
chmod 644 /opt/lmi/local/lib/pkgconfig/zlib.pc
cp zlib.h zconf.h /opt/lmi/local/include
chmod 644 /opt/lmi/local/include/zlib.h /opt/lmi/local/include/zconf.h
make[1]: Leaving directory '/opt/lmi/xml-scratch/zlib-1.2.8'
That appears to have succeeded. And libxml2 seems to build okay until:
CCLD libxml2.la
*** Warning: linker path does not have real file for library -lz.
*** I have the capability to make that library automatically link in when
*** you link to this library. But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because I did check the linker path looking for a file starting
*** with libz and none of the candidates passed a file format test
*** using a file magic. Last file checked:
C:/opt/lmi/MinGW-6_3_0/i686-w64-mingw32/lib/libz.a
*** The inter-library dependencies that have been dropped here will be
*** automatically added whenever a program is linked with this library
*** or is declared to -dlopen it.
...which soon leads to:
install_libxml2_libxslt.make:191: recipe for target 'libxml2-2.9.4' failed
make: *** [libxml2-2.9.4] Error 2
Oddly, everything just seems to work in my Cygwin VM, which uses
approximately the same versions of all cygwin packages. One notable
difference is that I have these files:
1d9871d4ffd71645ac3d029d8de0e03a
*/cache_for_lmi/downloads/zlib128-dll/DLL_FAQ.txt
624cb4c112f0f393af1b182598e62272 */cache_for_lmi/downloads/zlib128-dll/foo.gz
0af2d6af45fd78e4f288583333978cce
*/cache_for_lmi/downloads/zlib128-dll/include/zconf.h
64d8a5180bd54ff5452886e4cbb21e14
*/cache_for_lmi/downloads/zlib128-dll/include/zlib.h
4508be33473b384e424a344b37392889
*/cache_for_lmi/downloads/zlib128-dll/lib/zdll.lib
093438656dfa76f4f88614a0a941e57f
*/cache_for_lmi/downloads/zlib128-dll/lib/zlib.def
51927169536b2934a200945ac55a694d
*/cache_for_lmi/downloads/zlib128-dll/README.txt
6333538c2e1b14b27833aa4b212d9024
*/cache_for_lmi/downloads/zlib128-dll/test/example_d.exe
d8532a804c43d9947929153334453b12
*/cache_for_lmi/downloads/zlib128-dll/test/minigzip_d.exe
092246147f60e83bcf378167f0626149 */cache_for_lmi/downloads/zlib128-dll/USAGE.txt
092246147f60e83bcf378167f0626149
*/cache_for_lmi/downloads/zlib128-dll/USAGE.txt.bak
007488d386fe558a18b202fc4374e5b6
*/cache_for_lmi/downloads/zlib128-dll/USAGE.txt.diff
36885fae5e9f82de42d6f2d0edda4935
*/cache_for_lmi/downloads/zlib128-dll/USAGE.txt.modified
b8a9e91134e7c89440a0f95470d5e47b */cache_for_lmi/downloads/zlib128-dll/zlib1.dll
which are not present on the corporate machine. They aren't in
tar -tzvf /cache_for_lmi/downloads/zlib-1.2.8.tar.gz
so one hypothesis is that I installed some binary package that
provides them, perhaps as an early experimental alternative to
building zlib from source. Other than that, I don't see any
major differences.
But at this point I abandoned this line of research because the
lzma idea occurred to me.
- Re: [lmi] Building shared zlib,
Greg Chicares <=
- Re: [lmi] Building shared zlib, Vadim Zeitlin, 2017/08/29
- Re: [lmi] Building shared zlib, Greg Chicares, 2017/08/29
- [lmi] Ignore "harmless" 'configure' warnings? [Was: Using libxml2 with compression], Greg Chicares, 2017/08/30
- Re: [lmi] Ignore "harmless" 'configure' warnings?, Vadim Zeitlin, 2017/08/30
- Re: [lmi] Ignore "harmless" 'configure' warnings?, Greg Chicares, 2017/08/30
- Re: [lmi] Ignore "harmless" 'configure' warnings?, Vadim Zeitlin, 2017/08/30