[Axiom-developer] RE: gcl-2.6.8pre on MAC OSX 10.2

[Page, Bill]

Camm, 

On Wednesday, October 18, 2006 11:25 AM you wrote:
> > 
> > I have succeeded in compiling gcl-2.6.8pre on MAC OSX 10.2 on the
> > SourceForge compile farm 'ppc-osx3' server, however some patches
> > were necessary. This machine has Xcode installed by not Fink.
> > 
> > First, I checked out gcl-2.6.8pre from cvs on October 15, 2006,
> > created a tarball and scp'd it and the standard gnu gettext-0.15
> > and sed-4.1.4 tarballs to my home directory on SourceForge.
> > 
> > Next I compiled and installed gettext and sed with
> >   --prefix=/home/users/b/bi/billpage/osx
> > creating the ~/osx/bin and ~/osx/lib directories. These are
> > apparently required to satisfy the gcl build dependencies on
> > OSX 10.2. (Note: A Fink installation might also have provided
> > these in the /sw directory.)
> > 
> 
> I thought that gettext was no longer required, as it was included
> in the local bfd build (from configure.in:)
> 
>       cd binutils/bfd && chmod +x configure && ./configure 
> --with-included-gettext && cd ../..
>

If I remove my local osx/bin from the PATH and try again I get
the error message "msgfmt: command not found" shown below.

---------

$ ./configure --prefix=/home/users/b/bi/billpage/osx \
     --disable-tkconfig  --disable-statsysbfd --enable-locbfd
$ make
cd binutils/intl && make
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  intl-compat.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  bindtextdom.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  dcgettext.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  dgettext.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  gettext.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  finddomain.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  loadmsgcat.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  localealias.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  textdomain.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  l10nflist.c
gcc -c -DLOCALEDIR=\"/usr/local/share/locale\"
-DGNULOCALEDIR=\"/usr/local/share/locale\"
-DLOCALE_ALIAS_PATH=\"/usr/local/share/locale:.\" -DHAVE_CONFIG_H -I.
-I.  -g -O2  explodename.c
rm -f libintl.a
ar cru libintl.a intl-compat.o bindtextdom.o dcgettext.o dgettext.o
gettext.o finddomain.o loadmsgcat.o localealias.o textdomain.o
l10nflist.o explodename.o
ranlib libintl.a
cd binutils/bfd && make
make  all-recursive
Making all in doc
make[3]: Nothing to be done for `all'.
Making all in po
( if test 'x.' != 'x.'; then \
    posrcprefix='../'; \
  else \
    posrcprefix="../"; \
  fi; \
  rm -f SRC-POTFILES-t SRC-POTFILES \
    && (sed -e '/^#/d' \
            -e '/^[     ]*$/d' \
            -e "address@hidden@   $posrcprefix& \\\\@" < ./SRC-POTFILES.in \
        | sed -e '$s/\\$//') > SRC-POTFILES-t \
    && chmod a-w SRC-POTFILES-t \
    && mv SRC-POTFILES-t SRC-POTFILES )
( rm -f BLD-POTFILES-t BLD-POTFILES \
    && (sed -e '/^#/d' \
            -e '/^[     ]*$/d' \
            -e "address@hidden@   ../& \\\\@" < ./BLD-POTFILES.in \
        | sed -e '$s/\\$//') > BLD-POTFILES-t \
    && chmod a-w BLD-POTFILES-t \
    && mv BLD-POTFILES-t BLD-POTFILES )
cd .. \
  && CONFIG_FILES=po/Makefile.in:po/Make-in \
     CONFIG_HEADERS= /bin/sh ./config.status
config.status: creating po/Makefile.in
config.status: executing depfiles commands
config.status: executing default commands
file=./`echo fr | sed 's,.*/,,'`.gmo \
  && rm -f $file && PATH=../src:$PATH msgfmt -o $file fr.po
/bin/sh: msgfmt: command not found
make[3]: *** [fr.gmo] Error 127
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [binutils/bfd/libbfd.a] Error 2

---------

gcl-2.6.8pre/binutils/bfd/config.log confirms:

 Invocation command line was

  $ ./configure --with-included-gettext

But apparently recursive makefile in bfd/po does not make
use of the included gettext. Maybe this is a binutils bug? 
 
> We do need sed.  I guess OSX has none, or it is incompatible?
>

This OSX 10.2 has an old sed that is not compatible.
 
> Is Fink so non-standard that it must be avoided?  What is
> the canonical way to get gnu software on OSX?
> 

I don't know. This SourceForge server is the closest I have
ever been to a MAC and it's probably a few thousand miles
away from here... :-) But I have heard that many people would
like to try to avoid installing "foreign" package systems like
Fink.

> 
> > Then I added:
> > 
> >   export PATH=/home/users/b/bi/billpage/osx/bin:$PATH
> >   export 
> LIBRARY_PATH=/home/users/b/bi/billpage/osx/lib:$LIBRARY_PATH
> >   export CPPFLAGS="-no-cpp-precomp"
> >   cd osx
> > 
> > to ~/.profile so that after re-login the environment was set
> > appropriately.
> > 
> > I untarred gcl into the osx directory creating ~/osx/gcl-2.6.8pre
> > Then I applied the following patches (most of which have been
> > previously reported on the gcl email list by other people):
> > 
> > ------------------------
> > ppc-osx3:~/osx billpage$ diff -Naur old/gcl* new/gcl*
> > 
> > This patch required so libintl is found in $LIBRARY_PATH.
> > 
> > diff -Naur old/gcl-2.6.8pre/h/powerpc-macosx.defs
> > new/gcl-2.6.8pre/h/powerpc-macosx.defs
> > --- old/gcl-2.6.8pre/h/powerpc-macosx.defs      Thu Jul 15 
> 09:28:43 2004
> > +++ new/gcl-2.6.8pre/h/powerpc-macosx.defs      Sun Oct 15 
> 22:07:45 2006
> > @@ -6,7 +6,7 @@
> > 
> >  # Set this to avoid warnings when linking against libncurses.
> >  # This is due to the requirements of the two level namespace.
> > -LIBS := `echo $(LIBS) | sed -e 's/-lncurses/ /'` 
> /sw/lib/libintl.dylib
> > +LIBS := `echo $(LIBS) | sed -e 's/-lncurses/ /'` -lintl
> > 
> >  # Set this for the linker to operate correctly.
> >  MACOSX_DEPLOYMENT_TARGET = 10.2
> > @@ -32,4 +32,4 @@
> >  # This appears to be no longer necessary on Panther.
> >  ARRS = libtool -static -o
> > 
> > -FINAL_CFLAGS := `echo $(FINAL_CFLAGS) | sed -e 's:-g::g'`
> > \ No newline at end of file
> > +FINAL_CFLAGS := `echo $(FINAL_CFLAGS) | sed -e 's:-g::g'`
> > 
> 
> OK, this is going in, but I think we can lose the -lintl too.
> Anyone want to try to verify?
>

I think you are right.

If I 'make clean' and write:

  +++ new/gcl-2.6.8pre/h/powerpc-macosx.defs      Sun Oct 15 
  ...
  +LIBS := `echo $(LIBS) | sed -e 's/-lncurses/ /'`

and just adding:

  export PATH=/home/users/b/bi/billpage/osx/bin:$PATH

(to avoid gettext bug) then trying configure & make as above
works fine.

 
> > This patch is required to define sbrk.
> > 
> > diff -Naur old/gcl-2.6.8pre/h/powerpc-macosx.h
> > new/gcl-2.6.8pre/h/powerpc-macosx.h
> > --- old/gcl-2.6.8pre/h/powerpc-macosx.h Thu Dec  8 17:31:25 2005
> > +++ new/gcl-2.6.8pre/h/powerpc-macosx.h Sun Oct 15 21:32:23 2006
> > @@ -38,8 +38,9 @@
> >  #undef SET_REAL_MAXPAGE
> >  #define SET_REAL_MAXPAGE { my_sbrk(0); real_maxpage = (int)
> > mach_maplimit/PAGESIZE; }
> > 
> > -#define sbrk my_sbrk
> > +#include <unistd.h> /* to get sbrk defined */
> >  extern void *my_sbrk(int incr);
> > +#define sbrk my_sbrk
> > 
> > 
> >  /** (si::save-system "...") a.k.a. unexec implementation  */
> > 
> 
> I don't get this one, as we emulate sbrk here.  Where is the system
> definition required?
>

If I reverse this change and do 'make clean & configure & make'
I get the error "conflicting types for `my_sbrk'":

cd unixport && make saved_pre_gcl
ls: ../xgcl-2/*.o: No such file or directory
ls: ../mod/*.o: No such file or directory
ls: ../pcl/*.o: No such file or directory
ls: ../clcs/*.o: No such file or directory
ls: ../clcs/clcs_*.lisp: No such file or directory
gcc  -Wall -DVOL=volatile -fsigned-char -pipe -O3 -fomit-frame-pointer
-I/home/users/b/bi/billpage/osx/new/gcl-2.6.8pre/o   -c -o sys_pre_gcl.o
sys_pre_gcl.c
../h/../h/protoize.h:465: warning: could not use precompiled header
'/usr/include/unistd-gcc3.p', because:
../h/../h/protoize.h:465: warning: macro 'valloc' defined by
../h/config.h conflicts with precomp
In file included from ../h/../h/protoize.h:465,
                 from ../h/include.h:70,
                 from sys_pre_gcl.c:3:
/usr/include/unistd.h:203: conflicting types for `my_sbrk'
../h/config.h:42: previous declaration of `my_sbrk'
make[1]: *** [sys_pre_gcl.o] Error 1
make: *** [unixport/saved_pre_gcl] Error 2

-------

> > This patch is required to remove functions symbols from plt.
> > 
> > diff -Naur old/gcl-2.6.8pre/o/makefile new/gcl-2.6.8pre/o/makefile
> > --- old/gcl-2.6.8pre/o/makefile Fri Sep 15 10:45:18 2006
> > +++ new/gcl-2.6.8pre/o/makefile Mon Oct 16 22:03:52 2006
> > @@ -154,7 +154,7 @@
> >                              print a}' \
> >                         k=$(LEADING_UNDERSCORE) |\
> >                         sort | \
> > -                       grep -v '[^ \t_]_' |\
> > +                       grep -v 'restFP' | grep -v 'saveFP' 
> | grep -v
> > '[^ \t_]_' |\
> >                         $(AWK) '{A[++k]=$$0} END {for 
> (i=1;i<=k;i++) \
> >                                 
> printf("MY_PLT(%s)%s\n",A[i],i==k ? "" :
> > ",");}' >$@
> > 
> 
> OK.  This is quite ugly, but its a hack to begin with.  Suggestions
> most welcome.
>

I only vaguely understand what this is doing but 'restFP' and
'saveFP' are not removed from this list than the compile complains
about "function symbols" not used in a function ... or something
like that. The other names make sense to me. For example:

MY_PLT(__srget),
MY_PLT(__swbuf),
...

which derive from the implementation of getc(f) and putc(ch,f).
But see later in the message - I apparently have a problem with
__srget.

> > This patch is required to find malloc.h on some OSX machines.
> > 
> > diff -Naur old/gcl-2.6.8pre/o/unexmacosx.c
> > new/gcl-2.6.8pre/o/unexmacosx.c
> > --- old/gcl-2.6.8pre/o/unexmacosx.c     Thu Dec 15 10:48:43 2005
> > +++ new/gcl-2.6.8pre/o/unexmacosx.c     Tue Oct 17 18:55:04 2006
> > @@ -124,7 +124,13 @@
> >  #endif
> >  #include <mach-o/nlist.h>
> >  #include <mach-o/getsect.h>
> > +/* not <sys/malloc.h> */
> > +/* not <malloc.h> */
> > +#if defined (HAVE_MALLOC_MALLOC_H)
> >  #include <malloc/malloc.h>
> > +#else
> > +#include <objc/malloc.h>
> > +#endif
> > 
> >  #include <sys/mman.h>
> > 
> 
> OK, I take it you are using objc/malloc.h.  We need configure code
> to look for this, and bomb if it cannot find one, just on macosx.
> I'll take a stab unless someone else wants to.
>

Makes sense to me. You are the autoconf wizard. :-)
 
> May I suggest we also lose the diagnostic output on save-system
> on this platform?

Yes, it looks pretty ugly but maybe it is needed just a little
while longer (or optionally) to verify that si::save-system and
compiler::link are really working properly?

> 
> Thanks again!  Will post when something is checked in.
>

Thank you. I look forward to a finally finalized 2.6.8. The
evoluton of 2.6.8pre is causing us a little consternaton in
the current Axiom source distribution... :-)
 
> BTW, I'm pushing 2.6.8 (in the guise of 2.6.7) and all the apps
> through the Debian autobuilder system before releasing 2.6.8.
> Please let me know of any other 2.6.8 issues you may be
> encountering in your axiom work asap.  Bug fix only at this
> point, of course.
>

Hmmmm... well I do currently have a problem with the Axiom build.
After successfully building bootsys and depsys the build fails
building interpsys with the following message:

start address -T 0xb8f000 Finished loading
/home/users/b/bi/billpage/osx/axiom.build-improvements/obj/powerpc-apple
-darwin6.8/interp/bookvol5.o
Loading
/home/users/b/bi/billpage/osx/axiom.build-improvements/obj/powerpc-apple
-darwin6.8/interp/util.o

Error: Undefined symbol "___srget"
Fast links are on: do (si::use-fast-links nil) for debugging
Error signalled by LOAD.
Broken at LOAD.  Type :H for Help.
>>make[2]: ***
[/home/users/b/bi/billpage/osx/axiom.build-improvements/build/powerpc-ap
ple-darwin6.8/bin/interpsys] Error 255
make[1]: *** [interp/stamp] Error 2
make: *** [all-recursive] Error 1

---------

Something is strange about thid symbol "___srget" with the 3
underscore characters, I think??? The name "__srget" with 2
underscore characters is properly defined in /usr/include/stdio.h

I don't understand what is going on here.

Also prior to compiling depsys, bootsys was already successfully
created however it did have one oddity. The original Axiom load
commands like ')load postpar' run during building depsys fails
with an error message like "'postpar.8' does not exist" (Yes, that's
the digit 8 after the dot.). If I change the command to include the
.o like this: ')load postpar.o' everything seems fine and depsys
is built.

bootsys itself is actually built form a copy of gcl called 'lisp'
that is created using compiler::link. The 'lisp' image includes
several Axiom specific external routines. I.e.

echo '(compiler::link nil
 
"/home/users/b/bi/billpage/osx/axiom.build-improvements/build/powerpc-ap
ple-darwin6.8/bin/lisp" ' \
              ' (format nil "(progn (let ((*load-path* (cons ~S
*load-path*))'\
              ' (si::*load-types* ~S))' \
              ' (compiler::emit-fn t))' \
              ' (when (fboundp (quote si::sgc-on))' \
              ' (si::sgc-on t))' \
              ' (setq compiler::*default-system-p* t))"' \
              ' si::*system-directory* (quote (list ".lsp")))' \
              '
"/home/users/b/bi/billpage/osx/axiom.build-improvements/lsp/.././src/lib
/cfuns-c.o' \
               '
/home/users/b/bi/billpage/osx/axiom.build-improvements/lsp/.././src/lib/
sockio-c.o' \
               '
/home/users/b/bi/billpage/osx/axiom.build-improvements/lsp/.././src/lib/
libspad.a")' \
 | /home/users/b/bi/billpage/osx/bin/gcl

If I intervene and make Axiom use the original 'saved_gcl' to build
'bootsys' instead of using 'lisp', then the 'postpar.8' problem does
not occur and gcl finds the .o files anyway, as expected.

This makes me suspicious that something subtle may be wrong with
the output of 'compiler:link'. The size of the result images also
seem curious:

-rwxr-xr-x  1 billpage  100  18362444 Oct 17 19:08 saved_gcl
...
-rwxr-xr-x  1 billpage  100  13072984 Oct 18 04:01 lisp
-rwxr-xr-x  1 billpage  100  19159640 Oct 18 04:01 bootsys
-rwxr-xr-x  1 billpage  100   7719512 Oct 18 04:01 raw_lisp.tmp
-rw-r--r--  1 billpage  100         0 Oct 18 04:01 raw_lisp_map
-rwxr-xr-x  1 billpage  100  49588824 Oct 18 03:10 depsys

Remember that 'lisp' is create by 'compiler::link' from
saved_gcl plus some externals. Why is it smaller? Also the
"raw" files were left here don't look "normal" to me.

A test image of gcl created by

  $ gcl
  (si:save-system "test-image")
  (quit)

is actually *larger* than the original saved_gcl.

-rwxr-xr-x  1 billpage  100  23699532 Oct 18 11:07 test-image

Are all these problems related?

Any thing you can suggest would be greatly appreciated.

Regards,
Bill Page.

> > 
> > ---------
> > 
> > The resulting gcl binary (unixport/saved_gcl) in available here:
> > 
> >   http://page.axiom-developer.org/gcl-2.6.8-osx10-20061017.bin
> > 
> > I would be very happy if anyone with a MAC OSX machine would try
> > this version of gcl on their systems and let me know of any
> > problems.
> > 
> > I am currently working on completing the Axiom build based on the
> > new build-improvements branch.
> >