[Gcl-devel] Re: gcl on intel mac

[Matt Kaufmann]

Hi, Camm --

I'm impressed by the complexity with which you have to deal!  But I
don't know this kind of technical stuff; I just operate as a dumb Lisp
user.  Regarding:

>> Are the
>> ppc machines still to be supported?

Although I have a ppc Mac on my desktop, I have an Intel Mac laptop
that I use for actual work.  (The ppc Mac is basically a terminal.)
So if we generalize from me as a single data point, ppc isn't
important: I'd be very happy to have GCL running on my Intel Mac and
wouldn't really care much about it running on my ppc Mac.

Thanks --
-- Matt
   From: Camm Maguire <address@hidden>
   Date: Tue, 29 Jun 2010 19:47:30 -0400
   X-SpamAssassin-Status: No, hits=-2.6 required=5.0
   X-UTCS-Spam-Status: No, hits=-242 required=165

   Greetings!  Pleased to announce that I've got this working, though the
   final incarnation is still not clear in my mind.

   1) gcc-4.0, the standard on this box, passes all function calls
   through a jump table.  gcc-4.2 removes the section entirely and does
   calls directly.  This is of course one instruction faster.  The
   former, however, allows one not to relocate the .text section at all,
   as there are no external references.  (The other local section
   referenced is a "pointers" section, which gcc-4.2 also emits.  This
   definitely appears slower than say i386 Linux, as one has to do an
   extra copy and address indirection for each referenced variable. But
   there appears to be no choice here.)  

   The reason why skipping the relocation might be important is because
   the bfd library does not do this correctly, and gcl will have to
   override some of its functions.  Which means expanding, not
   contracting, our local patch, and this begins to appear fragile.  So
   question one is whether gcc-4.2 needs supporting on the mac.  On a
   totally open source platform this question is ridiculous as one must
   keep abreast with the latest, but I've been told that Apple has
   decided not to make latest gcc available for licensing reasons (latest
   is 4.4, soon to be 4.5).  So what is the proper, hopefully stable
   target for this port?

   2) The relocation records for either compiler are experimentally
   observed to be either SECTDIFF_32, PAIR_32, DISP32, or 32.  I don't
   know if I can count on this, but this is all I can test.

   3) I've been told that mach-o has a 64bit format that differs in some
   respects.  Are these machines out there?  Are they different?  Are the
   ppc machines still to be supported?  The ppc code (existing courtesy
   of Aurelien) does work on Matt's machine with a few minor bit rot
   changes.  But it does require a separate relocation file against a
   much earlier version of the bfd library.  I have not merged this with
   the intel code under one lib yet.  

   This brings up the general question of the gcl 2.6.8 release.  I had
   wanted to fix the mac port before releasing this.  Currently, all
   Debian platforms have the latest which build all apps
   (maxima,acl2,axiom,hol88) without error on all 13 Debian platforms.
   All synchronous app versions are currently in Debian testing for the
   next release.  Debian as you all know feeds Ubuntu.  I'm hesitant
   about upgrading the local bfd library in the 2.6.8 source tree to the
   latest upstream source at this point, but perhaps for no good reason.
   Debian uses the externally supplied latest bfd without issue where it
   uses bfd at all.  gclcvs uses the local tree for mips and alpha
   patches, but this is not in heavy use yet.  I think Tim might on
   occasion use the local bfd library in the gcl source, but I really
   don't know who else uses this.  In any case, this might argue either
   to backport the intel mac work to the earlier bfd lib we have
   alongside Aurelien's stuff, or to write the reloc stuff without bfd
   for the intel mac.

   Along these lines, the local gmp sources in the gcl tree do not
   compile on the intel mac either.  I had to compile upstream latest
   externally, and patch that too (minor) to boot.  I've been told there
   is some mechanism for the user to just install gmp on the mac, but I
   don't know how hard this is, and whether therefore we need to provide
   the local convenience copy.

   4) There appears to be no explicit section flag (set by bfd at least)
   which indicates that all relocation records refer to locally appended
   "IMPORT" sections (jump_table, pointers).  So this is just an
   assumption at this point, but one that appears to be robust against a
   wide variety of code.

   5) It would appear that we can either a) try to integrate patches into
   bfd upstream, ... failed in the past, b) minimize a separately
   maintained patch preferably in one wrapper function to
   bfd_get_relocated_section_contents, or c) skip bfd and implement the
   relocation directly for the four relocation types mentioned above,
   and abort if others are found later.  This decision depends of course
   on the bugs, so here they are:

   a) No relocation records generated for jumptable and pointers
   sections, so bfd_get_relocated_section_contents returns garbage for
   these sections

   b) bfd_mach_o_canonicalize_one_reloc miscalculates the target
   symbol here:

         unsigned int num = BFD_MACH_O_GET_R_SYMBOLNUM (symnum);
         res->addend = 0;
         res->address = addr;
         if (symnum & BFD_MACH_O_R_EXTERN)
           sym = syms + num;
         else
           {
             bfd_mach_o_section *sect;
             bfd_mach_o_section *ssp=NULL;
             int j;

             assert (num != 0);
             assert (num <= mdata->nsects);

             /* sect=mdata->sections[num-1]; <-BUG */
             for (j=0;j<mdata->nsects;j++) /* Correct */
               if ((sect=mdata->sections[j]) && addr>=sect->addr && addr < 
sect->addr+sect->size)
                 break;
             sym = sect->bfdsection->symbol_ptr_ptr;

             ...

   c) The relocation types SECTDIFF_32 and PAIR_32, via their howto
   definitions, erronously add in the output_section vma.  Aurelien ran
   into the same I believe, leading to the solution of subtracting the
   vma from the relocation "addend" in a howto 'special function'.  The
   other platforms need this vma to be set as they use it correctly, so
   we would have to either conditionally not set it on this platform, or
   write a howto special function wrapper to set the addend, or directly
   patch bfd_mach_o_canonicalize_one_reloc, which needs doing anyway
   given the above.

   d) Likewise, the PAIR_32 addend is set to the address offset
   of the instruction, but this is already in place in the data, leading
   to it being added twice.  So similarly, the addend needs this quantity
   subtracted here.

   e) pc_relative relocs (DISP32) subtract the instruction address in
   bfd_get_relocated_section_contents, but the data is already relative
   so this is redundant.  The addend needs correcting as above in this
   case too. 

   The current patch 1) provides two helper functions to set the pointer
   and jumptable sections 2) overrides bfd_mach_o_canonicalize_one_reloc
   with the one bug fix in b), and 3) wraps
   bfd_get_relocated_section_contents to skip .text, and call the
   functions in 1) when appropriate.  Supporting gcc-4.2 will require
   relocating .text too, and patching bfd_mach_o_canonicalize_one_reloc
   with the changes for c, d, and e as well.  This is all doable, I'm
   mostly concerned with maintenance going forward.  Hence I'd love
   suggestions and feedback.

   Take care,
   -- 
   Camm Maguire                                     address@hidden
   ==========================================================================
   "The earth is but one country, and mankind its citizens."  --  Baha'u'llah