Re: New Package Maintainer

[Sather User]

> From address@hidden Sat Dec 18 18:36:30 2010
> Date: Sat, 18 Dec 2010 08:05:59 +0000 (GMT)
> From: address@hidden
> To: address@hidden
> Subject: New Package Maintainer
>
> Hi Everyone!
>
>
> I'm the new maintainer of the Sather package. (all new to Sather and
> fsf!)

Fadi, congratulations!  (I didn't notice your appointment till a
couple of weeks ago.  I was in two hospitals for many weeks before and
many weeks after the date of your message, out of contact with the
world, having broken my left fibula near the ankle.)  Here is your
chance to give GNU Sather a new lease of life!

> (the current) Sather 1.2.3 compiles (and works) with :
> GNU/Linux i686 (kernel 2.6.18)
> - gcc 4.3.2
> - binutils 2.18.1
> - libdevel 6.8
> - libgc-dev 6.8
> - tk 8.4.16
> - tk-dev 8.4.16
>
>
> There is a bug in the browser. ("May crash later:")
> I think I'll start by looking at that (3-4 weeks?)...
> I have a lot to learn, and I think finding and fixing a bug is the
> best way to "get one's hands dirty".

Okay.  I guess by now you are well into it.  I haven't really looked
at the Browser.  I don't have that problem.  But I'm in the process of
trying to understand this stuff.  I'd like to know a little bit about
graphical debugging and POG, and see if the DualGui works, if pSather
works, and much more.  When I do "make optional" I find an executable
named dual_gui_server (at System/Platforms/dualgui) but I don't know
what to do with it but try cluelessly to run it.  It starts, waits,
and times out while I look dumbly on.

In my foggy groping in the direction of the Browser it seemed one way
was by writing baby test programs.  The Browser involves the class
TCL_TK and its callbacks, defined in the file
Library/System/TclTk/c_interface.c.

I would start by changing Tcl_GetResult (which does nothing) to
Tcl_GetStringResult in that file.  There may be error messages that
you were not seeing.  And by writing a few test programs such as:

-----------------------------------------------------------------

external C class MY_EX_TCL is
   Tcl_GetVar(interpreter:EXT_OB, varName:STR, flags:C_INT):EXT_OB;
   Tcl_GetStringResult(interpreter: EXT_OB): EXT_OB;
   Tcl_ResetResult(interpreter:EXT_OB);
end;

class MY_IN_TCL is
   const global_only:INT:=1;
   const leave_err_msg:INT:=512;
   include TCL_TK;                      -- <--------- TCL_TK
   Tcl_GetVar(varName:STR):STR is
      MY_EX_TCL::Tcl_ResetResult(interpreter);
      x::=MY_EX_TCL::Tcl_GetVar(interpreter,
                               varName,
                               #C_INT(global_only+leave_err_msg));
      if void(x) then
         y::=MY_EX_TCL::Tcl_GetStringResult(interpreter);
         if ~void(y) then
            #ERR+"What, "+varName+"?? Naughty! Naughty!\n";
            x:=y
         end
      end;
      return STR::create_from_c_string(x)
   end
end;

class MAIN is
   const s:STR:=
         "set f [open \"| gs -sDEVICE=bbox n6.ps |& cat\"]\n"
         "set i 0\n"
         "while {[gets $f line] > 0} {\n"
         "    set res([incr i]) $line\n"
         "    if {$i == 1} {\n"
         "        set gs_version [lindex $line 2]\n"
         "    }\n"
         "}\n"
         "close $f\n"
         "set res(nuffn) {}\n"
         "puts \"Good-bye.\"\n"
         "boo\n";
   main(args:ARRAY{STR}) is
      t::=MY_IN_TCL::create_no_window("","");
      protect
         n::=t.eval(s);
      when $STR then
         #ERR+exception+'\n'
      end;
      v::=t.Tcl_GetVar("gs_version");
      bb ::= t.Tcl_GetVar("res(4)");
      hbb ::= t.Tcl_GetVar("res(5)");
      nau ::= t.Tcl_GetVar("res(1000000)");
      s0 ::= t.Tcl_GetVar("res(nuffn)");
      o:OUT:=#;
      o+"version = "+v+"\n";
      o+"bb = "+bb+"\n";
      o+#FMT("hbb = %s\n", hbb);
      o+#FMT("nau = <>\n", nau);
      o+#FMT("nuffin = '%s'\n", s0);
   end
end
-----------------------------------------------------------------

and, with a new TCL class that does a Tcl_CreateInterp() and hardly
anything else beyond the above, let's add the command precinv to Tcl
so it can produce the inverse of its first argument to any desired
precision.  The example Tcl code prints the inverse of 19 to 50
significant figures:

-----------------------------------------------------------------

tclrat.module
-------------
-C_flag -ltcl
-C_flag -w
-main TEST_TCL
-o tclrat
tclrat.sa
shim.c


tclrat.sa
---------
external C class OUTER is
   come_back(val,prec:INT):C_CHAR_PTR is
      tell:STR := TEST_TCL::inv(val,prec);
      cptr:C_CHAR_PTR;
      SYS::inlined_C("#cptr=#tell->arr_part;");
      return cptr
   end;
   newcmd(interp:EXT_OB);
end;
class TEST_TCL is
   inv(number, precision:INT):STR is
      return #RAT(#INTI(1), #INTI(number)).str(precision);
   end;
   main is
      t ::= #TCL;
      OUTER::newcmd(t.interp);
      s ::= "set tn 19\n"
            "puts \"Inverses of $tn\"\n"
            "puts \"Native:    [format %e [expr 1.0/${tn}]]\"\n"
            "puts \"With RATs: [precinv $tn 50]\"";
      i::=t.Tcl_Eval(s);
      if i /= 0 then
         finres:STR :=  t.Tcl_GetStringResult;
         #OUT+"Result of Tcl_Eval = "+finres+"\n"
      end;
   end
end


shim.c
------
#include <stdlib.h>
#include <tcl.h>

int the_proc(ClientData c, Tcl_Interp *i, int argc, char *argv[])
{
  long num, prec;
  char *s;
  num = strtol(argv[1], NULL, 0);
  prec = strtol(argv[2], NULL, 0);
  s = (char *) OUTER_come_back(num,prec);
  Tcl_SetResult(i, s, TCL_VOLATILE);
  return TCL_OK;
}

newcmd(Tcl_Interp *interp)
{
  Tcl_CreateCommand(interp, "precinv", the_proc,
                    (ClientData)NULL, (Tcl_CmdDeleteProc *)NULL);
}

-----------------------------------------------------------------

In Browser/Tcl there are about 3400 lines of Tcl code, of which about
40 lines commence "tkkit_cb" or contain Tcl expressions commencing
"tkkit_cb".  In System/Platforms/gui/TclCode there are about 580
lines, of which 7 lines commence with the word "sather".  Those two
strings are new Tcl commands (like precinv above) which have been
added to Tcl by callbacks.  They are defined, sort of, in
Library/System/TclTk/c_interface.c.  But saying that is one thing,
grasping what it means is another.  To me, trying to understand how it
works is a challenge, a jigsaw puzzle that has pieces scattered all
over the place.  Where do you look first?  How does it hang together?

The idea of the TCL_TK callbacks, sather and tkkit_cb, is really (I
think) that you wouldn't write any C at all.  For that matter, you
wouldn't write any Tcl at all.  Because the Sather class TCL_TK is
about eliminating the points at which either happens, encapsulating
anything to do with another language in a Sather class, staying wholly
in Sather with its programming by contract, preconditions,
postconditions, invariants, type safety.  The rationale of the
language is compromised by diving into C or Tcl, doing things that are
"unsafe".

But then, the "capsules" don't fall from the sky.  Someone has to
write them, do the dirty work.  And debug a buggy Browser.

So, how do you do it?

One point about your bug hunt is that you probably need to be doing it
on a 32-bit computer if you are using GNU sather-1.2.3, because that
version has been intentionally crippled on x86_64 computers.  That is
the only new thing 1.2.3 does.  Its only other change is in the
copyright.  I sent a patch which fixes the problem to this list a
couple of years ago but the maintainer was not interested.  Probably
you should use 1.2.2 as the starting point for anything you do, just
by-pass 1.2.3.  But there also INT is hard-wired to be 32-bit, with
"const asize:INT:=32;".  'Twas ever thus.  The word size only becomes
configurable (at installation) with my patch or the equivalent.

The relevance: When a variable of type TCL_TK is created it has its
own Tcl interpreter, and callbacks such as tkkit_cb or sather_cb on a
TCL_TK variable must use the interpreter specific to the variable or
process.  Apparently this is made necessary by pSather, which might
have several interpreters using the callback in different threads.

         "Changed the order of the next two lines for Cliff's multiple
         interpreter version..."

(At Library/System/Gui/gui.sa, presumably referring to the Cliff
Draper mentioned in pLibrary/System/Socket/socket.sa.)

So, there is an FMAP (an associative array, in Tcl-talk; a hash, in
Perl-talk) associating each interpreter with its callback.  It might
be indexed by the interpreters except that they are EXT_OBs and trying
to do an is_eq:BOOL on a pair of them would likely spell trouble.  At
least, not a solid approach.

"Sather cannot follow these pointers." (Omohundro and Stoutamire)

So, the interpreter value, a Tcl_Interp* pointer, is cast to int by
to_int() in Library/System/TclTk/c_interface.c and the FMAP is indexed
with the corresponding INT.

(Incidentally, I think there is already an error there, with the (int)
cast.  See System/Common/c_header.h: "typedef long INT;".  The source
is riddled with mistaken casts to int which should be to long, one of
the things my unwanted patch addressed.)

So, you have Tcl on your 64-bit platform and you cast the Tcl_Interp*
to int or long, but GNU sather-1.2.3 has only 32-bit integers even on
x86_64.  Depending on the amount of RAM or on parameters
incomprehensible to me you may hit strife there.

Oops.  Just noticed that the above applies to sather and sather_cb but
not tkkit_cb.  It may be okay to use the Browser on x86_64.  It's just
the Gui and DualGui you need to avoid.  So the above is irrelevant to
your problem.  Will leave it, though.

Anyway, good luck.  If you are new to Sather and not lucky and not a
lot brighter than me (a very easy requirement) your 3-4 weeks may have
proved a slight under-estimate.

Notice that there is often useful documentation in the supplied
module files.  If you are working on the Browser the notes in
Library/Graphs/Graphs.module might be worth a look.

I don't have your problem and my Browser tree is unchanged from
Sather-1.2b (pre-GPL).  My only changes are in
Library/System/TclTk/c_interface.sa and
System/Platforms/tcltk/startup.tcl.  You might try reversing out the
differences in the Browser tree relative to sather-1.2b-gpl apart from
copyright changes.  If that doesn't fix it your problem is not in the
Browser tree but in Library/System/TclTk where I think it would have
to be in c_interface.sa, or else it is in System/Platforms/tcltk,
where startup.tcl has some obvious errors such as a puts with two
string arguments.  But c_interface.c fails to test the return value of
Tk_Init().  Might tell you something.

You can get sather-1.2b-gpl from the ICSI site and if you like I can
post a patch here which would allow you to compile it with modern GNU
tools.  Like current or recent GCC (which needs it for Objective C), it
includes its own Boehm GC source, but you would need to replace that
with a later version.

A problem I do see with the Browser is that selecting the "Save as
Postscript (eps)" item from the Graph drop-down menu doesn't work.  I
get a Tcl error message and no eps file.

However, the Browser is distraction.  Good luck with it, good
exercise, great if you can fix both problems, but there is more...
There are many things unfinished.

An external Fortran class providing interfaces to all the subroutines
of the BLAS was talked about years ago, there is an example of such
use in Doc/programmer-manual.ps, but it didn't eventuate.  That is
something you could deliver.

David Simons offered to an early GNU Sather maintainer some classes
interfacing to the PostgreSQL DBMS.  Here is an SQL query in a Sather
string variable to balance the string constant above, taken from
another program:

       cmd::="set search_path to lab,\"$user\",public; "
            "select i.matter, t.name as court, action.number, "
            "action.year, postal.district, p.surname, current_date as date "
            "from recentjobs r "
            "natural join alloc c "
            "join person p on p.k = c.\"to\" "
            "join firm f on f.k = p.firm "
            "join address a on f.address = a.key "
            "join postal on a.postal = postal.k "
            "join instance i using (agent, instance, ref) "
            "left outer join action on i.action = action.k "
            "left outer join court t on i.court = t.k "
            "where r.jobname = '"+mainarg+"'";

That contribution by Simons was in my opinion very important because
it turns Sather into an easy-to-use and high-performance business tool
(assisted by INTIs and RATs which ensure exact, unrounded, dollar
amounts).  But again the FSF maintainer of the time wasn't interested.

In Emacs there is a texinfo file from the pre-GNU era, for the Sather
1.0 specification, but no texinfo has been added during the GNU era.
This suggests another thing that needs to be done, perhaps starting
with a ps2ascii of some files in Doc.

FMT (formatted output class) seems weak, likely to segfault,
specially with FLTDs.

There are a number of errors, faulty parameter sequences, in
Library/Base/fltd.config, as well as (did I mention?) incorrect casts
in Library/Base/int.config.


Installing sather-1.2.3 I see this GC Warning:

"Creating C for installation compiler...

Boot/sacomp     -verbose -O_fast -O_no_move_while -O_no_hoist_const
-O_no_cse -only_reachable Compiler/sacomp.module -o Bin/sacomp -only_C
Initializing... (0 seconds)
Parsing non-library files... (1 seconds)
Finding types and parsing library... GC Warning: Repeated allocation
of very large block (appr. size 20480):
        May lead to memory leak and poor performance."

That is with libgc.so.1 compiled from gc6.8.

A problem that I think should not be encountered.  Somehow, repeated
allocation of a very large block needs to be justified and documented,
or else prevented, which may involve the maintainer getting into the
GC, perhaps with the help of others such as Hans Boehm.  Which makes
it a good idea to have a particular version of the GC included in the
Sather source, pinned down, under the control of the maintainer, as it
was in ICSI Sather-1.2b and its predecessors.

The linux "platform" (System/Platforms/linux) seems to do nothing but
supply the path and name of a particular dynamically linked libgc.so.
That's fine, but not a reason for dropping the GC source from the
Sather source tree, where it might/should be needed if someone
compiles Sather for the unix "platform".  I normally do that, even on
Linux.  That linux "platform" was there in early versions and
coexisted with the GC source at System/Common/GC.  There was no reason
to drop the latter.

In Doc/Changes under the heading "Version 1.2 Beta 2/24/98":

"This is a summary of the Sather 1.2 Beta (relative to Sather 1.1).
There were no language changes. Most work was concerned with the
parallel runtime which has been completely reimplemented."

The difference between 1.1 and 1.2b was essentially pSather (and, just
incidentally, some library classes added by Ben Gomes).  Yet pSather
is not working, I suspect.  GNU Sather 1.2.3 has the "platforms"
linux/lwp,linux_at,linux_at/smp, apparently a FSF achievement, perhaps
by Norbert Nemec.  I'm not using them and don't want to seem even
dumber, I try to hide it, but you might check out those pSather
platforms.  Because if they're not easy to install and use GNU Sather
is not an advance on Sather 1.1.

There are some superficial comments at Doc/pSather-notes but someone
needs to scatch below the surface.

Here are three very similar programs.  They don't do anything.  They
are extracts from an early draft program by Ben Gomes that reads its
options and validates the arguments of those options.  The idea is not
that this fragment will do anything useful.  The test is to compile
it.

-------------------------------------------------------------------
class MAIN is
   attr options: FMAP{STR,$OB};
   attr test1_failed,test2_failed: BOOL;

   add_option(opt_name: STR, opt_func: $OB) is
      options := options.insert(opt_name,opt_func);
   end;

   main is
      add_option("test1",#ROUT(test1(_)));
      add_option("test2",#ROUT(test2(_)));;
      add_option("test3",#ROUT(test3(_)));
   end;

  test1(str: STR) is
      #OUT+str+"\n";
      test1_failed := false;
   end;

  test2(str: STR) is
      #OUT+str+"\n";
      test2_failed := false;
   end;
   shared test3: BOOL := true;

end;
-------------------------------------------------------------------
------------------------> Sather 1.1 source file <-----------------------
class MAIN is
   attr options: FMAP{STR,$OB};
   attr test1_failed,test2_failed: BOOL;

   add_option(opt_name: STR, opt_func: $OB) is
      options := options.insert(opt_name,opt_func);
   end;

   main is
      add_option("test1",bind(test1(_)));
      add_option("test2",bind(test2(_)));;
      add_option("test3",bind(test3(_)));
   end;

  test1(str: STR) is
      #OUT+str+"\n";
      test1_failed := false;
   end;

  test2(str: STR) is
      #OUT+str+"\n";
      test2_failed := false;
   end;

-------------------------------------------------------------------
--------------------------> Sather 1.1 source file <-------------------------
class MAIN is

   attr options: FMAP{STR,$OB};

   attr test1_failed,test2_failed: BOOL;

   add_option(opt_name: STR, opt_func: $OB) is
      options := options.insert(opt_name,opt_func);
   end;

   main is
      x01 ::= bind(test1(_));
      add_option("test1",x01);
      x02 ::= bind(test2(_));
      add_option("test2",x02);
      x03 ::= bind(test3(_));
      add_option("test3",x03);
   end;

  test1(str: STR) is
      #OUT+str+"\n";
      test1_failed := false;
   end;

  test2(str: STR) is
      #OUT+str+"\n";
      test2_failed := false;
   end;

   shared test3: BOOL := true;

end;
-------------------------------------------------------------------

The first of those programs will compile and run without complaint
using the Sather-1.0.8 compiler.  The second, the source code, is the
result of automatic conversion to the syntax of later versions, the
Sather 1.1 language.  To convert with GNU Sather 1.2.3 you would use
the command

sacomp -only_parse -convert firstprog.sa

The only thing that was changed in the conversion was that "#ROUT" was
replaced with "bind".  And a comment was added at the top.

But when you try to compile it with GNU Sather the attempt will fail.
The following messages are issued:

n108.sa:11:24: No match for the internal call
MAIN::add_option(STR,bound-create-expression)
        Suggest:MAIN::add_option(STR,$OB)
n108.sa:12:24: No match for the internal call
MAIN::add_option(STR,bound-create-expression)
        Suggest:MAIN::add_option(STR,$OB)
n108.sa:13:24: No match for the internal call
MAIN::add_option(STR,bound-create-expression)
        Suggest:MAIN::add_option(STR,$OB)

The third program is the second after modification to obtain a
successful compile.  Extra lines have been added with a bunch of
otherwise redundant variables.  Those variables rather than the bind
expressions become the add_option arguments.  The problem is not a
failure of type inference.  There is type inference of the closures
when they use a certain type inference idiom, the simplest form of
assignment.  You don't have to specify the types, but it takes two
steps, it is lower-level and clumsy.  It's not that Sather 1.2b and
subsequent Gnu Sather has regressed from Sather 1.0.8, it's just that
it hasn't been finished yet, it's a beta, and fixing that clumsiness
is one of the things not yet done.

Benchmarks.  "Compiled Sather code runs within 10 percent of the
performance of handwritten C code on the MIPS machine and is
essentially as fast as handwritten C code on the Sparc architectures.
On a series of benchmark tests (towers of Hanoi, 8 queens, and the
like) Sather performed slightly better than C++ and several times
better than Eiffel." (Omohundro, 1993)

Are those claims valid today?  Then, the compiler was about 30,000
lines of Sather code.  Now the .sa files in the Compiler tree total
about 43,300 lines including comments.  In a different world, in which
something called the pee-see has achieved the processing power of a
real computer, of a Unix workstation, is no longer called a
microcomputer, and MIPS is gone and the Sun has finally set and
whatever remains of Solaris has a two-button mouse, and there is a
later version of Sather, not to mention mind-boggling new stuff in gcc
and gmp, is the current version as efficient, does it generate code
that runs as fast relative to C, C++ etc?

If not, a catch-up in compiler optimization becomes another project
for the maintainer.

Just playing with some benchmarks now (Doug Bagley's), I notice that
in a small number of them, Sather is faster than the C++ program
compiled with g++; in most it is faster than the Java program compiled
with gcj; in one it is faster than the C code; but it does relatively
poorly handling large arrays, an area where it once starred.  In
matrix handling, multiplying 30*30 square matices, it seems something
like half the speed of gcj code.  That is very bad and needs
attention.

Please make useful improvements, not just changes.  There has been
some silly stuff inflicted on GNU Sather, idle fiddles, coupled with
the kind of neglect that left Tcl_GetResult in c_interface.sa, and the
bugs I've been mentioning, which are all very old.

In view of all the loose ends mentioned above it is a pity, no credit
to the FSF, that the "b" of "1.2b" was dropped in later GNU versions
of Sather, perhaps because the passage of time had caused it to fall
out of beta.  You might think about renaming the present version
sather-1.2b3, and back versions correspondingly.  Then, you may be the
one who will release the real GNU sather-1.2.

Oh, and get rid of the News file.  Anyone who looks at it critically
will be unimpressed and may want to distance themselves from Sather.
Preferably, change the names back to the ones chosen by the language's
inventor.

There is a lot of documentation on the ICSI site.

> (I can dedicate about 4 to 6 hours of work per week on this... So
> I'll need help) Working on Sather could be a very enriching
> experience.  Does anyone care to join???

Count the above as my whole contribution.  From time to time I work on
a parallel fork from the pre-GPL Sather 1.2b, currently at 1.2b8, not
yet publicly released.  Anything I do will be under the terms of the
Sather Library General Public License.  But at 4-6 hours per week you
should race ahead.  I've told you what I know or guess.  Go for it.

Regards,
Mike

-- 
Michael Talbot-Wilson