A thought on portability...

[Barry McMullin]

(I've recently been blithely throwing out ideas about
application portability in the Swarm room, and Roger
challenged me to actually make some kind of serious
proposal - i.e. he called my bluff!)


Background:
-----------

When the C language was originally invented, it
followed in a long tradition of previous "high level"
languages in making the underlying representation of
the native data types (int, unsigned, float etc.)
deliberately implementation dependent.  This was (I
think) naively intended as an *aid* to portability.
The idea was that you could write your code using this
"generic" data types, secure in the knowledge that the
implementation (i.e. the compiler) would then choose
the "best" representation for this data type on the
particular target platform.

Those of us who have emerged (bloodied but unbowed!)
from experiences of actually porting the "same"
application to platforms where the underlying
representations turned out to be different, know that
this can be a mixed blessing to say the least.

A specific concern for scientific modelling is that,
with this approach to portability, while we may find
that our models in some sense automatically "exploit"
the "better" capability of different platforms, we also
find that the numerical results are *different* - when
we might really prefer that they be *exactly* the same
(but are arrived at faster, or whatever).

There is a relatively simple methodology for writing
applications (in C even!) such that numerical results
are "guaranteed" to be prortable across different
platforms, *despite* variations in the mapping between
native data types and underlying representation.  I
first came across this in a reference book on C
portability that I read about 10 years ago.  I can't
remember the title anymore - sorry (but for some
strange reason I do remember that the word "rabbit" was
in there somewhere!  If anyone else can identify it for
me, I'd appreciate it).

The basic idea is: don't use the native numeric data 
types!

Instead, we use the typedef mechanism to define
"portable", fixed representation, data types.

For example:

  typedef unsigned short   unsigned32;
  typedef          short   int32;     
  typedef double           ieee64;

Of course, these typedefs are put in a header
(PortableDataTypes.h or whatever) which is actually
just a sym link to a platform specific header
(SolarisPortableDataTypes.h or whatever) which maps the
portable data types onto the "appropriate" native data
types for the particular platform.  Making this sym
link would be part of configuring the app for the
platform.

The implied (and hopefully explicitly documented)
promise is that "unsigned32" will be an unsigned, 32
bit, binary representation on *all* platforms; "ieee64"
will be the IEEE standard, 64 bit, floating point
representation, on *all* platforms. etc.

In our app proper, we then eschew all use of the native
numerical data types, in favour of the portable, fixed
representation, ones - et voila, our apps compile and
yield exactly the same numerical results even across
platforms where the mapping onto the native data types
vary. (That at least is the theory - and I *have* seen
it work successfully on at least some apps...)

There *is* a potential problem in extending this to
referring to these formats with printf style formatting
strings, and I know of no straightforward, generic,
solution to that.  But many, if not all, cases of that
in Swarm apps might be handled via the "InFile" and
"OutFile" objects, assuming methods are added to those
to handle these portable numeric data types (the
*implementation* of those objects might then have to be
made platform specific, but at least this would be
hidden from the application level).

Of course, none of this has been much of an issue in
Swarm to date, because *almost* all users have been
using gcc on 32-bit targets, where the underlying
representation of the native numeric data types are all
been the same. BUT: it's liable to become a problem as
we move onto 64+ bit platforms.

My *proposal* is this:

1: Swarm should provide support for using portable data
types, by defining a standard set of names for such
types, and providing headers with typedefs for some set
of currently supported platforms,

AND

2: The existing Swarm libs should be reviewed, and,
where appropriate, altered, to use the portable numeric
data types instead of the native data types.  The *big*
example of this is random, but there may be other more
isolated examples (InFile and OutFile come to mind).


==================

There are a few significant potential drawbacks to the
proposal:

1: Not everyone may agree with this portability
philosophy - i.e. that numerical results should be
preserved exactly.

2: While I doubt that this change would break existing
apps on 32 bit platforms, it would require all apps to
be revised if they wanted to exploit this kind of
portability to new (non 32-bit) platforms.  And, of
course, I might even be wrong about it not breaking the
existing app/platforms...

3: It's work, and resources for core swarm development
are very limited.  Does it deserve to be regarded as
any kind of a priority?

=================



Well, there it is - comments and discussion welcome!


Regards,

Barry.