Re: archiver questions:

[Marcus G. Daniels]

>>>>> "PJ" == Paul E Johnson <address@hidden> writes:

PJ> Since the term "type without hierarchy" is not known to me, let me
PJ> see if I understand this. hdf5 archives can work for objects that
PJ> have ints, floats, doubles, etc, but no lists, C-arrays, Swarm
PJ> arrays, or such.  Correct?

HDF5 archives can load and save all of those.  However, there are two
ways to store objects, deep and shallow, and shallow serialization has
some constraints.  Specifically, shallow serialization for HDF5 can't 
have objects or arrays inside of it.  Anything that makes sense
to put in a simple table makes sense with shallow serialization.

PJ> 2. How can I select which IVARS get written by the
PJ> HDF5Archiver--i.e., exclude the compound types?

MD>  To do that, you'd need to implement a different -hdf5OutShallow:
MD> method for the subclass.

PJ> OK, I'm looking in DefClass.m and I see that hdf5OutShallow: is
PJ> not implemented, presumably because of this hierarchy type problem
PJ> discussed in point 1.  

It's not implemented because saving a class by itself isn't
useful in HDF5.  The type of an object saved to HDF5 is evident
and precise, whereas with Lisp it is not.  Thus, there is more support
in DefClass for saving class structure for Lisp.

DefObject.m's -hdf5OutShallow: is a better place to look.

PJ> lispOutShallow method uses stuff from
PJ> OutputStream.  I'm looking for an OutputStream equivalent for
PJ> hdf5, some way to tell it what variables to save.  Am I just
PJ> supposed to get make direct use of the hdf5 library at that point.

There is a wrapper class called HDF5 that can be used to write
customized serialization methods.  It's not documented, but you
can probably get an idea how it works from the Swarm source code
(defobj, collections, space).

PJ> Suppose I want to save the time and two ivar floats, fx and fy.
PJ> What do I put in the guts of this in my object:

It shouldn't be necessary to write special serialization method
for this so long as you aggregate the things-you-want-to-save into chunks
(classes).

PJ> So when I load this into R, it will be as-if I had used R's "read"
PJ> function to grab an ascii dataset with names in the first row.

If you use shallow serialization, yes.  With deep serialization, you'll
get a complicated hierarchy of tagged lists that needs to be
converted back into table.  Such a conversion could be coded in R
to pull out relevant information from a larger body of data.
 
MD> Alternatively, if you have a class that is sparse, i.e. you don't
MD> care about most of the instance variables, then you can point the
MD> archiver at the instance and save the ivars that you do care about
MD> providing archiver keys for each one.

PJ> I do not understand "providing the archiver keys for each one".

You can save information selectively by choosing what arguments you
give to -put{Shallow,Deep}:object:.  The string key is the first
argument and the object associated with the string is the second.
Thus, if you had a large class with many ivars, but only wanted to
save a few, you could use the instance variable name as the key (first
argument) and the instance variable value as the object (second object).

In certain circumstances, doing it this way may be easier than writing
custom serialization methods, since writing serialization methods
means learning about the Input/Output streams in the case of Lisp and
the HDF5/HDF5CompoundType interfaces in the case of HDF5.  I can
explain how to do this, but I suspect in your case you don't really
need to do this (or for that matter resort the approach in the
paragraph above).

PJ> What you wrote makes me think I should create a single archiver
PJ> object in some high level place like main.m, and make it available
PJ> to several different objects that need to save values.

Yes, and in most cases, the {lisp,hdf5}AppArchiver (global) archiver
is what you want.  For intensive output logging it's arguably cleaner
if the output goes to an unrelated file, so that it can be deleted
independently of (perhaps) more-valuable configuration data.

                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.