[PATCH] Serialization #4

[Guilhem Lavaux]

Hi,

Here is a new part of the patch for serialization. It is mainly about 
restructuring the class lookup logic. It appeared that you need to build 
a class lookup table and a class hierarchy for a given input stream. 
Without that, if you write something with ObjectOutputStream and then 
read something else the previous code was using the already built class 
hierarchy to decode this new stream. The solution was obviously to build 
a full lookup table for each instance of ObjectInputStream. At the same 
time, ObjectStreamClass continue to manage the "real" class hierarchy 
(i.e. the one that exists in the VM at the running time).

This patch also contains an internal API change for 
ObjectStreamClass.setClass and some (extensive) addings to this method. 
setClass now also builds a consistent array of the accepted fields for 
the input stream:
* first we take all non transient, non static fields.

* then if there are the explicitly exported fields (using 
"serialPersistentFields" or by detecting the available fields)
* if the previous case was executed then the other "exportable" fields 
are implicitly imported but not exported .
These states will be used in a next patch.

getSerialPersistentFields has been simplified because exceptions should 
be caught in the calling method so they can be handled correctly by 
setClass.
getSerialPersistentFields will be updated to its true version in a next 
patch.

I'm sorry if this patch is too big but changing but this is mainly due 
to the change in the class lookup logic.

Cheers,
Guilhem.

P.S.: Modifications concerning ObjectOutputStream will follow after all 
modifications about ObjectInputStream.

ChangeLog entry:

2003-11-27  Guilhem Lavaux <address@hidden>

        * java/io/ObjectInputStream.java:
        (lookupClass): New method.
        (currentLoader): New method.
        (inputGetObjectStreamClasses): New method.
        (assignNewHandle): Documented.
        (currentClassLoader): Documented.

        * java/io/ObjectStreamClass.java:
        (setClass) Changed API. Better handling of the imported/exported fields.
        (getSerialPersistentFields) Make it throw previously caught exceptions 
so
        they can handled in setClass.

--- /home/guilhem/ext2/PROJECTS/classpath-public/java/io/ObjectInputStream.java 
2003-11-26 21:59:05.000000000 +0100
+++ java/io/ObjectInputStream.java      2003-11-27 21:25:06.000000000 +0100
@@ -99,10 +99,11 @@
     this.blockDataInput = new DataInputStream (this);
     this.realInputStream = new DataInputStream (in);
     this.nextOID = baseWireHandle;
-    this.objectLookupTable = new Hashtable ();
-    this.validators = new Vector ();
+    this.objectLookupTable = new Hashtable();
+    this.validators = new Vector();
+    this.classLookupTable = new Hashtable();
     setBlockDataMode (true);
-    readStreamHeader ();
+    readStreamHeader();
   }
 
 
@@ -203,7 +204,7 @@
              Class cl = resolveProxyClass(intfs);
              setBlockDataMode(oldmode);
              
-             ObjectStreamClass osc = ObjectStreamClass.lookup(cl);
+             ObjectStreamClass osc = lookupClass (cl);
              assignNewHandle (osc);
              
              if (!is_consumed)
@@ -332,7 +333,7 @@
              int handle = assignNewHandle (obj);
              this.currentObject = obj;
              ObjectStreamClass[] hierarchy =
-               ObjectStreamClass.getObjectStreamClasses (clazz);
+               inputGetObjectStreamClasses (clazz);
              
              for (int i=0; i < hierarchy.length; i++)
                {
@@ -455,8 +456,10 @@
          new ObjectStreamField (field_name, class_name);
       }
              
+    Class clazz = resolveClass (osc);
     boolean oldmode = setBlockDataMode (true);
-    osc.setClass (resolveClass (osc));
+    osc.setClass (clazz, lookupClass (clazz.getSuperclass()));
+    classLookupTable.put (clazz, osc);
     setBlockDataMode (oldmode);
              
     return osc;
@@ -550,16 +553,78 @@
   protected Class resolveClass (ObjectStreamClass osc)
     throws ClassNotFoundException, IOException
   {
+    return Class.forName (osc.getName(), true, currentLoader()); 
+  }
+  
+  private ClassLoader currentLoader ()
+  {
     SecurityManager sm = System.getSecurityManager ();
     if (sm == null)
       sm = new SecurityManager () {};
+ 
+    return currentClassLoader (sm);
+  }
 
-    ClassLoader cl = currentClassLoader (sm);
+  /**
+   * Lookup a class stored in the local hashtable. If it is not
+   * use the global lookup function in ObjectStreamClass to build
+   * the ObjectStreamClass. This method is requested according to
+   * the behaviour detected in the JDK by Kaffe's team.
+   *
+   * @param clazz Class to lookup in the hash table or for which
+   * we must build a descriptor. 
+   * @return A valid instance of ObjectStreamClass corresponding
+   * to the specified class.
+   */
+  private ObjectStreamClass lookupClass (Class clazz)
+  {
+    ObjectStreamClass oclazz;
+    
+    oclazz = (ObjectStreamClass) classLookupTable.get(clazz);
+    if (oclazz == null)
+      return ObjectStreamClass.lookup (clazz);
+    else
+      return oclazz;
+  }
+
+  /**
+   * Reconstruct class hierarchy the same way
+   * address@hidden 
java.io.ObjectStreamClass.getObjectStreamClasses(java.lang.Class)} does
+   * but using lookupClass instead of ObjectStreamClass.lookup. This
+   * dup is necessary localize the lookup table. Hopefully some future 
rewritings will
+   * be able to prevent this.
+   *
+   * @param clazz This is the class for which we want the hierarchy.
+   *
+   * @return An array of valid address@hidden java.io.ObjectStreamClass} 
instances which
+   * represent the class hierarchy for clazz.
+   */
+  private ObjectStreamClass[] inputGetObjectStreamClasses (Class clazz)
+  {
+    ObjectStreamClass osc = lookupClass (clazz);
+
+    ObjectStreamClass[] ret_val;
 
-    if (cl == null)
-      return Class.forName (osc.getName ());
+    if (osc == null)
+      return new ObjectStreamClass[0];
     else
-      return cl.loadClass (osc.getName ());
+    {
+      Vector oscs = new Vector ();
+
+      while (osc != null)
+      {
+       oscs.addElement (osc);
+       osc = osc.getSuper ();
+      }
+
+      int count = oscs.size ();
+      ObjectStreamClass[] sorted_oscs = new ObjectStreamClass[ count ];
+
+      for (int i = count - 1; i >= 0; i--)
+       sorted_oscs[ count - i - 1 ] = (ObjectStreamClass)oscs.elementAt (i);
+
+      return sorted_oscs;
+    }
   }
 
   /**
@@ -1148,7 +1213,12 @@
     throw new IOException ("Subclass of ObjectInputStream must implement 
readObjectOverride");
   }
 
-  // assigns the next availible handle to OBJ
+  /**
+   * Assigns the next available handle to <code>obj</code>.
+   *
+   * @param obj The object for which we want a new handle.
+   * @return A valid handle for the specified object.
+   */
   private int assignNewHandle (Object obj)
   {
     this.objectLookupTable.put (new Integer (this.nextOID),
@@ -1300,7 +1370,7 @@
   {
     ObjectStreamField[] stream_fields = stream_osc.fields;
     ObjectStreamField[] real_fields =
-      ObjectStreamClass.lookup (stream_osc.forClass ()).fields;
+      lookupClass (stream_osc.forClass ()).fields;
 
     boolean default_initialize, set_value;
     String field_name = null;
@@ -1493,14 +1563,20 @@
       }
   }
 
-  // this native method is used to get access to the protected method
-  // of the same name in SecurityManger
+  /**
+   * This native method is used to get access to the protected method
+   * of the same name in SecurityManger.
+   *
+   * @param sm SecurityManager instance which should be called.
+   * @return The current class loader in the calling stack.
+   */
   private static native ClassLoader currentClassLoader (SecurityManager sm);
 
   private static Field getField (Class klass, String name)
     throws java.lang.NoSuchFieldException
   {
     final Field f = klass.getDeclaredField(name);
+    
     AccessController.doPrivileged(new PrivilegedAction()
       {
        public Object run()
@@ -1744,6 +1820,7 @@
   private boolean isDeserializing;
   private boolean fieldsAlreadyRead;
   private Vector validators;
+  private Hashtable classLookupTable;
 
   private static boolean dump;
 
--- /home/guilhem/ext2/PROJECTS/classpath-public/java/io/ObjectStreamClass.java 
2003-11-26 21:58:23.000000000 +0100
+++ java/io/ObjectStreamClass.java      2003-11-26 21:46:43.000000000 +0100
@@ -291,7 +291,18 @@
     this.fields = fields;
   }
 
-  void setClass (Class cl) throws InvalidClassException
+  /**
+   * This method builds the internal description corresponding to a Java Class.
+   * As the constructor only assign a name to the current ObjectStreamClass 
instance,
+   * that method sets the serial UID, chose the fields which will be 
serialized,
+   * and compute the position of the fields in the serialized stream.
+   *
+   * @param cl The Java class which is used as a reference for building the 
descriptor.
+   * @param superClass The descriptor of the super class for this class 
descriptor.
+   * @throws InvalidClassException if an incompatibility between computed UID 
and
+   * already set UID is found.
+   */
+  void setClass (Class cl, ObjectStreamClass superClass) throws 
InvalidClassException
   {
     this.clazz = cl;
 
@@ -312,11 +323,89 @@
       }
 
     isProxyClass = clazz != null && Proxy.isProxyClass (clazz);
-    ObjectStreamClass osc = (ObjectStreamClass)classLookupTable.get (clazz);
-    if (osc == null)
-      classLookupTable.put (clazz, this);
-    superClass = lookupForClassObject (clazz.getSuperclass ());
+    this.superClass = superClass;
     calculateOffsets ();
+    
+    try
+      {
+       ObjectStreamField[] exportedFields = getSerialPersistentFields (clazz); 
 
+
+       if (exportedFields == null)
+         return;
+
+       ObjectStreamField[] newFieldList = new 
ObjectStreamField[exportedFields.length + fields.length];
+       int i, j, k;
+
+       /* We now check the import fields against the exported fields.
+        * There should not be contradiction (e.g. int x and String x)
+        * but extra virtual fields can be added to the class.
+        */
+
+       Arrays.sort(exportedFields);
+
+       i = 0; j = 0; k = 0;
+       while (i < fields.length && j < exportedFields.length)
+       {
+         int comp = fields[i].getName().compareTo 
(exportedFields[j].getName());
+
+         if (comp < 0)
+           {
+             newFieldList[k] = fields[i];
+             fields[i].setPersistent(false);
+             fields[i].setToSet(false);
+             i++;
+           }
+         else if (comp > 0)
+           {
+             /* field not found in imported fields. We add it
+              * in the list of supported fields.
+              */
+             newFieldList[k] = exportedFields[j];
+             newFieldList[k].setPersistent(true);
+             newFieldList[k].setToSet(false);
+             j++;
+           }
+         else
+           {
+             if (!fields[i].getType().equals (exportedFields[j].getType()))
+               throw new InvalidClassException (
+                           "serialPersistentFields must be compatible with" +
+                           " imported fields (about " + fields[i].getName() + 
")");
+             newFieldList[k] = fields[i];
+             fields[i].setPersistent(true);
+             i++;
+             j++;
+           }
+         k++;
+       }
+
+       if (i < fields.length)
+         for (;i<fields.length;i++,k++)
+         {
+           fields[i].setPersistent(false);
+           fields[i].setToSet(false);
+           newFieldList[k] = fields[i];
+         }
+       else
+         if (j < exportedFields.length)
+           for (;j<exportedFields.length;j++,k++)
+             {
+               exportedFields[j].setPersistent(true);
+               exportedFields[j].setToSet(false);
+               newFieldList[k] = exportedFields[j];
+             }
+       
+       fields = new ObjectStreamField[k];
+       System.arraycopy (newFieldList, 0, fields, 0, k);
+      }
+    catch (NoSuchFieldException ignore)
+      {
+       return;
+      }
+    catch (IllegalAccessException ignore)
+      {
+       return;
+      }
   }
 
   void setSuperclass (ObjectStreamClass osc)
@@ -435,7 +524,9 @@
       }
     }
     catch (NoSuchFieldException ignore)
-    {}
+      {}
+    catch (IllegalAccessException ignore)
+      {}
 
     int num_good_fields = 0;
     Field[] all_fields = cl.getDeclaredFields ();
@@ -612,25 +703,16 @@
 
   // Returns the value of CLAZZ's private static final field named
   // `serialPersistentFields'.
-  private ObjectStreamField[] getSerialPersistentFields (Class clazz)
+  private ObjectStreamField[] getSerialPersistentFields (Class clazz) 
+    throws NoSuchFieldException, IllegalAccessException
   {
     ObjectStreamField[] o = null;
-    try
-      {
-       // Use getDeclaredField rather than getField for the same reason
-       // as above in getDefinedSUID.
-       Field f = clazz.getDeclaredField ("serialPersistentFields");
-       f.setAccessible(true);
-       o = (ObjectStreamField[])f.get (null);
-      }
-    catch (java.lang.NoSuchFieldException e)
-      {
-      }
-    catch (java.lang.IllegalAccessException e)
-      {
-      }
 
-    return o;
+    // Use getDeclaredField rather than getField for the same reason
+    // as above in getDefinedSUID.
+    Field f = clazz.getDeclaredField ("serialPersistentFields");
+    f.setAccessible(true);
+    return (ObjectStreamField[])f.get (null);
   }
 
   public static final ObjectStreamField[] NO_FIELDS = {};