[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Orekit Developers] Is serialization of propagators really useful



Hi all,

MAISONOBE Luc wrote:
There are two reasons for the current behaviour for FactoryManagedFrames. The first reason was that we really need to have a single root hierarchy, so when deserializing a frame we have to plug it in the existing frame, we cannot let it create an independent tree by deserializing all its ancestors. The second reason was that we wanted to avoid creating too many copies of similar frames. These instances are not singleton per se (and don't need to) but they have some characteristics of singleton, only for memory consumption purposes. However, since now the data are hold by TimeStampedCache which is not serializable and the same cache can be shared by many instances (even in different thread, as it was created specifically for this purpose), the second reason about memory consumption problems could be ignored. So you are right, it is worth looking at what we want to save.
I think the single root hierarchy is a good design. The issue with serialization is that it provides "hidden" constructors that are often overlooked. I've attached a small program that creates a second root frame using serialization. Interestingly, it works even with a SecurityManager in place. I got the idea from Joshua Bloch's chapter on Serialization in Effective Java, which explains this and other problems with serialization. I think the proxy objects strategy that you suggested would fix this problem as long as readObject() throws an exception. The proxy strategy might make it harder to serialize subclasses because they will repeat their parents serialized form in addition to their new information.

Thomas Neidhart wrote:
Regarding the serialization of immutable objects: I think Evan made a good point, that the context in which an object is created (e.g. which ephemerides loaded) and on which it also depends somehow must be known when deserializing it again. Looking at the problem from a pure technical point of view, serializing only the bits and bytes the object is composed of may be sufficient, but imho it is also very important to know exactly what the data is all about, so you can use it correctly for later calculations or analysis.

We can serialize a CelestialBody, or an Orbit, but how can we ensure, that the same environment at its creation time is present at the time of deserialization? And more importantly, how can we indicate that there is a mis-alignment to prevent improper use of data results which are very difficult to track down?

Maybe something like this should not be the goal of a library like orekit, but then I would remove serialization support completely to not give the users a wrong impression on the use of it (And I know that I now sound very much like Gilles ;-). Imho, shifting away from serializable algorithms is a good idea. I do not think there exists a use-case where distributed computing can / shall be achieved by transmitting both function and data.
Perhaps Orekit could acheive the same effect as Java Serialization by adding support for parsing and emitting standardized file formats. For example, an Orbit could be persisted in a CCSDS Orbit Parameter Message. This approach has the advantages that it is release independent, human readable, developers can refer to a standard to understand the context of the data, and object creation always happens through a constructor. The downside is, of course, the extra effort to write parsers and emitters, but this may be comparable to the effort required to implement Serializeable correctly. Orekit already has some support for this with TLEs(read+write) and SP3(read) files.

Regards,
Evan

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.lang.reflect.Field;
import java.util.Arrays;

import org.orekit.errors.OrekitException;
import org.orekit.frames.Frame;
import org.orekit.frames.FramesFactory;
import org.orekit.frames.Transform;
import org.orekit.frames.TransformProvider;
import org.orekit.time.AbsoluteDate;

/**
 * Use serialization to create multiple root frames. Violates Frame's contract
 * and can be done with a security manager in place.
 * 
 * @author Evan Ward
 * 
 */
public class Crack {
    /**
     * "Steals" a reference to the root frame before the root frame's
     * readResolve method is called.
     * 
     * @author Evan Ward
     * 
     */
    public static class Stealer implements TransformProvider {
        /**
         * uid
         */
        private static final long serialVersionUID = -616481646237679422L;
        /**
         * a clone of the root frame
         */
        public static Frame imposter = null;
        /**
         * the root frame
         */
        public Frame frame;

        @Override
        public Transform getTransform(AbsoluteDate date) throws OrekitException {
            return Transform.IDENTITY;
        }

        /**
         * keep a reference to the partially de-serialized root frame.
         * 
         * @return this
         */
        public Object readResolve() {
            // frame had not been through its own readResolve at this point.
            imposter = frame;
            return this;
        }
    }

    public static void main(String[] args) throws IOException,
            ClassNotFoundException, OrekitException {
        final Frame root = FramesFactory.getGCRF();
        // serialized data
        byte[] data;

        // create corrupted data using privileged reflection
        // data = createCorruptedSerializedForm(root);

        // use pre-generated corrupt data. Could be from a file or network.
        data = serializedForm;

        // de-serialization
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(
                data));
        Frame rootSer = (Frame) ois.readObject();
        // read in root is the same due to readResolve
        System.out.format("%s ?= %s : %b%n", root, rootSer, root == rootSer);
        // Impostor root is different
        System.out.format("%s ?= %s : %b, imposter's parent: %s%n", root,
                Stealer.imposter, root == Stealer.imposter,
                Stealer.imposter.getParent());
        // get transform between two root frames? => NPE
        Stealer.imposter.getTransformTo(root, AbsoluteDate.J2000_EPOCH);
    }

    /**
     * @param root
     * @return corrupted serialized form.
     * @throws IOException
     */
    public static byte[] createCorruptedSerializedForm(final Frame root)
            throws IOException {
        System.setSecurityManager(null);
        Stealer stealer = new Stealer();
        stealer.frame = root;
        setTranformProvider(root, stealer);
        final ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(baos);
        oos.writeObject(root);
        System.out.println(Arrays.toString(baos.toByteArray()));
        byte[] data = baos.toByteArray();
        return data;
    }

    /**
     * Set Frame#transformProvider This method uses privileged reflection, but
     * the same could be accomplished by editing the serialized form.
     * 
     * @param root
     * @param stealer
     */
    private static void setTranformProvider(Frame root, Stealer stealer) {
        try {
            // System.out.println(Arrays.asList(root.getClass().getSuperclass().getDeclaredFields()));
            Field tfp = root.getClass().getSuperclass()
                    .getDeclaredField("transformProvider");
            tfp.setAccessible(true);
            tfp.set(root, stealer);
        } catch (NoSuchFieldException e) {
            throw new RuntimeException(e);
        } catch (SecurityException e) {
            throw new RuntimeException(e);
        } catch (IllegalArgumentException e) {
            throw new RuntimeException(e);
        } catch (IllegalAccessException e) {
            throw new RuntimeException(e);
        }
    }

    /**
     * Corrupted serialized form of the root Frame.
     */
    public static final byte[] serializedForm = { -84, -19, 0, 5, 115, 114, 0,
            40, 111, 114, 103, 46, 111, 114, 101, 107, 105, 116, 46, 102, 114,
            97, 109, 101, 115, 46, 70, 114, 97, 109, 101, 36, 76, 97, 122, 121,
            82, 111, 111, 116, 72, 111, 108, 100, 101, 114, 36, 49, -37, 41,
            -86, 21, 106, -73, -14, 121, 2, 0, 0, 120, 114, 0, 23, 111, 114,
            103, 46, 111, 114, 101, 107, 105, 116, 46, 102, 114, 97, 109, 101,
            115, 46, 70, 114, 97, 109, 101, -97, 29, -4, 52, -58, 57, -87,
            -103, 2, 0, 4, 90, 0, 14, 112, 115, 101, 117, 100, 111, 73, 110,
            101, 114, 116, 105, 97, 108, 76, 0, 4, 110, 97, 109, 101, 116, 0,
            18, 76, 106, 97, 118, 97, 47, 108, 97, 110, 103, 47, 83, 116, 114,
            105, 110, 103, 59, 76, 0, 6, 112, 97, 114, 101, 110, 116, 116, 0,
            25, 76, 111, 114, 103, 47, 111, 114, 101, 107, 105, 116, 47, 102,
            114, 97, 109, 101, 115, 47, 70, 114, 97, 109, 101, 59, 76, 0, 17,
            116, 114, 97, 110, 115, 102, 111, 114, 109, 80, 114, 111, 118, 105,
            100, 101, 114, 116, 0, 37, 76, 111, 114, 103, 47, 111, 114, 101,
            107, 105, 116, 47, 102, 114, 97, 109, 101, 115, 47, 84, 114, 97,
            110, 115, 102, 111, 114, 109, 80, 114, 111, 118, 105, 100, 101,
            114, 59, 120, 112, 1, 116, 0, 4, 71, 67, 82, 70, 112, 115, 114, 0,
            13, 67, 114, 97, 99, 107, 36, 83, 116, 101, 97, 108, 101, 114, -9,
            113, -47, 53, -127, 116, -128, -62, 2, 0, 1, 76, 0, 5, 102, 114,
            97, 109, 101, 113, 0, 126, 0, 3, 120, 112, 113, 0, 126, 0, 5 };;

}