Re: [Evms] EVMS Conference call (04/10/01) minutes

[Steve Dobbelstein]

Andrew Clausen wrote:

> Please keep address@hidden cc'd.  There are a few ppl
> there who will be interested in this stuff.

Will do.

> > First, thanks for starting to tackle the user space
> > installation/configuration issues.  You have brought up
> > several and we know there will be many more.  :)
>
> ;-)
>
> > As a point of clarification, we are not envisioning keeping
> > a list of commands to do.  Rather, the Engine just updates
> > the in-memory structures.  When the commit command comes,
> > each plug-in (disk manager, segment manager, feature, etc)
> > figures out what it needs to do to configure the system to
> > the new image represented in memory.
>
> I don't think this can work for partitioning, though it
> probably can for LVM style stuff.  (As I said before,
> automatic partitioning is NP complete, and we believe
> it is IN PRACTICE too hard)

I may be missing your point and seeing the world from a different
angle.  What we had in mind was that the user plays around with
the in-memory structures to get the configuration he/she wants.
On the commit, the various plug-ins know what they need to do to
put their stuff on disk.  Segment managers (a.k.a. partition
managers), rather than trying to figure out deltas to the
partitioning scheme would just blast the new partition data to
the disk.  Thus, the setting of new partitions is easy.  Or am I
missing your point?

> > Naturally, the Engine needs to coordinate the order of how the
> > commit changes are done.  Here is our current thinking of how
> > to order the various components of commit should be executed.
> >
> > Notify the kernel code of any deleted volumes.
> > Run the "kill list", a list of sectors that should be wiped out.
> >    (Plug-ins will notify the Engine of sectors they need
> >     wiped out, for example, feature metadata that is being
> >     removed.)
> > Call the File System Interface Module for any volumes that are
> >    to be shrunk.
> > Commit segments (partitions)
> > Commit containers (e.g., volume groups)
> > Commit features
> > Call the kernel code to rediscover the disk configuration
> > Call the File System Interface Module for any volumes that are
> >    to be expanded.
>
> This algorithm cannot be adapted for partitioning.  Think
> about swapping the on-disk order of two partitions, for example.

In order to keep the algorithm simple I had left out the operations
needed for moving data from one partition to another and for sliding
a partition to a new location on the disk.  Those operations are
more involved.  We want to implement them (perhaps in a later version
if they are quite complex) and have been trying to figure out all the
details necessary to make them work.  Any input, comments, gotchas on
the topic are appreciated.

> > > * handling references is a PITA.  If you delete two LV's that
> > > where linked (linear raid), then how do you resurrect these
> > > references?
> >
> > Hmmm.  Good question.
>
> I think the answer here is:
> * the UUID in the two LVs is saved in the inverse operation
> of the two "delete LVs"
> * the UUIDs of the two volumes that are being used by the
> linking are remembered in the inverse operation of the
> "remove linear RAID"
>
> > > * the operations must be completely non-interactive (eg,
> > > no questions like "do you want to convert from fat16 -> fat32"),
> > > because you need to apply each operation twice (first
> > > "in-memory", then on disk)
> > >          This probably isn't a disaster, because stuff
> > > like this can be passed as parameters.  (And if you NEED
> > > to make a choice, then you can fail, and ask them to
> > > provide better parameters)
> > >
> >
> > Option 1 is the one I had envisioned.  An undo would be an inverse
> > operation of what was done.  More specifically, the Engine API
> > would not be the one to provide the undo.  It would be the user
> > interface that would figure out what Engine API(s) should be called
> > to undo the previous action.
>
> My big concern with this is the information that needs to be
> stored, and restored.  The "Engine API" knows about the information,
> not the front end.  This is my main objection.
>
> I think it should be the Engine's responsibility to construct
> a reverse operation, if this is the route we take.

If we want to keep the Engine out of the undo business, then we could
make sure the Engine provides the information that the front end will
need to do an undo.  (Yuck.  Not my favorite option.  I'm just kicking
around ideas.)

As we look further into the details of what is needed for an undo we
may find that the Engine will have to take a more active roll.  Your
comments are making me think we will have to head that way.

> > Of course, undo is more complex than simply executing an inverse
> > operation.  For example, what if the user interface ends up calling
> > a special purpose ioctl to one of the plug-ins?  How do you undo
> > that?
>
> Hang on... isn't "undo" before "commit".  Isn't calling the
> ioctl part of "commit"?

The plug-ins may have specific calls for configuration that only update
the in-memory structures and do not write to disk.  The front end and
the Engine may not know how to undo those operations (unless the plug-in
provides and "undo" for the specific operation).

BTW, I used the term "ioctl" as an easy reference for "a function call
that is out of the normal flow of calls available in the Engine".  In
the design, the calls are not really ioctls but are calls to the Engine's
DirectPluginCommunication() function.  The Engine just passes the
information given in DirectPluginCommunication() on to the plug-in.
The Engine does not know what the call will do.  It may update in-memory
structures or not.  DirectPluginCommunication() is meant to provide a
way for plug-ins and front ends to communicate about things that were
not provided in the current version of the Engine's architecture.  When
processing a DirectPluginCommunication() the plug-in should not write
anything to disk.  That should be saved for the commit.


> > One idea that might help implement the undo function is the concept
> > of "snapshots".  I'm not talking about LVM type snapshots.  (Maybe
> > I need a better term.)  The idea is that the Engine provides the
> > ability to take a snapshot of the in-memory data structures,
> > essentially saving the current state.  The user could then roll back
> > to a current level of snapshot.  The plug-ins would have to support
> > snapshotting their own internal data.  The user interface could
> > provide the snapshotting function to the user, or it could
> > incorporate it into its own "undo" functionality -- it would take a
> > snapshot when it doesn't know how to execute an inverse function for
> > the undo.
>
> This is quite similar to checkpoints.

I like the term "checkpoints" better than "snapshots".

> > Doing intermediate commits sounds hard and risky to me.
> > It sounds hard because, as you mentioned, you have to save
> > the various check points and how to undo them.
>
> I don't think saving the actual checkpoints is hard.  You
> can basically do it by duplicating the existing state,
> and saving it on a list.  (I have a proof-of-concept for
> this in 1.5.x, which will be scrapped for now, due to the
> reference issues)

Yes, doing checkpoints is not hard.  However, it can get memory
expensive when you are dealing with clusters of 1024 disks,
for example.  The amount of memory you have will limit how many
checkpoints you can keep.

> The hard bit is undoing/keeping "references".
>
> >  It also sounds
> > hard because you have to determine the various checkpoints where
> > you can guarantee that what you commit will leave the system in
> > a usable state.
>
> This is trivial.  2 obvious options:
> (1) commit after each operation
> (2) only parrelelize operations that touch non-overlapping parts
> of the disk.
>
> Further optimization is more tricky, but probably
> unnessecary... unless you have millions of operations from
> "hill climbing"... (which we might need to resort to!)
>
> > Intermediate commits also end up doing unnecessary work if
> > the user is just playing "what if?" with the configuration tool.
>
> Nonono.  The user shouldn't commit unless he/she WANTS the disk
> that way.
>
> > The user may want to see how a new configuration might look and
> > then quit without making changes.  If intermediate commits are
> > made they all have to be undone.
>
> No, the user should have 100% control over when commits occur.
> The above design allows this.  (Why did you think otherwise?
> Maybe I explained something badly)
>
> > Intermediate commits sound risky to me because any time you make
> > changes to the disk you run the risk of hosing thing up.
> >
> > <snip>
>
> I don't see why - but I think you misunderstood what I was
> saying.  Clearer now?

Oops.  I misunderstood what you were saying.  I thought you were
talking about the Engine doing intermediate commits under the covers
as the user was manipulating the front end.  I agree that the user
should be the one to say when changes should be committed to disk.

> Andrew Clausen
>

Steve Dobbelstein