Re: [Cfengine-develop] my own branch of the tree

[Luke A. Kanies]

On Mon, 19 Jan 2004, Bas van der Vlies wrote:

> address@hidden wrote:
>
> Luke,
>
> can you not explain to our small list what you see as your vision for loops
> in cfengine and what you mean by "abstraction" etc with sufficient detail
> so that we can determine the best solution and move on constructively?
> We should do this soon, while I have time to listen to the discussion.
>
> I would also like input from the rest of the developers list, especially
> on how to implement the solution without damaging  backward compatibility
> or convergence.

Here are some examples of abstraction that is difficult or impossible in
cfengine.

Inetd:

I build my inetd file from scratch using cfengine.  As I'm sure you all
know, an inetd config file is composed of individual configurations for
many daemons.  Each daemon configuration is on its own line, and each line
has a fixed number of fields.  In perl, you would describe it a lot like
this (I'm using the field names from memory, so they're not correct):

'telnet' => {
        binary => '/usr/sbin/in.telnetd',
        arguments => '',
        user => 'root',
        protocol => 'tcp',
        wait => 'nowait',
        ...
}

Okay, so I want to use cfengine to selectively enable services on hosts,
something like the following:

control:
        server1::
                <enable telnet>
                <enable ftp>
        server2::
                <enable ftp>
                <enable rstatd>

That <enable telnet> needs to result in the configuration for telnet being
put into inetd.conf.  In other words, that short statement somehow needs
to refer to the entire configuration of telnet, along with which file it
goes into and what class to set so that inetd gets HUPed if something
changes.  This is abstraction:  I'm abstracting "enable telnet" into a
specific configuration, where it goes, and how to restart inetd if the
config changes.

If I wanted to do this with cfengine right now without stepping outside of
cfengine at all, it would be, um, messy.  It would probably look something
like this:

control:
        telnet_binary = ( "/usr/sbin/in.telnetd" )
        telnet_user = ( "root" )
        telnet_protocol = ( "tcp" )
        ...

editfiles:
        server1::
                { ${inetd_conf}
                        SetLine "telnet ${telnet_protocol} ..."
                        AppendIfNoLineMatching telnet
                        SetLine "ftp ${ftp_protocol} ..."
                        AppendIfNoLineMatching ftp
                        Define restart_inetd
                }
        server2::
                { ${inetd_conf}
                        SetLine "ftp ${ftp_protocol} ...
                        AppendIfNoLineMatching ftp
                        SetLine "rstatd ${rstatd_protocol} ..."
                        AppendIfNoLineMatching rstatd
                        Define restart_inetd
                }

Notice that I had to repeat a bunch of information each time:  I had to
repeat the format for the inetd config file, I had to repeat how to get
the data into the file, I had to repeat the definition of whether the data
was actually in there or not (i.e., whether _any_ ftp configuration is
acceptable, or only that exact configuration).  In addition, once I've
defined for server1 how to add ftp, I have to repeat that exact
configuration all over again.

Sure, I could have switched it around so that it was "server1|server2"::,
but that would have reduced abstraction in a different direction.

Notice also that I have about six variables for every single inetd
daemon, and setting them all is incredibly messy.

Now, compare that to how I am currently doing this at one of my client
sites:

I have an external data file that is essentially one big hash file,
containing every single inetd config possible across all of my platforms.
I have named each of these configs something symbolic to me but
essentially meaningless to the software (e.g., "telnet_sun", "telnet_hp").
I have a simple perl script which accepts a list of these names on the
command line, compiles the actual configurations from the data file, and
compares the result to the file on disk.  If the file on disk is out of
data, the script overwrites it and prints out "+restart_inetd", thus
notifying cfengine that inetd needs to be restarted.

Within cfengine, this is what I do:

control:
        server1::
                inetd_svcs = ( "${inetd_svcs} telnet_sun ftp_sun" )
        server2::
                inetd_svcs = ( "${inetd_svcs} ftp_sun rstatd_sun" )

This reduces the amount of repetition merely to the symbolic name I am
using for servers.  It's trivial to read and understand, and it works
very, very well.  I consider this to be completely within "the cfengine
way" because the process of writing the inetd.conf file is idempotent; the
perl script compares the in-memory file to the on-disk file, and only
writes it and sets the class if the files are different.

This is generally what I mean by "abstraction", and if you can find a
means of providing this level of functionality within cfengine without a)
stepping outside of cfengine with an external script, or b) generating
cfengine code, then you're much more capable than I.

Packages:

An even messier problem is the way that I'm installing packages using
cfengine.

I have created a packaging system entirely using cfengine and some
external code generation scripts.  I really, really like it because I can
create a cfengine configuration script for each package and know that
every package set up this way will always be set up correctly forever.

However, to do this, I am having to generate all of the code that actually
does any work.

I have a list of packages and revisions, e.g., perl 5.8.2, cfengine
2.1.0p1, cfengine 2.0.7, openssl 0.9.7b, etc.  I want to be able to
install multiple revisions of the same package, but I want one specific
version to be listed as the "current"; its files will be directly linked
to /usr/local (e.g., the current cfengine will be linked as
/usr/local/sbin/cfagent, but other versions will be
/usr/local/sbin/cfagent-<version>).

Cfengine needs to copy the files for the package down, link the files into
place (ignoring different directories to copy for each package; I seldom
want to copy info, but sometimes I have to copy share), and then verify
the package is running correctly.  I also need the ability to configure
different versions of a package differently.

I have absolutely no idea how you would do this without generating code.
Really.  No idea.

What I am doing is something that works but with which I'm not terribly
happy.

I do the following:

control:
        server1::
                pkgs = ( "${pkgs} cfengine--2.1.0p1 openssl--0.9.7c" )
        server2::
                pkgs = ( "${pkgs} wuftp-1.0" )

Then I have a module which accepts that list of packages and generates
cfengine code that does the copying and linking.  It then includes a
cfengine file at $CFINPUTS/packages/<package>.cf and/or
$CFINPUTS/packages/<package>-<revision>.cf to execute after the package is
copied.  The copies only happen once a day, but the links and
configuration happen every time cfengine runs.

I know that I could do the actual work within a cfengine method, but how
would I iterate across the list of packages?  In other words, how do I
abstract the list of packages I want to install from how they are actually
being installed?

This is what i mean by "abstraction".

I would be glad to give more information.  I am hoping to give a talk on
this packaging system at OSCON in Portland, OR this year, and I may submit
it as a LISA talk, also.

Luke

-- 
Westheimer's Discovery:
        A couple of months in the laboratory can frequently save a
        couple of hours in the library.