Re: (perl)script for cfengine analysis

[Thomas Glanzmann]

>  shellcommands:
> 
>         debian.Hr00.Min00:: 
>         "/usr/local/sbin/debian-update.sh 2>&1 | logger -p local0.notice"
>         timeout=600 # timeout=seconds
> 
> This way you can (again) leverage existing syslog parsing tools.

I see your point.

> If this doesn't work (though I fully expect it to) you'd have to collect
> the data somehow in a central spot via database insert, shared NFS
> mount, CGI post, etc and parse it from there. That kinda sucks - maybe
> somebody else has a better idea.

My idea of collection data is via eMail. And then write a script to evaluate the
data on a per day basis. We already have that script but is unusable for the
amount of data we get from cfengine. We used it to process our rdist ouput. But
rdist was only run once a day on a few hosts. cfengine is running once a hour on
128 hosts.

However our syslog evalutation needs anyway some cleanup. I let you know how I
solved the problem if I am done.

By the way nice 'outlook information in your header' :)

Greetings,
        Thomas
--
Thomas Glanzmann  ++49 (0) 9131 85-27574   Department of Computer Science III
Martensstrasse 3  D-91058 Erlangen Germany   University of Erlangen-Nuremberg
            http://www3.informatik.uni-erlangen.de/Research/UMLinux/
>From Stefan.wipf@gs.com Thu Dec 26 11:32:57 2002
Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13)
        id 18Ravg-0000x0-00
        for help-cfengine@gnu.org; Thu, 26 Dec 2002 11:32:16 -0500
Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13)
        id 18RauY-0000Cv-00
        for help-cfengine@gnu.org; Thu, 26 Dec 2002 11:31:16 -0500
Received: from inswsod02.gs.com ([207.17.37.11])
        by monty-python.gnu.org with esmtp (Exim 4.10.13)
        id 18Rat2-0007M7-00
        for help-cfengine@gnu.org; Thu, 26 Dec 2002 11:29:32 -0500
Received: from insdbod01.inz.gs.com (unknown [207.17.36.74])
        by inswsod02.gs.com (Postfix) with ESMTP
        id E55151C04B; Thu, 26 Dec 2002 11:29:29 -0500 (EST)
Received: from hlch80e.htc.com by insdbod01.inz.gs.com with ESMTP;
        Thu, 26 Dec 2002 11:29:29 -0500
Received: by hlch80e.htc.com with Internet Mail Service (5.5.2655.55)
        id <YXG8FQ91>; Thu, 26 Dec 2002 10:29:29 -0600
Received: from htc.com (coke.htc.com [139.172.58.10]) by hlch80e.htc.com with
        SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2655.55)
        id YXG8FQ9D; Thu, 26 Dec 2002 10:29:24 -0600
From: "Wipf, Stefan" <Stefan.wipf@gs.com>
To: "Wheeler, John" <JWheeler@us.britannica.com>
Message-Id: <3E0B2E64.1C13FE83@htc.com>
Date: Thu, 26 Dec 2002 10:29:24 -0600
X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.8 sun4u)
X-Accept-Language: en
MIME-Version: 1.0
References: <3E77AFA3658B514B9D8BB982ADDC934E1C4478@chiex02.britannica.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
cc: help-cfengine@gnu.org
cc: "Wipf, Stefan" <Stefan.wipf@gs.com>
Subject: Re: "include" files
X-BeenThere: help-cfengine@gnu.org
X-Mailman-Version: 2.1b5
Precedence: list
List-Id: Users list for GNU cfengine  <help-cfengine.gnu.org>
List-Help: <mailto:help-cfengine-request@gnu.org?subject=help>
List-Post: <mailto:help-cfengine@gnu.org>
List-Subscribe: <http://mail.gnu.org/mailman/listinfo/help-cfengine>,
        <mailto:help-cfengine-request@gnu.org?subject=subscribe>
List-Archive: <http://mail.gnu.org/pipermail/help-cfengine>
List-Unsubscribe: <http://mail.gnu.org/mailman/listinfo/help-cfengine>,
        <mailto:help-cfengine-request@gnu.org?subject=unsubscribe>
X-List-Received-Date: Thu, 26 Dec 2002 16:32:58 -0000

In regard to #1 you are correct, The final action sequence
is constructed by appending to it all actionsequences in the
order in which they were imported.  Two words of caution:
1) As you likely do not want to perform identical actions
over and over again, you should think about using 
'actionsequence classes' for better control. 
2) By default, the 'import' statement in cfengine does not
nest as expected.  In our opinion it should behave more 
like 'include' statements in other languages, so we modified
the cfengine code
(we posted a previous message regarding this issue, you
should be able to find it in the archive)

In regard to #2, we chose to disable this new feature of
cfengine (by modifying the code). 
Structuring the actionsequence the way we have
gives us much tighter control than blindly going over the
entire actionsequence again.  Going over the entire action
sequence more than once only really works well if the order
of you actions doesn't matter and your actions occur only
once in the final actionsequence.
 
Here are our diffs to the original source code for
cfengine-2.0.5pre2:

/* nest import statements the way other languages do */
parse.c:
127c127
< { struct Item *ptr;
---
> { struct Item *ptr, *nextptr;
216c216,224
<    ParseFile(filename,sp);
---
>    nextptr = ptr->next; 
>    VIMPORT = NULL;  
>    ParseFile(filename,sp); 
>    if (VIMPORT != NULL) 
>       {  
>       ptr->next = VIMPORT;
>       while (VIMPORT->next != NULL) { VIMPORT = VIMPORT->next; }
>       VIMPORT->next = nextptr;
>       }

/* turn off recursion over global actionsequence */
cfagent.c:
167c171,172
< DoTree(2,"Main Tree"); 
---
> DoTree(1,"Main Tree");


"Wheeler, John" wrote:
> 
> I'm just starting to break out my files and follow a similar
> philosophy. I have two questions:
> 1) If I have control sections in my includes, are they all
> concatenated in order of include?
> 
> In other works, if I have:
> 
> cfagent.conf
>         import:
>                 cf.blah
>                 cf.blah2
> 
> cf.blah
>         actionsequence ( files copy shellcommands )
> 
> cf.blah2
>         actionsequence ( copy tidy files shellcommands )
> 
> will the following action sequence be compiled:
> files
> copy
> shellcommands
> copy
> tidy
> files
> shellcommands
> 
> ?
> 
> 2) I've been searching for why cfengine has two passes. When I run
> cfagent, it states that it does
> everything in the action sequence twice (pass 1 and pass 2).  I vaguly
> remember reading about this
> somewhere, but I can't remember where.
> 
> > -----Original Message-----
> > From: Wipf, Stefan [mailto:Stefan.wipf@gs.com]
> > Sent: Tuesday, December 03, 2002 10:36 AM
> > To: Martin A. Brooks
> > Cc: help-cfengine@gnu.org
> > Subject: Re: "include" files
> >
> >
> > This does not scale as well as you might think.  You might
> > also want to copy, link then copy again.  We have arranged
> > our config files by function instead:
> >
> > cfagent.conf:
> >  import:
> >    any::
> >      cf.globalclasses
> >      cf.globalvariables
> >      cf.main
> >
> > in cf.globalclasses and cf.globalvariables we define classes
> > and variables that may be used by any 'agent' cf.main imports
> > the individual self-contained 'agents'
> >
> > cf.main:
> >  import:
> >    any::
> >      cf.nis
> >      cf.hosts
> >      cf.auto
> >      cf.dns
> >      cf.ntp
> >      cf.inetd
> >      cf.passwd
> >      ...
> >
> > here is a simple example of an agent:
> >
> > cf.ntp:
> >  control:
> >   actionsequence = (
> >                      copy._ntp
> >                      shellcommands._ntp
> >                     )
> >  groups:
> >   _ntp_server    = ( somemachinename )
> >
> >  copy:
> >   _ntp._ntp_server::
> >     ${gold}/etc/ntp.conf.default   dest=/etc/ntp.conf
> > define=_ntp_bounce
> >   _ntp.!_ntp_server::
> >     ${gold}/etc/ntp.conf.ntpserver dest=/etc/ntp.conf
> > define=_ntp_bounce
> >
> >  shellcommands:
> >   _ntp._ntp_bounce::
> >     "/etc/init.d/xntpd restart"
> >
> >
> > The actionsequence class '_ntp' is what makes the whole thing
> > work. See 'actionsequence classes' in the Tutorial
> >
> >
> >
> >

-- 
Stefan Wipf
swipf@htc.com