From MAILER-DAEMON Thu Jun 01 15:38:32 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Flt03-0007MW-Vp for mharc-logs-devel@gnu.org; Thu, 01 Jun 2006 15:38:32 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Flt01-0007LJ-Cy for logs-devel@nongnu.org; Thu, 01 Jun 2006 15:38:29 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Flszz-0007KN-9B for Logs-devel@nongnu.org; Thu, 01 Jun 2006 15:38:28 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Flszz-0007KH-31 for Logs-devel@nongnu.org; Thu, 01 Jun 2006 15:38:27 -0400 Received: from [64.233.182.186] (helo=nf-out-0910.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Flt6I-0008KU-6l for Logs-devel@nongnu.org; Thu, 01 Jun 2006 15:44:58 -0400 Received: by nf-out-0910.google.com with SMTP id y38so611846nfb for ; Thu, 01 Jun 2006 12:38:25 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=oW39hpja/umoGOhRlhp9hFNB3nq/ich1dryarACmGemzNYWLccWuFKTG8UVaQ2SgNn0Q+pX2Ans9snLwqGF2oha/dJupWzBMPoIAeHIkWDogB069NZlElhzxmQJMQsJ4fQucW2aORh0Tq4ER1TS1H0ucDGPQotCWui/GKEIEaHo= Received: by 10.48.211.19 with SMTP id j19mr47045nfg; Thu, 01 Jun 2006 12:38:25 -0700 (PDT) Received: by 10.48.213.11 with HTTP; Thu, 1 Jun 2006 12:38:25 -0700 (PDT) Message-ID: <5dd3d61b0606011238v2b334f7co94335b6afafc7ba7@mail.gmail.com> Date: Thu, 1 Jun 2006 14:38:25 -0500 From: "Vijay Lakshminarayanan" To: Logs-devel@nongnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: [Logs-devel] :relative-timeout vs :timeout X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jun 2006 19:38:29 -0000 Jim What's the difference between :timeout (+ *now* (* timeout INTERNAL-TIME-UNITS-PER-SECOND)) and :relative-timeout timeout ? Was :relative-timeout recently introduced? Thanks Vijay From MAILER-DAEMON Thu Jun 01 16:42:34 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Flu01-0006r9-TV for mharc-logs-devel@gnu.org; Thu, 01 Jun 2006 16:42:33 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Flu00-0006r1-Qm for logs-devel@nongnu.org; Thu, 01 Jun 2006 16:42:32 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fltzy-0006qp-Fo for Logs-devel@nongnu.org; Thu, 01 Jun 2006 16:42:31 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fltzy-0006qm-DF for Logs-devel@nongnu.org; Thu, 01 Jun 2006 16:42:30 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1Flu6I-0004ey-1n for Logs-devel@nongnu.org; Thu, 01 Jun 2006 16:49:02 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1Fltzq-0003U5-I3; Thu, 01 Jun 2006 14:42:22 -0600 Date: Thu, 1 Jun 2006 14:42:21 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] :relative-timeout vs :timeout In-Reply-To: <5dd3d61b0606011238v2b334f7co94335b6afafc7ba7@mail.gmail.com> Message-ID: References: <5dd3d61b0606011238v2b334f7co94335b6afafc7ba7@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jun 2006 20:42:33 -0000 Hi Vijay, There is both a relative timeout and an absolute timeout (once again, borrowed from Logsurfer). the difference is that an absolute timeout happens at a particular time and a relative timeout is an amount of time since you've last matched a message. make sense? Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 On Thu, 1 Jun 2006, Vijay Lakshminarayanan wrote: > Jim > > What's the difference between > > :timeout (+ *now* (* timeout INTERNAL-TIME-UNITS-PER-SECOND)) > and > :relative-timeout timeout > ? > > Was :relative-timeout recently introduced? > > Thanks > Vijay > > > _______________________________________________ > LoGS-devel mailing list > LoGS-devel@nongnu.org > http://lists.nongnu.org/mailman/listinfo/logs-devel > From MAILER-DAEMON Sat Jun 03 20:22:00 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FmgNT-00039E-UQ for mharc-logs-devel@gnu.org; Sat, 03 Jun 2006 20:21:59 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FmgNS-00038p-JN for logs-devel@nongnu.org; Sat, 03 Jun 2006 20:21:58 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FmgNQ-00034T-Et for Logs-devel@nongnu.org; Sat, 03 Jun 2006 20:21:57 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FmgNQ-00034D-6w for Logs-devel@nongnu.org; Sat, 03 Jun 2006 20:21:56 -0400 Received: from [64.233.182.187] (helo=nf-out-0910.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FmgUE-00066e-1c for Logs-devel@nongnu.org; Sat, 03 Jun 2006 20:28:58 -0400 Received: by nf-out-0910.google.com with SMTP id y38so1229766nfb for ; Sat, 03 Jun 2006 17:21:55 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=bj8wfAw9b39Cv2zF5KM13RUDDhlaw2f6i00zGsc/LoAXG+FOzIfKjDaYH1Mh4M+w/1RY3oDuNPKe7cS4i9/PegFFSFSiDK/POIgbu0worO/NnYXCCuUMeJfsf4h5rLEOCD6hPD/k2qHAivKkoFrOJ47g3V+Y4TF27xQFV/ludGY= Received: by 10.48.14.19 with SMTP id 19mr1969367nfn; Sat, 03 Jun 2006 17:21:54 -0700 (PDT) Received: by 10.48.213.16 with HTTP; Sat, 3 Jun 2006 17:21:54 -0700 (PDT) Message-ID: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> Date: Sat, 3 Jun 2006 19:21:54 -0500 From: "Vijay Lakshminarayanan" To: Logs-devel@nongnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: [Logs-devel] rule macro attempt #1 X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Jun 2006 00:21:59 -0000 Hi Jim Here's a very poor first attempt at making a macro to define rules easily. I don't have ideas for how to include environment, continue, kill-rule-after-usage etc. There are currently four clauses accepted (and that too not too well) and :actions aren't allowed, but I think it isn't too bad as an idea (can't say the same about the program). ;;; Influenced by Peter Norvig's LOOP implementation (defstruct rool (name '()) (match '()) (timeout '()) (relative-timeout '()) (delete-rule '()) (continuep '()) (actions '())) (defmacro rule (&rest exprs) (let ((r (make-rool))) (parse-rule r exprs) (fill-rule-template r))) (defun fill-rule-template (rool) `(make-instance 'logs::rule ,@(loop as (key fn) in '((:match rool-match) (:name rool-name) (:timeout rool-timeout) (:relative-timeout rool-relative-timeout) (:actions rool-actions)) as res = (funcall fn rool) if res append `(,key ,res)))) (defun parse-rule (rool exprs) (unless (null exprs) (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) (defun parse-keyword (rool keyword exprs) (if (get keyword 'handle-fn) (funcall (get keyword 'handle-fn) rool exprs) (error "Unknown keyword ~S" keyword))) (defun handle-name (rool exprs) (destructuring-bind (name . cdr) exprs (if (and (symbolp name) (null (rool-name rool))) (setf (rool-name rool) `',name) (error "Invalid name ~S" name)) cdr)) (defun handle-match (rool exprs) (let ((matches (rool-match rool))) ;; For now, we ignore. Later we add or something (declare (ignore matches)) (destructuring-bind (car . cdr) exprs (case car (regexp (let ((msg (gensym "MESSAGE")) (regex (pop cdr))) (setf (rool-match rool) `(lambda (,msg) (cl-ppcre:scan-to-strings ,regex ,msg))) cdr)) (message-length (let ((msg (gensym "MESSAGE")) (relop (pop cdr)) (num (pop cdr))) (setf (rool-match rool) `(lambda (,msg) (,relop (length (message ,msg)) ,num))) cdr)) (t (setf (rool-match rool) car) cdr))))) (defun handle-timeout (rool exprs) (destructuring-bind (preposition . (time . cdr)) exprs (case preposition (in (setf (rool-relative-timeout rool) time)) (at (setf (rool-timeout rool) time))) cdr)) (setf (get 'match 'handle-fn) #'handle-match) (setf (get 'name 'handle-fn) #'handle-name) (setf (get 'timeout 'handle-fn) #'handle-timeout) ;;;;; ========= Here are some examples with their expansions: CL-USER> (macroexpand '(rule match regexp "a.*b")) (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH (LAMBDA (#:MESSAGE1939) (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1939))) T CL-USER> (macroexpand '(rule name 'simple-1 match regexp "a.*b")) (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH (LAMBDA (#:MESSAGE1940) (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1940)) :NAME 'SIMPLE-1) T CL-USER> (macroexpand '(rule name 'simple-1 match message-length <= 80)) (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH (LAMBDA (#:MESSAGE1941) (<= (LENGTH (MESSAGE #:MESSAGE1941)) 80)) :NAME 'SIMPLE-1) T CL-USER> (macroexpand '(rule name 'simple-1 match #'match-all)) (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH #'MATCH-ALL :NAME 'SIMPLE-1) T CL-USER> (macroexpand '(rule name 'simple-1 timeout in (+ 4 8))) (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :NAME 'SIMPLE-1 :RELATIVE-TIMEOUT (+ 4 8)) T CL-USER> (macroexpand '(rule name 'simple-1 timeout at 85933)) (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :NAME 'SIMPLE-1 :TIMEOUT 85933) T Please give me your thoughts. In which time hopefully I'll get a better implementation idea :) Thanks Vijay From MAILER-DAEMON Sun Jun 04 14:13:23 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fmx6J-00056T-2i for mharc-logs-devel@gnu.org; Sun, 04 Jun 2006 14:13:23 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fmx6G-00055i-Qt for logs-devel@nongnu.org; Sun, 04 Jun 2006 14:13:20 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fmx6C-00052u-1t for Logs-devel@nongnu.org; Sun, 04 Jun 2006 14:13:19 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fmx6B-00052r-QL for Logs-devel@nongnu.org; Sun, 04 Jun 2006 14:13:15 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FmxD9-0005Cn-Ri for Logs-devel@nongnu.org; Sun, 04 Jun 2006 14:20:28 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1Fmx63-000836-L9; Sun, 04 Jun 2006 12:13:08 -0600 Date: Sun, 4 Jun 2006 12:13:05 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> Message-ID: References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Jun 2006 18:13:21 -0000 Hi Vijay I've only had a few moments to look this over. So far, it looks pretty good! I've changed HANDLE-NAME to: (defun handle-name (rool exprs) (destructuring-bind (name . cdr) exprs ;; rule names don't have to be symbols ;; symbols are (obviously) a pretty good idea, but ;; strings, etc. should work as well (if (null (rool-name rool)) (setf (rool-name rool) `',name) (error "Invalid name ~S" name)) cdr)) Jim > Here's a very poor first attempt at making a macro to define rules > easily. I don't have ideas for how to include environment, continue, > kill-rule-after-usage etc. There are currently four clauses accepted > (and that too not too well) and :actions aren't allowed, but I think > it isn't too bad as an idea (can't say the same about the program). > > ;;; Influenced by Peter Norvig's LOOP implementation > > (defstruct rool > (name '()) > (match '()) > (timeout '()) (relative-timeout '()) > (delete-rule '()) (continuep '()) > (actions '())) > > (defmacro rule (&rest exprs) > (let ((r (make-rool))) > (parse-rule r exprs) > (fill-rule-template r))) > > (defun fill-rule-template (rool) > `(make-instance > 'logs::rule > ,@(loop as (key fn) in '((:match rool-match) > (:name rool-name) > (:timeout rool-timeout) > (:relative-timeout rool-relative-timeout) > (:actions rool-actions)) > as res = (funcall fn rool) > if res append `(,key ,res)))) > > (defun parse-rule (rool exprs) > (unless (null exprs) > (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) > > (defun parse-keyword (rool keyword exprs) > (if (get keyword 'handle-fn) > (funcall (get keyword 'handle-fn) rool exprs) > (error "Unknown keyword ~S" keyword))) > > (defun handle-name (rool exprs) > (destructuring-bind (name . cdr) exprs > (if (and (symbolp name) > (null (rool-name rool))) > (setf (rool-name rool) `',name) > (error "Invalid name ~S" name)) > cdr)) > > (defun handle-match (rool exprs) > (let ((matches (rool-match rool))) > ;; For now, we ignore. Later we add or something > (declare (ignore matches)) > (destructuring-bind (car . cdr) exprs > (case car > (regexp (let ((msg (gensym "MESSAGE")) > (regex (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (cl-ppcre:scan-to-strings ,regex ,msg))) > cdr)) > (message-length (let ((msg (gensym "MESSAGE")) > (relop (pop cdr)) > (num (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (,relop (length (message ,msg)) ,num))) > cdr)) > (t (setf (rool-match rool) car) cdr))))) > > (defun handle-timeout (rool exprs) > (destructuring-bind (preposition . (time . cdr)) exprs > (case preposition > (in (setf (rool-relative-timeout rool) time)) > (at (setf (rool-timeout rool) time))) > cdr)) > > (setf (get 'match 'handle-fn) #'handle-match) > (setf (get 'name 'handle-fn) #'handle-name) > (setf (get 'timeout 'handle-fn) #'handle-timeout) > > ;;;;; ========= > > Here are some examples with their expansions: > > CL-USER> (macroexpand '(rule match regexp "a.*b")) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :MATCH > (LAMBDA (#:MESSAGE1939) > (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1939))) > T > CL-USER> (macroexpand '(rule name 'simple-1 match regexp "a.*b")) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :MATCH > (LAMBDA (#:MESSAGE1940) > (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1940)) > :NAME > 'SIMPLE-1) > T > CL-USER> (macroexpand '(rule name 'simple-1 > match message-length <= 80)) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :MATCH > (LAMBDA (#:MESSAGE1941) > (<= (LENGTH (MESSAGE #:MESSAGE1941)) 80)) > :NAME > 'SIMPLE-1) > T > CL-USER> (macroexpand '(rule name 'simple-1 > match #'match-all)) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH #'MATCH-ALL :NAME 'SIMPLE-1) > T > CL-USER> (macroexpand '(rule name 'simple-1 timeout in (+ 4 8))) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :NAME > 'SIMPLE-1 > :RELATIVE-TIMEOUT > (+ 4 8)) > T > CL-USER> (macroexpand '(rule name 'simple-1 timeout at 85933)) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :NAME 'SIMPLE-1 :TIMEOUT 85933) > T > > Please give me your thoughts. In which time hopefully I'll get a > better implementation idea :) > > Thanks > Vijay > > > _______________________________________________ > LoGS-devel mailing list > LoGS-devel@nongnu.org > http://lists.nongnu.org/mailman/listinfo/logs-devel > From MAILER-DAEMON Sun Jun 04 15:00:25 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fmxpp-00010E-Rk for mharc-logs-devel@gnu.org; Sun, 04 Jun 2006 15:00:25 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fmxpo-0000yL-AJ for logs-devel@nongnu.org; Sun, 04 Jun 2006 15:00:24 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fmxpl-0000v8-Ip for Logs-devel@nongnu.org; Sun, 04 Jun 2006 15:00:23 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fmxpl-0000v0-6V for Logs-devel@nongnu.org; Sun, 04 Jun 2006 15:00:21 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1Fmxwj-0001gi-Ar for Logs-devel@nongnu.org; Sun, 04 Jun 2006 15:07:33 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FmxpR-0008Ts-HM; Sun, 04 Jun 2006 13:00:18 -0600 Date: Sun, 4 Jun 2006 12:59:59 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> Message-ID: References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -2.9 (--) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Jun 2006 19:00:25 -0000 Hello, I was wondering how you're envisioning this working for rules that create rules ... that create rules? (make-instance 'rule :actions (list (lambda (message) (enqueue *root-ruleset* (make-instance 'rule ...))))) Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 On Sat, 3 Jun 2006, Vijay Lakshminarayanan wrote: > Hi Jim > > Here's a very poor first attempt at making a macro to define rules > easily. I don't have ideas for how to include environment, continue, > kill-rule-after-usage etc. There are currently four clauses accepted > (and that too not too well) and :actions aren't allowed, but I think > it isn't too bad as an idea (can't say the same about the program). > > ;;; Influenced by Peter Norvig's LOOP implementation > > (defstruct rool > (name '()) > (match '()) > (timeout '()) (relative-timeout '()) > (delete-rule '()) (continuep '()) > (actions '())) > > (defmacro rule (&rest exprs) > (let ((r (make-rool))) > (parse-rule r exprs) > (fill-rule-template r))) > > (defun fill-rule-template (rool) > `(make-instance > 'logs::rule > ,@(loop as (key fn) in '((:match rool-match) > (:name rool-name) > (:timeout rool-timeout) > (:relative-timeout rool-relative-timeout) > (:actions rool-actions)) > as res = (funcall fn rool) > if res append `(,key ,res)))) > > (defun parse-rule (rool exprs) > (unless (null exprs) > (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) > > (defun parse-keyword (rool keyword exprs) > (if (get keyword 'handle-fn) > (funcall (get keyword 'handle-fn) rool exprs) > (error "Unknown keyword ~S" keyword))) > > (defun handle-name (rool exprs) > (destructuring-bind (name . cdr) exprs > (if (and (symbolp name) > (null (rool-name rool))) > (setf (rool-name rool) `',name) > (error "Invalid name ~S" name)) > cdr)) > > (defun handle-match (rool exprs) > (let ((matches (rool-match rool))) > ;; For now, we ignore. Later we add or something > (declare (ignore matches)) > (destructuring-bind (car . cdr) exprs > (case car > (regexp (let ((msg (gensym "MESSAGE")) > (regex (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (cl-ppcre:scan-to-strings ,regex ,msg))) > cdr)) > (message-length (let ((msg (gensym "MESSAGE")) > (relop (pop cdr)) > (num (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (,relop (length (message ,msg)) ,num))) > cdr)) > (t (setf (rool-match rool) car) cdr))))) > > (defun handle-timeout (rool exprs) > (destructuring-bind (preposition . (time . cdr)) exprs > (case preposition > (in (setf (rool-relative-timeout rool) time)) > (at (setf (rool-timeout rool) time))) > cdr)) > > (setf (get 'match 'handle-fn) #'handle-match) > (setf (get 'name 'handle-fn) #'handle-name) > (setf (get 'timeout 'handle-fn) #'handle-timeout) > > ;;;;; ========= > > Here are some examples with their expansions: > > CL-USER> (macroexpand '(rule match regexp "a.*b")) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :MATCH > (LAMBDA (#:MESSAGE1939) > (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1939))) > T > CL-USER> (macroexpand '(rule name 'simple-1 match regexp "a.*b")) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :MATCH > (LAMBDA (#:MESSAGE1940) > (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1940)) > :NAME > 'SIMPLE-1) > T > CL-USER> (macroexpand '(rule name 'simple-1 > match message-length <= 80)) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :MATCH > (LAMBDA (#:MESSAGE1941) > (<= (LENGTH (MESSAGE #:MESSAGE1941)) 80)) > :NAME > 'SIMPLE-1) > T > CL-USER> (macroexpand '(rule name 'simple-1 > match #'match-all)) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH #'MATCH-ALL :NAME 'SIMPLE-1) > T > CL-USER> (macroexpand '(rule name 'simple-1 timeout in (+ 4 8))) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > :NAME > 'SIMPLE-1 > :RELATIVE-TIMEOUT > (+ 4 8)) > T > CL-USER> (macroexpand '(rule name 'simple-1 timeout at 85933)) > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :NAME 'SIMPLE-1 :TIMEOUT 85933) > T > > Please give me your thoughts. In which time hopefully I'll get a > better implementation idea :) > > Thanks > Vijay > > > _______________________________________________ > LoGS-devel mailing list > LoGS-devel@nongnu.org > http://lists.nongnu.org/mailman/listinfo/logs-devel > From MAILER-DAEMON Sun Jun 04 16:04:24 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fmypk-0001eD-1w for mharc-logs-devel@gnu.org; Sun, 04 Jun 2006 16:04:24 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fmypi-0001e8-Fh for logs-devel@nongnu.org; Sun, 04 Jun 2006 16:04:22 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fmypf-0001dT-Oz for Logs-devel@nongnu.org; Sun, 04 Jun 2006 16:04:21 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fmypf-0001cy-Ic for Logs-devel@nongnu.org; Sun, 04 Jun 2006 16:04:19 -0400 Received: from [64.233.182.185] (helo=nf-out-0910.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Fmywe-0005lc-Ix for Logs-devel@nongnu.org; Sun, 04 Jun 2006 16:11:32 -0400 Received: by nf-out-0910.google.com with SMTP id y38so1306924nfb for ; Sun, 04 Jun 2006 13:04:18 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=b2FkIIQZtDLdm2JHt0j/FBCRPgWwJ9GsFw3UP0VXVym/QNN8rjTHJckYR19oMuLBE0hDByefZLtTtq/pJqfkRLMjiMTW1eBXNHXolFiQEXDjLjJ8A4D3HxeCpEtrmIGEO74SWE4pvnKXEQ4N3OWWfCBEFuFf2JMuUXgAoA10OG8= Received: by 10.49.68.13 with SMTP id v13mr2598648nfk; Sun, 04 Jun 2006 13:04:18 -0700 (PDT) Received: by 10.48.213.16 with HTTP; Sun, 4 Jun 2006 13:04:18 -0700 (PDT) Message-ID: <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> Date: Sun, 4 Jun 2006 15:04:18 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Jun 2006 20:04:23 -0000 On 6/4/06, Jim Prewett wrote: > I was wondering how you're envisioning this working for rules that create > rules ... that create rules? > > (make-instance 'rule > :actions > (list > (lambda (message) > (enqueue *root-ruleset* (make-instance 'rule ...))))) We just use the macro again, I guess. (rule performing (lambda (message) (enqueue *root-ruleset* (rule ...)))) In time we should have a single function which does the enqueing. (defmethod root-ruleset-enqueue ((rule rule)) (enqueue *root-ruleset* rule)) (rule performing #'root-ruleset-enqueue on (rule ...)) => (make-instance 'rule :actions (list (lambda (#:sym123) (funcall #'root-ruleset-enqueue (rule ...))))) This is on macroexpand-1 (by my idea). A more complete set will have to be worked on I guess. performing fn on Y => (list (lambda (#:sym123) ; must be some way to refer to this (funcall fn Y))) performing foo and bar on something => (list foo (funcall bar something) ; can we inline this to ; (bar something) ? ) performing something with (message matches sub-matches) => (list (lambda (message matches sub-matches) (declare (ignorable message matches sub-matches)) something)) ; we use variable capture to our advantage? These are getting complicated. but IMHO I think it is the right track to be on. after which we could easily implement the SEC functions > Jim > > James E. Prewett Jim@Prewett.org download@hpc.unm.edu > Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ > Designated Security Officer OpenPGP key: pub 1024D/31816D93 > HPC Systems Engineer III UNM HPC 505.277.8210 > > On Sat, 3 Jun 2006, Vijay Lakshminarayanan wrote: > > > Hi Jim > > > > Here's a very poor first attempt at making a macro to define rules > > easily. I don't have ideas for how to include environment, continue, > > kill-rule-after-usage etc. There are currently four clauses accepted > > (and that too not too well) and :actions aren't allowed, but I think > > it isn't too bad as an idea (can't say the same about the program). > > > > ;;; Influenced by Peter Norvig's LOOP implementation > > > > (defstruct rool > > (name '()) > > (match '()) > > (timeout '()) (relative-timeout '()) > > (delete-rule '()) (continuep '()) > > (actions '())) > > > > (defmacro rule (&rest exprs) > > (let ((r (make-rool))) > > (parse-rule r exprs) > > (fill-rule-template r))) > > > > (defun fill-rule-template (rool) > > `(make-instance > > 'logs::rule > > ,@(loop as (key fn) in '((:match rool-match) > > (:name rool-name) > > (:timeout rool-timeout) > > (:relative-timeout rool-relative-timeout) > > (:actions rool-actions)) > > as res = (funcall fn rool) > > if res append `(,key ,res)))) > > > > (defun parse-rule (rool exprs) > > (unless (null exprs) > > (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) > > > > (defun parse-keyword (rool keyword exprs) > > (if (get keyword 'handle-fn) > > (funcall (get keyword 'handle-fn) rool exprs) > > (error "Unknown keyword ~S" keyword))) > > > > (defun handle-name (rool exprs) > > (destructuring-bind (name . cdr) exprs > > (if (and (symbolp name) > > (null (rool-name rool))) > > (setf (rool-name rool) `',name) > > (error "Invalid name ~S" name)) > > cdr)) > > > > (defun handle-match (rool exprs) > > (let ((matches (rool-match rool))) > > ;; For now, we ignore. Later we add or something > > (declare (ignore matches)) > > (destructuring-bind (car . cdr) exprs > > (case car > > (regexp (let ((msg (gensym "MESSAGE")) > > (regex (pop cdr))) > > (setf (rool-match rool) > > `(lambda (,msg) > > (cl-ppcre:scan-to-strings ,regex ,msg))) > > cdr)) > > (message-length (let ((msg (gensym "MESSAGE")) > > (relop (pop cdr)) > > (num (pop cdr))) > > (setf (rool-match rool) > > `(lambda (,msg) > > (,relop (length (message ,msg)) ,num))) > > cdr)) > > (t (setf (rool-match rool) car) cdr))))) > > > > (defun handle-timeout (rool exprs) > > (destructuring-bind (preposition . (time . cdr)) exprs > > (case preposition > > (in (setf (rool-relative-timeout rool) time)) > > (at (setf (rool-timeout rool) time))) > > cdr)) > > > > (setf (get 'match 'handle-fn) #'handle-match) > > (setf (get 'name 'handle-fn) #'handle-name) > > (setf (get 'timeout 'handle-fn) #'handle-timeout) > > > > ;;;;; ========= > > > > Here are some examples with their expansions: > > > > CL-USER> (macroexpand '(rule match regexp "a.*b")) > > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > > :MATCH > > (LAMBDA (#:MESSAGE1939) > > (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1939))) > > T > > CL-USER> (macroexpand '(rule name 'simple-1 match regexp "a.*b")) > > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > > :MATCH > > (LAMBDA (#:MESSAGE1940) > > (CL-PPCRE:SCAN-TO-STRINGS "a.*b" #:MESSAGE1940)) > > :NAME > > 'SIMPLE-1) > > T > > CL-USER> (macroexpand '(rule name 'simple-1 > > match message-length <= 80)) > > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > > :MATCH > > (LAMBDA (#:MESSAGE1941) > > (<= (LENGTH (MESSAGE #:MESSAGE1941)) 80)) > > :NAME > > 'SIMPLE-1) > > T > > CL-USER> (macroexpand '(rule name 'simple-1 > > match #'match-all)) > > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :MATCH #'MATCH-ALL :NAME 'SIMPLE-1) > > T > > CL-USER> (macroexpand '(rule name 'simple-1 timeout in (+ 4 8))) > > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE > > :NAME > > 'SIMPLE-1 > > :RELATIVE-TIMEOUT > > (+ 4 8)) > > T > > CL-USER> (macroexpand '(rule name 'simple-1 timeout at 85933)) > > (MAKE-INSTANCE 'ORG.PREWETT.LOGS::RULE :NAME 'SIMPLE-1 :TIMEOUT 85933) > > T > > > > Please give me your thoughts. In which time hopefully I'll get a > > better implementation idea :) > > > > Thanks > > Vijay > > > > > > _______________________________________________ > > LoGS-devel mailing list > > LoGS-devel@nongnu.org > > http://lists.nongnu.org/mailman/listinfo/logs-devel > > > From MAILER-DAEMON Sun Jun 04 19:12:31 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fn1ln-000873-JA for mharc-logs-devel@gnu.org; Sun, 04 Jun 2006 19:12:31 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fn1lm-00086W-Cv for logs-devel@nongnu.org; Sun, 04 Jun 2006 19:12:30 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fn1ll-00086K-Cr for Logs-devel@nongnu.org; Sun, 04 Jun 2006 19:12:30 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fn1ll-00086H-9J for Logs-devel@nongnu.org; Sun, 04 Jun 2006 19:12:29 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1Fn1sm-0000wA-DE for Logs-devel@nongnu.org; Sun, 04 Jun 2006 19:19:44 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1Fn1lj-0002FH-Eh; Sun, 04 Jun 2006 17:12:27 -0600 Date: Sun, 4 Jun 2006 17:12:25 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> Message-ID: References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Jun 2006 23:12:30 -0000 > On 6/4/06, Jim Prewett wrote: > > I was wondering how you're envisioning this working for rules that create > > rules ... that create rules? > > > > (make-instance 'rule > > :actions > > (list > > (lambda (message) > > (enqueue *root-ruleset* (make-instance 'rule ...))))) > > We just use the macro again, I guess. > > (rule performing (lambda (message) (enqueue *root-ruleset* (rule ...)))) > I was hoping that was the answer :) > In time we should have a single function which does the enqueing. I don't think you're quite understanding the ruleset tree structure. rulesets contain rules and other rulesets. You may want to have a rule that creates a new rule on a completely different part of the tree. In other words, I don't see specifying the ruleset you're adding the rule to to be a loss. Jim From MAILER-DAEMON Sun Jun 04 20:04:13 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fn2Zp-0004L5-1Q for mharc-logs-devel@gnu.org; Sun, 04 Jun 2006 20:04:13 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fn2Zn-0004L0-MR for logs-devel@nongnu.org; Sun, 04 Jun 2006 20:04:11 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fn2Zl-0004Kn-Vw for Logs-devel@nongnu.org; Sun, 04 Jun 2006 20:04:11 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fn2Zl-0004Kk-Pl for Logs-devel@nongnu.org; Sun, 04 Jun 2006 20:04:09 -0400 Received: from [64.233.182.185] (helo=nf-out-0910.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Fn2gn-00042h-9J for Logs-devel@nongnu.org; Sun, 04 Jun 2006 20:11:25 -0400 Received: by nf-out-0910.google.com with SMTP id y38so1324118nfb for ; Sun, 04 Jun 2006 17:04:08 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=c8RuywvwODsJfPaPUe+uq8PiMLTfg1uugLFCTXmys0a6He9YbCgPdmSWeZRZ7wLxhBJsQ/MnRZyEAgr6pKJjxw2dQB3lW/UIafLZnh4SMtLhw7g1P8uVQ3ZXJJwxcg6UbDTq1VyB3gQA/VRr7IcISTE73NQdUObUImDMjcjfeg8= Received: by 10.48.30.3 with SMTP id d3mr2728018nfd; Sun, 04 Jun 2006 17:04:08 -0700 (PDT) Received: by 10.48.213.16 with HTTP; Sun, 4 Jun 2006 17:04:08 -0700 (PDT) Message-ID: <5dd3d61b0606041704v31303955o4f41c785c68bb269@mail.gmail.com> Date: Sun, 4 Jun 2006 19:04:08 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jun 2006 00:04:11 -0000 On 6/4/06, Jim Prewett wrote: > > > I was wondering how you're envisioning this working for rules that create > > > rules ... that create rules? > > > > > > (make-instance 'rule > > > :actions > > > (list > > > (lambda (message) > > > (enqueue *root-ruleset* (make-instance 'rule ...))))) > > > > We just use the macro again, I guess. > > > > (rule performing (lambda (message) (enqueue *root-ruleset* (rule > ...)))) > > > > I was hoping that was the answer :) Oh so it's a Good Thing? That's a relief :-) > > In time we should have a single function which does the enqueing. > > I don't think you're quite understanding the ruleset tree structure. > rulesets contain rules and other rulesets. You may want to have a rule > that creates a new rule on a completely different part of the tree. You're right. I completely forgot about this -- I was totally immersed into defining the macro's behavior itself. > In other words, I don't see specifying the ruleset you're adding the rule > to to be a loss. > > Jim > Will come up with more and better quality code soon, Jim. Thanks Vijay From MAILER-DAEMON Sun Jun 04 21:31:14 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fn3w1-0003Oq-UP for mharc-logs-devel@gnu.org; Sun, 04 Jun 2006 21:31:13 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fn3w0-0003Of-Sx for logs-devel@nongnu.org; Sun, 04 Jun 2006 21:31:12 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fn3vx-0003O9-5z for Logs-devel@nongnu.org; Sun, 04 Jun 2006 21:31:11 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fn3vx-0003O6-1j for Logs-devel@nongnu.org; Sun, 04 Jun 2006 21:31:09 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1Fn42z-0000lV-3s for Logs-devel@nongnu.org; Sun, 04 Jun 2006 21:38:25 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1Fn3vu-0003Oa-FU; Sun, 04 Jun 2006 19:31:07 -0600 Date: Sun, 4 Jun 2006 19:31:03 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: <5dd3d61b0606041704v31303955o4f41c785c68bb269@mail.gmail.com> Message-ID: References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> <5dd3d61b0606041704v31303955o4f41c785c68bb269@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jun 2006 01:31:13 -0000 Hi Vijay, I've had fun playing with your code today (unfortunately, some yard work, etc. kept me busy for too much of it :). I've made a few minor modifications, and generally think your code has a nice feel to it :) kudos! Here is an example using some of the new things I've added: (rule name 'foo match regexp ".+" ;; set variable var1 to 42 setenv var1 = 42 ;; first action doing (lambda (x) (format t "message: ~A var1: ~A~%" (message x) var1)) ;; second action doing #'print-message ;; set continuep slot to t continue) ;;; Paul Graham, On Lisp, p191 (defmacro aif (test-form then-form &optional else-form) `(let ((it ,test-form)) (if it ,then-form ,else-form))) ;;; an 'accessor' for the handle-fns (defmacro handle-fn (keyword) `(get ,keyword 'handle-fn)) ;;; Influenced by Peter Norvig's LOOP implementation (defstruct rool (name '()) (match '()) (timeout '()) (relative-timeout '()) (delete-rule '()) (continuep '()) (actions '()) (environment '())) (defmacro rule (&rest exprs) (let ((r (make-rool))) (parse-rule r exprs) (fill-rule-template r))) (defun fill-rule-template (rool) `(make-instance 'logs::rule :environment ',(rool-environment rool) :actions (rool-actions ,rool) ,@(loop as (key fn) in '((:match rool-match) (:name rool-name) (:timeout rool-timeout) (:continuep rool-continuep) (:relative-timeout rool-relative-timeout)) as res = (funcall fn rool) if res append `(,key ,res)))) (defun parse-rule (rool exprs) (unless (null exprs) (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) (defun parse-keyword (rool keyword exprs) (aif (handle-fn keyword) (funcall it rool exprs) (error "Unknown keyword ~S" keyword))) (defun handle-name (rool exprs) (destructuring-bind (name . cdr) exprs ;; rule names don't have to be symbols ;; symbols are (obviously) a pretty good idea, but ;; strings, etc. should work as well (if (null (rool-name rool)) (setf (rool-name rool) `,name) (error "Invalid name ~S" name)) cdr)) (defun handle-match (rool exprs) (let ((matches (rool-match rool))) ;; For now, we ignore. Later we add or something (declare (ignore matches)) (destructuring-bind (car . cdr) exprs (case car (regexp (let ((msg (gensym "MESSAGE")) (matches (gensym)) (sub-matches (gensym)) (regex (pop cdr))) (setf (rool-match rool) `(lambda (,msg) (multiple-value-bind (,matches ,sub-matches) (cl-ppcre:scan-to-strings ,regex (message ,msg)) (when ,matches (values t '((sub-matches ,sub-matches))))))) cdr)) (message-length (let ((msg (gensym "MESSAGE")) (relop (pop cdr)) (num (pop cdr))) (setf (rool-match rool) `(lambda (,msg) (,relop (length (message ,msg)) ,num))) cdr)) (t (setf (rool-match rool) car) cdr))))) (defun handle-timeout (rool exprs) (destructuring-bind (preposition . (time . cdr)) exprs (case preposition (in (setf (rool-relative-timeout rool) time)) (at (setf (rool-timeout rool) time))) cdr)) (defun handle-setenv (rool exprs) (destructuring-bind (variable equal value &rest rest) exprs (pushnew (list variable value) (rool-environment rool) :test (lambda (x y) (equal (car x) (car y)))) rest)) ;; add the given function to the actions list (defun handle-do (rool exprs) (destructuring-bind (todo &rest rest) exprs (format t "rool: ~A todo: ~A rest: ~A~%" rool todo rest) (append (rool-actions rool) (list (eval todo))) rest)) ;; set the continue slot to t (defun handle-continue (rool exprs) (setf (rool-continuep rool) t) exprs) (setf (handle-fn 'match) #'handle-match) (setf (handle-fn 'name) #'handle-name) (setf (handle-fn 'timeout) #'handle-timeout) ;; new (setf (handle-fn 'continue) #'handle-continue) (setf (handle-fn 'setenv) #'handle-setenv) (setf (handle-fn 'doing) #'handle-do) Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 On Sun, 4 Jun 2006, Vijay Lakshminarayanan wrote: > On 6/4/06, Jim Prewett wrote: > > > > I was wondering how you're envisioning this working for rules that > > create > > > > rules ... that create rules? > > > > > > > > (make-instance 'rule > > > > :actions > > > > (list > > > > (lambda (message) > > > > (enqueue *root-ruleset* (make-instance 'rule ...))))) > > > > > > We just use the macro again, I guess. > > > > > > (rule performing (lambda (message) (enqueue *root-ruleset* (rule > > ...)))) > > > > > > > I was hoping that was the answer :) > > Oh so it's a Good Thing? That's a relief :-) > > > > In time we should have a single function which does the enqueing. > > > > I don't think you're quite understanding the ruleset tree structure. > > rulesets contain rules and other rulesets. You may want to have a rule > > that creates a new rule on a completely different part of the tree. > > You're right. I completely forgot about this -- I was totally > immersed into defining the macro's behavior itself. > > > In other words, I don't see specifying the ruleset you're adding the rule > > to to be a loss. > > > > Jim > > > > Will come up with more and better quality code soon, Jim. > > Thanks > Vijay > From MAILER-DAEMON Mon Jun 05 02:59:22 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fn93Z-0002xC-Ul for mharc-logs-devel@gnu.org; Mon, 05 Jun 2006 02:59:22 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fn93Y-0002x7-Ix for logs-devel@nongnu.org; Mon, 05 Jun 2006 02:59:20 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fn93X-0002wv-09 for Logs-devel@nongnu.org; Mon, 05 Jun 2006 02:59:19 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fn93W-0002ws-Pj for Logs-devel@nongnu.org; Mon, 05 Jun 2006 02:59:18 -0400 Received: from [64.233.182.191] (helo=nf-out-0910.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Fn9Ac-0003Dj-1I for Logs-devel@nongnu.org; Mon, 05 Jun 2006 03:06:38 -0400 Received: by nf-out-0910.google.com with SMTP id y38so1356158nfb for ; Sun, 04 Jun 2006 23:59:17 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=NzPUmutf/ZvwdC7OubkoBgxo4Yg80xAXx9/LLeCpUxRx68+6IE31lxt37hHGtx/TJQvlFOxho5TMRtu16nhQDSYdizkCvPxbdh2EDVyJH5DcU9x6M/h3KbZiD49rcn+uGeZx4c/2PI0+DYbKUMq9zrIiPikWaAUuUo0mhhZUCu0= Received: by 10.48.255.7 with SMTP id c7mr2901474nfi; Sun, 04 Jun 2006 23:59:17 -0700 (PDT) Received: by 10.48.213.16 with HTTP; Sun, 4 Jun 2006 23:59:16 -0700 (PDT) Message-ID: <5dd3d61b0606042359i7cd39de7hbbfc7ba338f98425@mail.gmail.com> Date: Mon, 5 Jun 2006 01:59:16 -0500 From: "Vijay Lakshminarayanan" To: Logs-devel@nongnu.org Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> <5dd3d61b0606041704v31303955o4f41c785c68bb269@mail.gmail.com> Cc: X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jun 2006 06:59:20 -0000 Hi Jim I have made one or two more changes. I was shooting myself in the foot trying to use cl-yacc. I'm sure it's a great tool but I couldn't get the hang of it to parse LoGS rules. anyway, you were right about it being a large hammer. I think we can do without it. I've extended set-env to accept a LOOP like AND syntax: (rule with foo = "one" and bar = "two" do (lambda (x) (list foo bar))) I have a few more questions. I'll ask them in the morning. I've added a keyword FILTER and have a question there. Thanks Jim! ;;; Paul Graham, On Lisp, p191 (defmacro aif (test-form then-form &optional else-form) `(let ((it ,test-form)) (if it ,then-form ,else-form))) ;;; an 'accessor' for the handle-fns (defmacro handle-fn (keyword) `(get ,keyword 'handle-fn)) (defmacro alias (keyword) `(get ,keyword 'alias)) ;;; Influenced by Peter Norvig's LOOP implementation (defstruct rool (name '()) (match '()) (timeout '()) (relative-timeout '()) (delete-rule '()) (continuep '()) (actions '()) (environment '()) (delete '())) (defmacro rule (&rest exprs) (let ((r (make-rool))) (parse-rule r exprs) (fill-rule-template r))) (defun fill-rule-template (rool) `(make-instance 'logs::rule :environment ',(rool-environment rool) ,@(loop as (key fn) in '((:match rool-match) (:name rool-name) (:timeout rool-timeout) (:continuep rool-continuep) (:relative-timeout rool-relative-timeout) (:actions rool-actions)) as res = (funcall fn rool) if res append `(,key ,res)))) (defun parse-rule (rool exprs) (unless (null exprs) (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) (defun parse-keyword (rool keyword exprs) (let* ((keyword (or (alias keyword) keyword))) (aif (handle-fn keyword) (funcall it rool exprs) (error "Unknown keyword ~S" keyword)))) (defun handle-name (rool exprs) (destructuring-bind (name . cdr) exprs ;; rule names don't have to be symbols ;; symbols are (obviously) a pretty good idea, but ;; strings, etc. should work as well (if (null (rool-name rool)) (setf (rool-name rool) name) (error "Invalid name ~S" name)) cdr)) (defun handle-match (rool exprs) (let ((matches (rool-match rool))) ;; For now, we ignore. Later we add ;; facilities to add other messages (declare (ignore matches)) (destructuring-bind (car . cdr) exprs (case car (regexp (let ((msg (gensym "MESSAGE")) (matches (gensym)) (sub-matches (gensym)) (regex (pop cdr))) (setf (rool-match rool) `(lambda (,msg) (multiple-value-bind (,matches ,sub-matches) (cl-ppcre:scan-to-strings ,regex (logs::message ,msg)) (when ,matches (values t '((sub-matches ,sub-matches))))))) cdr)) (message-length (let ((msg (gensym "MESSAGE")) (relop (pop cdr)) (num (pop cdr))) (setf (rool-match rool) `(lambda (,msg) (,relop (length (logs::message ,msg)) ,num))) cdr)) (t (setf (rool-match rool) car) cdr))))) (defun handle-filter (rool exprs) (destructuring-bind (regexp regex . cdr) exprs (let ((exprs (handle-match rool exprs))) (setf (rool-continuep rool) t ;; There has to be some way to set the ;; whole to just NIL. ;; I see the following possibilities: ;; we parse EXPRS and remove all ;; action clauses. ;; We change the design of LoGS such ;; that actions is actually (by ;; default) (block NIL (list action-1 ;; action-2)) and then we just do a ;; (setf (rool-actions rool) '(return)) (rool-actions rool) nil)))) (defun handle-timeout (rool exprs) (destructuring-bind (preposition time . cdr) exprs (case preposition (in (setf (rool-relative-timeout rool) time)) (at (setf (rool-timeout rool) time))) cdr)) (defun handle-setenv (rool exprs) (destructuring-bind (variable = value &rest rest) exprs (declare (ignore =)) (pushnew (list variable value) (rool-environment rool) :test (lambda (x y) (equal (car x) (car y)))) (if (eq (car rest) 'and) ;; More than one variable has been ;; defined. Save that too. (handle-setenv rool (cdr rest)) rest))) ;; add the given function to the actions list (defun handle-do (rool exprs) (destructuring-bind (todo &rest rest) exprs (format t "rool: ~A~%todo: ~A~%rest: ~A~%~ rool-actions:~S~%~50~~%" rool todo rest (rool-actions rool)) (setf (rool-actions rool) (append (rool-actions rool) (list (eval todo)))) rest)) ;; set the continue slot to t (defun handle-continue (rool exprs) (setf (rool-continuep rool) t) exprs) ;; set the delete slot to t (defun handle-delete (rool exprs) (setf (rool-delete rool) t) exprs) ;;; Set aliases for keywords so we have synonyms ;;; and define the handler function (defmacro defhandler (keywords alias handle-fn) `(setf ,@(if (consp keywords) (loop as keyword in keywords appending `((alias ',keyword) ',alias)) `((alias ',keywords) ',alias)) (handle-fn ',alias) ,handle-fn)) (defhandler (match matching) :match #'handle-match) (defhandler (name named) :name #'handle-name) (defhandler (do doing) :do #'handle-do) (defhandler continue :continue #'handle-continue) (defhandler timeout :timeout #'handle-timeout) (defhandler (set-env with) :set-env #'handle-setenv) (defhandler delete :delete #'handle-delete) ;;; *eof* On 6/4/06, Jim Prewett wrote: > > Hi Vijay, > > I've had fun playing with your code today (unfortunately, some yard work, > etc. kept me busy for too much of it :). I've made a few minor > modifications, and generally think your code has a nice feel to it :) > kudos! > > Here is an example using some of the new things I've added: > > (rule > name 'foo > match regexp ".+" > ;; set variable var1 to 42 > setenv var1 = 42 > ;; first action > doing (lambda (x) (format t "message: ~A var1: ~A~%" (message x) > var1)) > ;; second action > doing #'print-message > ;; set continuep slot to t > continue) > > ;;; Paul Graham, On Lisp, p191 > (defmacro aif (test-form then-form &optional else-form) > `(let ((it ,test-form)) > (if it ,then-form ,else-form))) > > ;;; an 'accessor' for the handle-fns > (defmacro handle-fn (keyword) > `(get ,keyword 'handle-fn)) > > ;;; Influenced by Peter Norvig's LOOP implementation > > (defstruct rool > (name '()) > (match '()) > (timeout '()) > (relative-timeout '()) > (delete-rule '()) > (continuep '()) > (actions '()) > (environment '())) > > (defmacro rule (&rest exprs) > (let ((r (make-rool))) > (parse-rule r exprs) > (fill-rule-template r))) > > (defun fill-rule-template (rool) > `(make-instance > 'logs::rule > :environment ',(rool-environment rool) > :actions (rool-actions ,rool) > ,@(loop as (key fn) in '((:match rool-match) > (:name rool-name) > (:timeout rool-timeout) > (:continuep rool-continuep) > (:relative-timeout rool-relative-timeout)) > as res = (funcall fn rool) > if res append `(,key ,res)))) > > (defun parse-rule (rool exprs) > (unless (null exprs) > (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) > > (defun parse-keyword (rool keyword exprs) > (aif (handle-fn keyword) > (funcall it rool exprs) > (error "Unknown keyword ~S" keyword))) > > (defun handle-name (rool exprs) > (destructuring-bind (name . cdr) exprs > ;; rule names don't have to be symbols > ;; symbols are (obviously) a pretty good idea, but > ;; strings, etc. should work as well > (if (null (rool-name rool)) > (setf (rool-name rool) `,name) > (error "Invalid name ~S" name)) > cdr)) > > (defun handle-match (rool exprs) > (let ((matches (rool-match rool))) > ;; For now, we ignore. Later we add or something > (declare (ignore matches)) > (destructuring-bind (car . cdr) exprs > (case car > (regexp (let ((msg (gensym "MESSAGE")) > (matches (gensym)) > (sub-matches (gensym)) > (regex (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (multiple-value-bind (,matches ,sub-matches) > (cl-ppcre:scan-to-strings ,regex (message ,msg)) > (when ,matches > (values t > '((sub-matches ,sub-matches))))))) > cdr)) > (message-length (let ((msg (gensym "MESSAGE")) > (relop (pop cdr)) > (num (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (,relop (length (message ,msg)) ,num))) > cdr)) > (t (setf (rool-match rool) car) cdr))))) > > (defun handle-timeout (rool exprs) > (destructuring-bind (preposition . (time . cdr)) exprs > (case preposition > (in (setf (rool-relative-timeout rool) time)) > (at (setf (rool-timeout rool) time))) > cdr)) > > (defun handle-setenv (rool exprs) > (destructuring-bind (variable equal value &rest rest) exprs > (pushnew (list variable value) > (rool-environment rool) > :test > (lambda (x y) (equal (car x) (car y)))) > rest)) > > ;; add the given function to the actions list > (defun handle-do (rool exprs) > (destructuring-bind (todo &rest rest) exprs > (format t "rool: ~A todo: ~A rest: ~A~%" > rool todo rest) > (append (rool-actions rool) (list (eval todo))) > rest)) > > ;; set the continue slot to t > (defun handle-continue (rool exprs) > (setf (rool-continuep rool) t) > exprs) > > (setf (handle-fn 'match) #'handle-match) > (setf (handle-fn 'name) #'handle-name) > (setf (handle-fn 'timeout) #'handle-timeout) > > ;; new > (setf (handle-fn 'continue) #'handle-continue) > (setf (handle-fn 'setenv) #'handle-setenv) > (setf (handle-fn 'doing) #'handle-do) > > > Jim > > James E. Prewett Jim@Prewett.org download@hpc.unm.edu > Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ > Designated Security Officer OpenPGP key: pub 1024D/31816D93 > HPC Systems Engineer III UNM HPC 505.277.8210 > > On Sun, 4 Jun 2006, Vijay Lakshminarayanan wrote: > > > On 6/4/06, Jim Prewett wrote: > > > > > I was wondering how you're envisioning this working for rules that > > > create > > > > > rules ... that create rules? > > > > > > > > > > (make-instance 'rule > > > > > :actions > > > > > (list > > > > > (lambda (message) > > > > > (enqueue *root-ruleset* (make-instance 'rule ...))))) > > > > > > > > We just use the macro again, I guess. > > > > > > > > (rule performing (lambda (message) (enqueue *root-ruleset* (rule > > > ...)))) > > > > > > > > > > I was hoping that was the answer :) > > > > Oh so it's a Good Thing? That's a relief :-) > > > > > > In time we should have a single function which does the enqueing. > > > > > > I don't think you're quite understanding the ruleset tree structure. > > > rulesets contain rules and other rulesets. You may want to have a rule > > > that creates a new rule on a completely different part of the tree. > > > > You're right. I completely forgot about this -- I was totally > > immersed into defining the macro's behavior itself. > > > > > In other words, I don't see specifying the ruleset you're adding the rule > > > to to be a loss. > > > > > > Jim > > > > > > > Will come up with more and better quality code soon, Jim. > > > > Thanks > > Vijay > > > From MAILER-DAEMON Mon Jun 05 08:17:41 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FnE1c-0000T6-Tc for mharc-logs-devel@gnu.org; Mon, 05 Jun 2006 08:17:40 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FnE1b-0000Rm-OM for logs-devel@nongnu.org; Mon, 05 Jun 2006 08:17:39 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FnE1a-0000QI-Lp for Logs-devel@nongnu.org; Mon, 05 Jun 2006 08:17:39 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FnE1a-0000QA-Hg for Logs-devel@nongnu.org; Mon, 05 Jun 2006 08:17:38 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FnE8i-0005hI-Qq for Logs-devel@nongnu.org; Mon, 05 Jun 2006 08:25:01 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FnE1X-0000Ty-M8; Mon, 05 Jun 2006 06:17:36 -0600 Date: Mon, 5 Jun 2006 06:17:33 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rule macro attempt #1 In-Reply-To: <5dd3d61b0606042359i7cd39de7hbbfc7ba338f98425@mail.gmail.com> Message-ID: References: <5dd3d61b0606031721k27e821f0w5654f205c80ff0ac@mail.gmail.com> <5dd3d61b0606041304y49d5d180o272e1f77d0dfccf@mail.gmail.com> <5dd3d61b0606041704v31303955o4f41c785c68bb269@mail.gmail.com> <5dd3d61b0606042359i7cd39de7hbbfc7ba338f98425@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jun 2006 12:17:39 -0000 > I have made one or two more changes. I was shooting myself in the > foot trying to use cl-yacc. I'm sure it's a great tool but I couldn't > get the hang of it to parse LoGS rules. anyway, you were right about > it being a large hammer. I think we can do without it. That advice came from a colleague of mine, Roy Hiembach. His job is mostly to help our users with their programming isssues; for the most part, he gives really good advice :) He convinced me that any form of parser-generator would likely be overkill > I've extended set-env to accept a LOOP like AND syntax: > > (rule with foo = "one" and bar = "two" do (lambda (x) (list foo bar))) cool! I like it! > I have a few more questions. I'll ask them in the morning. I've > added a keyword FILTER and have a question there. I'll look at this later today. Jim > > Thanks Jim! > > ;;; Paul Graham, On Lisp, p191 > (defmacro aif (test-form then-form &optional else-form) > `(let ((it ,test-form)) > (if it ,then-form ,else-form))) > > ;;; an 'accessor' for the handle-fns > (defmacro handle-fn (keyword) > `(get ,keyword 'handle-fn)) > > (defmacro alias (keyword) > `(get ,keyword 'alias)) > > ;;; Influenced by Peter Norvig's LOOP implementation > (defstruct rool > (name '()) > (match '()) > (timeout '()) > (relative-timeout '()) > (delete-rule '()) > (continuep '()) > (actions '()) > (environment '()) > (delete '())) > > (defmacro rule (&rest exprs) > (let ((r (make-rool))) > (parse-rule r exprs) > (fill-rule-template r))) > > (defun fill-rule-template (rool) > `(make-instance > 'logs::rule > :environment ',(rool-environment rool) > ,@(loop as (key fn) in '((:match rool-match) > (:name rool-name) > (:timeout rool-timeout) > (:continuep rool-continuep) > (:relative-timeout rool-relative-timeout) > (:actions rool-actions)) > as res = (funcall fn rool) > if res append `(,key ,res)))) > > (defun parse-rule (rool exprs) > (unless (null exprs) > (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) > > (defun parse-keyword (rool keyword exprs) > (let* ((keyword (or (alias keyword) keyword))) > (aif (handle-fn keyword) > (funcall it rool exprs) > (error "Unknown keyword ~S" keyword)))) > > (defun handle-name (rool exprs) > (destructuring-bind (name . cdr) exprs > ;; rule names don't have to be symbols > ;; symbols are (obviously) a pretty good idea, but > ;; strings, etc. should work as well > (if (null (rool-name rool)) > (setf (rool-name rool) name) > (error "Invalid name ~S" name)) > cdr)) > > (defun handle-match (rool exprs) > (let ((matches (rool-match rool))) > ;; For now, we ignore. Later we add > ;; facilities to add other messages > (declare (ignore matches)) > (destructuring-bind (car . cdr) exprs > (case car > (regexp (let ((msg (gensym "MESSAGE")) > (matches (gensym)) > (sub-matches (gensym)) > (regex (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (multiple-value-bind (,matches ,sub-matches) > (cl-ppcre:scan-to-strings > ,regex > (logs::message ,msg)) > (when ,matches > (values t > '((sub-matches ,sub-matches))))))) > cdr)) > (message-length (let ((msg (gensym "MESSAGE")) > (relop (pop cdr)) > (num (pop cdr))) > (setf (rool-match rool) > `(lambda (,msg) > (,relop (length (logs::message > ,msg)) ,num))) > cdr)) > (t (setf (rool-match rool) car) cdr))))) > > (defun handle-filter (rool exprs) > (destructuring-bind (regexp regex . cdr) exprs > (let ((exprs (handle-match rool exprs))) > (setf (rool-continuep rool) t > ;; There has to be some way to set the > ;; whole to just NIL. > ;; I see the following possibilities: > ;; we parse EXPRS and remove all > ;; action clauses. > ;; We change the design of LoGS such > ;; that actions is actually (by > ;; default) (block NIL (list action-1 > ;; action-2)) and then we just do a > ;; (setf (rool-actions rool) '(return)) > (rool-actions rool) nil)))) > > (defun handle-timeout (rool exprs) > (destructuring-bind (preposition time . cdr) exprs > (case preposition > (in (setf (rool-relative-timeout rool) time)) > (at (setf (rool-timeout rool) time))) > cdr)) > > (defun handle-setenv (rool exprs) > (destructuring-bind (variable = value &rest rest) exprs > (declare (ignore =)) > (pushnew (list variable value) > (rool-environment rool) > :test > (lambda (x y) (equal (car x) (car y)))) > (if (eq (car rest) 'and) > ;; More than one variable has been > ;; defined. Save that too. > (handle-setenv rool (cdr rest)) > rest))) > > ;; add the given function to the actions list > (defun handle-do (rool exprs) > (destructuring-bind (todo &rest rest) exprs > (format t "rool: ~A~%todo: ~A~%rest: ~A~%~ > rool-actions:~S~%~50~~%" > rool todo rest (rool-actions rool)) > (setf (rool-actions rool) (append (rool-actions rool) (list (eval todo)))) > rest)) > > ;; set the continue slot to t > (defun handle-continue (rool exprs) > (setf (rool-continuep rool) t) > exprs) > > ;; set the delete slot to t > (defun handle-delete (rool exprs) > (setf (rool-delete rool) t) > exprs) > > ;;; Set aliases for keywords so we have synonyms > ;;; and define the handler function > (defmacro defhandler (keywords alias handle-fn) > `(setf ,@(if (consp keywords) > (loop as keyword in keywords > appending `((alias ',keyword) ',alias)) > `((alias ',keywords) ',alias)) > (handle-fn ',alias) ,handle-fn)) > > (defhandler (match matching) :match #'handle-match) > (defhandler (name named) :name #'handle-name) > (defhandler (do doing) :do #'handle-do) > (defhandler continue :continue #'handle-continue) > (defhandler timeout :timeout #'handle-timeout) > (defhandler (set-env with) :set-env #'handle-setenv) > (defhandler delete :delete #'handle-delete) > > ;;; *eof* > > > On 6/4/06, Jim Prewett wrote: > > > > Hi Vijay, > > > > I've had fun playing with your code today (unfortunately, some yard work, > > etc. kept me busy for too much of it :). I've made a few minor > > modifications, and generally think your code has a nice feel to it :) > > kudos! > > > > Here is an example using some of the new things I've added: > > > > (rule > > name 'foo > > match regexp ".+" > > ;; set variable var1 to 42 > > setenv var1 = 42 > > ;; first action > > doing (lambda (x) (format t "message: ~A var1: ~A~%" (message x) > > var1)) > > ;; second action > > doing #'print-message > > ;; set continuep slot to t > > continue) > > > > ;;; Paul Graham, On Lisp, p191 > > (defmacro aif (test-form then-form &optional else-form) > > `(let ((it ,test-form)) > > (if it ,then-form ,else-form))) > > > > ;;; an 'accessor' for the handle-fns > > (defmacro handle-fn (keyword) > > `(get ,keyword 'handle-fn)) > > > > ;;; Influenced by Peter Norvig's LOOP implementation > > > > (defstruct rool > > (name '()) > > (match '()) > > (timeout '()) > > (relative-timeout '()) > > (delete-rule '()) > > (continuep '()) > > (actions '()) > > (environment '())) > > > > (defmacro rule (&rest exprs) > > (let ((r (make-rool))) > > (parse-rule r exprs) > > (fill-rule-template r))) > > > > (defun fill-rule-template (rool) > > `(make-instance > > 'logs::rule > > :environment ',(rool-environment rool) > > :actions (rool-actions ,rool) > > ,@(loop as (key fn) in '((:match rool-match) > > (:name rool-name) > > (:timeout rool-timeout) > > (:continuep rool-continuep) > > (:relative-timeout rool-relative-timeout)) > > as res = (funcall fn rool) > > if res append `(,key ,res)))) > > > > (defun parse-rule (rool exprs) > > (unless (null exprs) > > (parse-rule rool (parse-keyword rool (car exprs) (cdr exprs))))) > > > > (defun parse-keyword (rool keyword exprs) > > (aif (handle-fn keyword) > > (funcall it rool exprs) > > (error "Unknown keyword ~S" keyword))) > > > > (defun handle-name (rool exprs) > > (destructuring-bind (name . cdr) exprs > > ;; rule names don't have to be symbols > > ;; symbols are (obviously) a pretty good idea, but > > ;; strings, etc. should work as well > > (if (null (rool-name rool)) > > (setf (rool-name rool) `,name) > > (error "Invalid name ~S" name)) > > cdr)) > > > > (defun handle-match (rool exprs) > > (let ((matches (rool-match rool))) > > ;; For now, we ignore. Later we add or something > > (declare (ignore matches)) > > (destructuring-bind (car . cdr) exprs > > (case car > > (regexp (let ((msg (gensym "MESSAGE")) > > (matches (gensym)) > > (sub-matches (gensym)) > > (regex (pop cdr))) > > (setf (rool-match rool) > > `(lambda (,msg) > > (multiple-value-bind (,matches ,sub-matches) > > (cl-ppcre:scan-to-strings ,regex (message > > ,msg)) > > (when ,matches > > (values t > > '((sub-matches ,sub-matches))))))) > > cdr)) > > (message-length (let ((msg (gensym "MESSAGE")) > > (relop (pop cdr)) > > (num (pop cdr))) > > (setf (rool-match rool) > > `(lambda (,msg) > > (,relop (length (message ,msg)) ,num))) > > cdr)) > > (t (setf (rool-match rool) car) cdr))))) > > > > (defun handle-timeout (rool exprs) > > (destructuring-bind (preposition . (time . cdr)) exprs > > (case preposition > > (in (setf (rool-relative-timeout rool) time)) > > (at (setf (rool-timeout rool) time))) > > cdr)) > > > > (defun handle-setenv (rool exprs) > > (destructuring-bind (variable equal value &rest rest) exprs > > (pushnew (list variable value) > > (rool-environment rool) > > :test > > (lambda (x y) (equal (car x) (car y)))) > > rest)) > > > > ;; add the given function to the actions list > > (defun handle-do (rool exprs) > > (destructuring-bind (todo &rest rest) exprs > > (format t "rool: ~A todo: ~A rest: ~A~%" > > rool todo rest) > > (append (rool-actions rool) (list (eval todo))) > > rest)) > > > > ;; set the continue slot to t > > (defun handle-continue (rool exprs) > > (setf (rool-continuep rool) t) > > exprs) > > > > (setf (handle-fn 'match) #'handle-match) > > (setf (handle-fn 'name) #'handle-name) > > (setf (handle-fn 'timeout) #'handle-timeout) > > > > ;; new > > (setf (handle-fn 'continue) #'handle-continue) > > (setf (handle-fn 'setenv) #'handle-setenv) > > (setf (handle-fn 'doing) #'handle-do) > > > > > > Jim > > > > James E. Prewett Jim@Prewett.org download@hpc.unm.edu > > Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ > > Designated Security Officer OpenPGP key: pub 1024D/31816D93 > > HPC Systems Engineer III UNM HPC 505.277.8210 > > > > On Sun, 4 Jun 2006, Vijay Lakshminarayanan wrote: > > > > > On 6/4/06, Jim Prewett wrote: > > > > > > I was wondering how you're envisioning this working for rules that > > > > create > > > > > > rules ... that create rules? > > > > > > > > > > > > (make-instance 'rule > > > > > > :actions > > > > > > (list > > > > > > (lambda (message) > > > > > > (enqueue *root-ruleset* (make-instance 'rule ...))))) > > > > > > > > > > We just use the macro again, I guess. > > > > > > > > > > (rule performing (lambda (message) (enqueue *root-ruleset* (rule > > > > ...)))) > > > > > > > > > > > > > I was hoping that was the answer :) > > > > > > Oh so it's a Good Thing? That's a relief :-) > > > > > > > > In time we should have a single function which does the enqueing. > > > > > > > > I don't think you're quite understanding the ruleset tree structure. > > > > rulesets contain rules and other rulesets. You may want to have a rule > > > > that creates a new rule on a completely different part of the tree. > > > > > > You're right. I completely forgot about this -- I was totally > > > immersed into defining the macro's behavior itself. > > > > > > > In other words, I don't see specifying the ruleset you're adding the > > rule > > > > to to be a loss. > > > > > > > > Jim > > > > > > > > > > Will come up with more and better quality code soon, Jim. > > > > > > Thanks > > > Vijay > > > > > > From MAILER-DAEMON Sun Jun 11 22:21:27 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fpc3T-0003Gc-02 for mharc-logs-devel@gnu.org; Sun, 11 Jun 2006 22:21:27 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fpc3R-0003GH-LZ for logs-devel@nongnu.org; Sun, 11 Jun 2006 22:21:25 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fpc3O-0003F4-69 for Logs-devel@nongnu.org; Sun, 11 Jun 2006 22:21:23 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fpc3N-0003Et-GC for Logs-devel@nongnu.org; Sun, 11 Jun 2006 22:21:21 -0400 Received: from [64.233.184.238] (helo=wr-out-0506.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FpcC0-00010F-7w for Logs-devel@nongnu.org; Sun, 11 Jun 2006 22:30:16 -0400 Received: by wr-out-0506.google.com with SMTP id i21so1044122wra for ; Sun, 11 Jun 2006 19:21:20 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=iOBd/tu+oWipup6ScYiaxxZ2NyXpy8K9n07TvGbT2s31cQVD5qbe1QVdZtJyzmLFnTELofJbC4ELGGMfkwYZMWDbzkVzMkNi/Dn9SxfZ5JpGulViXq1myB1wftigndX196Wbzvbi7KWvmFH4n7zyG1xrD1aV5JUvjXVC1ahu4Po= Received: by 10.64.178.2 with SMTP id a2mr4310014qbf; Sun, 11 Jun 2006 19:21:20 -0700 (PDT) Received: by 10.64.114.12 with HTTP; Sun, 11 Jun 2006 19:21:20 -0700 (PDT) Message-ID: <5dd3d61b0606111921v48300b2fsa4b32470886cee16@mail.gmail.com> Date: Sun, 11 Jun 2006 21:21:20 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" , Logs-devel@nongnu.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606091211p2e389be6n83f64bf049b116e2@mail.gmail.com> <5dd3d61b0606092153h49929599g7ba26c5612b4ff3f@mail.gmail.com> <5dd3d61b0606110708w3e5e0dd5pe2e77e82e3db81e7@mail.gmail.com> Cc: Subject: [Logs-devel] Re: LoGS roadmap(ish) X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2006 02:21:26 -0000 On 6/11/06, Jim Prewett wrote: > > > > I don't like the semantics of 'matching "abc" and "def"'; I would have > > > that mean instead 'matches regexp "abc" and matches regexp "def"'. So, a > > > matching pattern could be "abcdef" or "defabc" or "abc hello def" ... > > > > > > I can't honestly think of a time where I would want that to mean match the > > > concatenation of two (N) regexps. :) > > > > > > To make the example a little more concrete; I may want to match an SSH > > > message that indicates that user 'vijay' has logged in: 'matching "vijay" > > > and "sshd"' or some such thing. > > > > All right. I've changed it now. All regexps are OR'd. When I first > > wrote it, my only thought was: this way we can break long regexps over > > lines. So [matches regexp "abc" and matches "def"] results in > > > > "(abc)|(def)". > > I think you're closer, but I still don't think I like the behaviour. I > would expect matches "abc" and matches "def" to produce code that would > expand to something like: > > (lambda (message) > (and > (cl-ppcre:scan "abc" (message message)) > (cl-ppcre:scan "def" (message message)))) > > I'm thinking I'd want to say something like: > matches "sshd" and matches "login failed" > doing #'alert-jim-to-login-failure > This too is done :-) I was taking a look at ASDF-INSTALL. It seems complicated but it is one of those things we must do to make life easy for the masses. I myself use ASDF to load LoGS everytime (since SBCL includes it by default). If it is all right with you, shall I add a file named "cl-cli.asd" to the repository? I have one on my disk and I copy it each and every time I check out the latest version. It'll be better to just add it to the repository, IMO. Here's the file. It is a severely stripped down version of logs.asd :) ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: CL-USER; Base: 10 -*- (defsystem "cl-cli" :description "Command line processing utility" :version "0.0.1-pre" :author "James E. Prewett " :licence "GPL" :depends-on () :components ((:file "cl-cli"))) I took the version number from cl-cli-version Thanks Jim Vijay From MAILER-DAEMON Mon Jun 12 08:02:21 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fpl7c-00014a-Om for mharc-logs-devel@gnu.org; Mon, 12 Jun 2006 08:02:20 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fpl7a-00013i-LC for logs-devel@nongnu.org; Mon, 12 Jun 2006 08:02:18 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fpl7X-00011t-If for Logs-devel@nongnu.org; Mon, 12 Jun 2006 08:02:18 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fpl7X-00011h-Bi for Logs-devel@nongnu.org; Mon, 12 Jun 2006 08:02:15 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FplGF-00016s-A3 for Logs-devel@nongnu.org; Mon, 12 Jun 2006 08:11:15 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1Fpl7O-0003qY-ON; Mon, 12 Jun 2006 06:02:07 -0600 Date: Mon, 12 Jun 2006 06:02:04 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com In-Reply-To: <5dd3d61b0606111921v48300b2fsa4b32470886cee16@mail.gmail.com> Message-ID: References: <5dd3d61b0606091211p2e389be6n83f64bf049b116e2@mail.gmail.com> <5dd3d61b0606092153h49929599g7ba26c5612b4ff3f@mail.gmail.com> <5dd3d61b0606110708w3e5e0dd5pe2e77e82e3db81e7@mail.gmail.com> <5dd3d61b0606111921v48300b2fsa4b32470886cee16@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org Subject: [Logs-devel] Re: LoGS roadmap(ish) X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2006 12:02:18 -0000 > This too is done :-) I was taking a look at ASDF-INSTALL. It seems > complicated but it is one of those things we must do to make life easy > for the masses. > > I myself use ASDF to load LoGS everytime (since SBCL includes it by default). That would be great! I'm all for 'ease-of-use' type stuff. I'm becoming a bigger and bigger fan of ASDF. Its super-cool! Now, try explaining ASDF to a C programmer.... Its like 'make', but on steroids! :) Are you running Linux 2.6.x? I don't have a 2.6 system right now, so I can't run the latest & greatest SBCL (with the generate-application feature to save-lisp-and-die (or whatever it is); you can make an executeable binary file with SBCL apparently.) Maybe you could check that out for me some time and give me the scoop. Could we make LoGS.exe that we could execute ala: # LoGS.exe --ruleset /etc/LoGSruleset.lisp --file /var/log/messages ... ? That would be *sexy* :) > If it is all right with you, shall I add a file named "cl-cli.asd" to > the repository? I have one on my disk and I copy it each and every > time I check out the latest version. It'll be better to just add it > to the repository, IMO. yes! Please! (I didn't realize that I hadn't yet added an ASDF file for cl-cli!) > Here's the file. It is a severely stripped down version of logs.asd :) > > ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: CL-USER; Base: 10 -*- > (defsystem "cl-cli" > :description "Command line processing utility" > :version "0.0.1-pre" > :author "James E. Prewett " > :licence "GPL" > :depends-on () > :components ((:file "cl-cli"))) > > I took the version number from cl-cli-version I'm wondering if we should completely separate off CL-CLI: I use that code for just about everything I write (in Lisp) that takes command line arguments. :) Its not the prettiest, but I think its a useful library :) I'd also like to clean up cl-cli. Its pretty kludgy IMO right now. Just for your future reference... I consider you to be a (full-fledged) member of the LoGS crew. I trust you to continue doing good work and make decent decisions about things like adding files to the CVS repository. knock yourself out! Hopefully we'll never come to a point where you start screwing things up so badly I end up wasting all my time cleaning up your messes ;) If we do get there, trust me, I'll let you know :) I'd rather say "umm... shouldn't it be more like this... " later than stifle your creativity now :) That is a lot of why I'm such a big fan of 'unit testing'. I can make massive changes to the LoGS codebase and quickly identify anything I've broken: Between 'unit tests' and CVS, I'm free from worrying about what I'm going to break which I find allows me to be more creative. :) I've added your name to the list of LoGS contributors on my webpage (I hope that is OK; If not, I'll happily remove it). Would you like a link to somewhere (both Dave and I have a link off to our main homepage things)? Thanks, Jim From MAILER-DAEMON Tue Jun 13 02:29:54 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fq2PS-0007OO-7b for mharc-logs-devel@gnu.org; Tue, 13 Jun 2006 02:29:54 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fq2PQ-0007OD-Dl for logs-devel@nongnu.org; Tue, 13 Jun 2006 02:29:52 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fq2PM-0007Ni-Gz for Logs-devel@nongnu.org; Tue, 13 Jun 2006 02:29:51 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fq2PM-0007Nf-ET for Logs-devel@nongnu.org; Tue, 13 Jun 2006 02:29:48 -0400 Received: from [64.233.184.230] (helo=wr-out-0506.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Fq2YF-0006Dn-EK for Logs-devel@nongnu.org; Tue, 13 Jun 2006 02:38:59 -0400 Received: by wr-out-0506.google.com with SMTP id 69so1311834wra for ; Mon, 12 Jun 2006 23:29:47 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=tyB2Y+vdmv1wPYLXOj/Zwkuwbg3f8WZnEpGHwLPsilb6Z7Cuf6jaVSTUsX5LXvBUOIaRWI6JCudvAbak6qQIlhZTniB0+ZyJYBJo+OJfDYENFEJBOAp89O3yyhXfAM769+ozWU/z5n6yEZu5gWXlGfdPqbuh1xebMdSz6faFVDA= Received: by 10.65.59.4 with SMTP id m4mr2733681qbk; Mon, 12 Jun 2006 23:29:47 -0700 (PDT) Received: by 10.64.114.12 with HTTP; Mon, 12 Jun 2006 23:29:47 -0700 (PDT) Message-ID: <5dd3d61b0606122329g4c0f2831s1859b5bd9b8eeea@mail.gmail.com> Date: Tue, 13 Jun 2006 01:29:47 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" , Logs-devel@nongnu.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: [Logs-devel] rdl work X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2006 06:29:52 -0000 Hi Jim, RDL needs a lot more work. Here are the things that are _essential_ before we can even say that we have something of the sort. The way I see it, RDL must be the entire interface to LoGS. (Adding more later, while we're doing a log analysis even, also should be possible but for the most part, it should be through the RDL, IMO.) If RDL were entire and complete, our users would be able to write out their rules/rulesets/contexts all in one file, and name it: my-rules.logs and finally say: $ Run_LoGS my-rules.logs and job done (for the most part, anyway). (What the newbies wouldn't know is that my-rules.logs is actually a Common Lisp macro ;-) Here's my idea of a complete RDL: * specify rules easily. (This is not done. Try rewriting some of the rules in Parlance.lisp and you'll know what I mean.) * manipulate rulesets so we can order rules as trees and the like. * specify the files LoGS must scan. * Contexts. Where do we fit these? Do we specify them separately or along with the rules or with the rulesets? Please add more if we need more. We could have (rule named 'example saved-into (context max-lines = 12 min-lines = 10 timeout in 7 timeout at 80425 finally delete doing foo and bar)) Though written this way, the actual setting is in reverse since contexts save rule results. Maybe (context named 'c1) (rule matching "acb" saved-into c1) I don't know. What would you suggest, Jim? Likewise, rules must find ways of getting into the ruleset tree. Here we could use the fact that rules have names to our advantage, I guess. (rule named 'level-0) (rule named 'level-0.0 after 'tree-root) (rule named 'level-0.1 after 'tree-root) (rule named 'level-0.2 after 'tree-root) (rule named 'level-0.0.0 after 'level-0.0) (rule named 'level-0.2.0 after 'level-0.2) would have the following tree structure: ((level-0 ((level-0.0 ((level-0.0.0)))) ((level-0.1)) ((level-0.2 ((level-0.2.0)))))) The order of 0.0 and 0.1 would depend on the order they are specified in (the siblings are arranged in order of their `birth') These are just ideas I'm throwing around. Please kill the bad ones and see if there are any good ones. Thanks Jim Vijay btw, why isn't there any WITH-FILE-FOLLOWER macro? Or a WITH-LOG-FILE macro? (both would do the same) From MAILER-DAEMON Tue Jun 13 07:54:05 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fq7TB-0006QX-1N for mharc-logs-devel@gnu.org; Tue, 13 Jun 2006 07:54:05 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fq7T9-0006Pg-AW for logs-devel@nongnu.org; Tue, 13 Jun 2006 07:54:03 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fq7T7-0006PT-II for Logs-devel@nongnu.org; Tue, 13 Jun 2006 07:54:02 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fq7T7-0006PQ-Fd for Logs-devel@nongnu.org; Tue, 13 Jun 2006 07:54:01 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1Fq7c3-0003Sp-Gr for Logs-devel@nongnu.org; Tue, 13 Jun 2006 08:03:15 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1Fq7T4-0004ZR-QZ for Logs-devel@nongnu.org; Tue, 13 Jun 2006 05:53:59 -0600 Date: Tue, 13 Jun 2006 05:53:56 -0600 (MDT) From: Jim Prewett To: Logs-devel@nongnu.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Subject: [Logs-devel] added mainline.lisp X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2006 11:54:03 -0000 Hi all, I've added mainline.lisp to the CVS repo. The idea was to move the mainline type functions out to their own file so that when compiling LoGS, the compiler would have fewer undefined symbol warnings. I'd like to (as much as is possible) have the file that defines a symbol loaded before a function in a different file that uses that symbol is loaded. I believe this is mostly an aesthetic issue (I don't like my compiler complaining :) and shouldn't affect performance (I'm not exactly clear on when that sort of thing actually can affect performance :). You'd be surprised at how hard I had to work to get this done (I had attempted many far less elegant solutions, none of which turned out to be a very good idea at all :). Oh well! :) all tests still pass and the compiler is a little quieter with the warnings now. :) Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 From MAILER-DAEMON Tue Jun 13 11:17:18 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FqAdq-0003BZ-H4 for mharc-logs-devel@gnu.org; Tue, 13 Jun 2006 11:17:18 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FqAdo-0003AN-10 for logs-devel@nongnu.org; Tue, 13 Jun 2006 11:17:16 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FqAdm-00039b-5P for Logs-devel@nongnu.org; Tue, 13 Jun 2006 11:17:15 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FqAdm-00039V-1J for Logs-devel@nongnu.org; Tue, 13 Jun 2006 11:17:14 -0400 Received: from [64.233.184.224] (helo=wr-out-0506.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FqAmk-0006Vn-3E for Logs-devel@nongnu.org; Tue, 13 Jun 2006 11:26:30 -0400 Received: by wr-out-0506.google.com with SMTP id 69so1403149wra for ; Tue, 13 Jun 2006 08:17:13 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=HC6qofF5zlgpy2tsa9ayG4AG+JlYyjRUoW9Doj3sFF5J92mKjsVfWmCrVW5PxvewSjk3XUm4GATkf0gE2Qh8UH2kbz1lyMAc6Z66XA+lPztgLnyFZqStn8ZJmWqgXG8r1tV4YZMIldU5c8G6NLY5rabV9LCj8zoOE/rGnFk8z68= Received: by 10.64.150.16 with SMTP id x16mr3090238qbd; Tue, 13 Jun 2006 08:17:12 -0700 (PDT) Received: by 10.64.114.12 with HTTP; Tue, 13 Jun 2006 08:17:12 -0700 (PDT) Message-ID: <5dd3d61b0606130817v4aa5a450n855348c620c5277e@mail.gmail.com> Date: Tue, 13 Jun 2006 10:17:12 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] added mainline.lisp In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2006 15:17:16 -0000 Jim I'm not really sure you've updated the latest file. The mainline file I have just has two functions moved from LoGS.lisp to itself. The defvars, defparameters and defconstants are where they were. Thanks Vijay On 6/13/06, Jim Prewett wrote: > > Hi all, > > I've added mainline.lisp to the CVS repo. The idea was to move the > mainline type functions out to their own file so that when compiling LoGS, > the compiler would have fewer undefined symbol warnings. > > I'd like to (as much as is possible) have the file that defines a symbol > loaded before a function in a different file that uses that symbol is > loaded. I believe this is mostly an aesthetic issue (I don't like my > compiler complaining :) and shouldn't affect performance (I'm not exactly > clear on when that sort of thing actually can affect performance :). > > You'd be surprised at how hard I had to work to get this done (I had > attempted many far less elegant solutions, none of which turned out to be > a very good idea at all :). Oh well! :) > > all tests still pass and the compiler is a little quieter with the > warnings now. :) > > Jim > > James E. Prewett Jim@Prewett.org download@hpc.unm.edu > Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ > Designated Security Officer OpenPGP key: pub 1024D/31816D93 > HPC Systems Engineer III UNM HPC 505.277.8210 > > > _______________________________________________ > LoGS-devel mailing list > LoGS-devel@nongnu.org > http://lists.nongnu.org/mailman/listinfo/logs-devel > From MAILER-DAEMON Tue Jun 13 11:51:18 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FqBAk-0005QY-TA for mharc-logs-devel@gnu.org; Tue, 13 Jun 2006 11:51:18 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FqBAj-0005O1-Hu for logs-devel@nongnu.org; Tue, 13 Jun 2006 11:51:17 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FqBAg-0005Il-Re for Logs-devel@nongnu.org; Tue, 13 Jun 2006 11:51:16 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FqBAg-0005IW-ML for Logs-devel@nongnu.org; Tue, 13 Jun 2006 11:51:14 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FqBJe-0000vH-P7 for Logs-devel@nongnu.org; Tue, 13 Jun 2006 12:00:31 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FqBAY-0008Ef-Ib; Tue, 13 Jun 2006 09:51:07 -0600 Date: Tue, 13 Jun 2006 09:51:04 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] added mainline.lisp In-Reply-To: <5dd3d61b0606130817v4aa5a450n855348c620c5277e@mail.gmail.com> Message-ID: References: <5dd3d61b0606130817v4aa5a450n855348c620c5277e@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2006 15:51:18 -0000 Hi Vijay, Yes, that is what I intended. LoGS.lisp sets up most of the parameters and mainline.lisp is just those two functions (they should be loaded after the other classes are defined, etc. IMO) Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 On Tue, 13 Jun 2006, Vijay Lakshminarayanan wrote: > Jim > > I'm not really sure you've updated the latest file. The mainline file > I have just has two functions moved from LoGS.lisp to itself. The > defvars, defparameters and defconstants are where they were. > > Thanks > Vijay > > On 6/13/06, Jim Prewett wrote: > > > > Hi all, > > > > I've added mainline.lisp to the CVS repo. The idea was to move the > > mainline type functions out to their own file so that when compiling LoGS, > > the compiler would have fewer undefined symbol warnings. > > > > I'd like to (as much as is possible) have the file that defines a symbol > > loaded before a function in a different file that uses that symbol is > > loaded. I believe this is mostly an aesthetic issue (I don't like my > > compiler complaining :) and shouldn't affect performance (I'm not exactly > > clear on when that sort of thing actually can affect performance :). > > > > You'd be surprised at how hard I had to work to get this done (I had > > attempted many far less elegant solutions, none of which turned out to be > > a very good idea at all :). Oh well! :) > > > > all tests still pass and the compiler is a little quieter with the > > warnings now. :) > > > > Jim > > > > James E. Prewett Jim@Prewett.org download@hpc.unm.edu > > Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ > > Designated Security Officer OpenPGP key: pub 1024D/31816D93 > > HPC Systems Engineer III UNM HPC 505.277.8210 > > > > > > _______________________________________________ > > LoGS-devel mailing list > > LoGS-devel@nongnu.org > > http://lists.nongnu.org/mailman/listinfo/logs-devel > > > From MAILER-DAEMON Fri Jun 16 19:08:52 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FrNQq-0005w6-NY for mharc-logs-devel@gnu.org; Fri, 16 Jun 2006 19:08:52 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FrNQp-0005vm-8i for logs-devel@nongnu.org; Fri, 16 Jun 2006 19:08:51 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FrNQo-0005uZ-6N for Logs-devel@nongnu.org; Fri, 16 Jun 2006 19:08:50 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FrNQo-0005uS-0v for Logs-devel@nongnu.org; Fri, 16 Jun 2006 19:08:50 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FrNaW-0005ps-Ki for Logs-devel@nongnu.org; Fri, 16 Jun 2006 19:18:52 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FrNQe-0003sT-5u for Logs-devel@nongnu.org; Fri, 16 Jun 2006 17:08:42 -0600 Date: Fri, 16 Jun 2006 17:08:37 -0600 (MDT) From: Jim Prewett To: Logs-devel@nongnu.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Subject: [Logs-devel] rename Data_Sources files? X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2006 23:08:51 -0000 Hello, I'm thinking we should move files named things like File-Follower_CMUCL.low.lisp to things like File-Follower_CMUCL_low.lisp Any objections? Jim p.s. why doesn't CVS have a 'rename' feature?! James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 From MAILER-DAEMON Fri Jun 16 23:08:27 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FrRAh-0002N1-FQ for mharc-logs-devel@gnu.org; Fri, 16 Jun 2006 23:08:27 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FrRAf-0002Mr-MD for logs-devel@nongnu.org; Fri, 16 Jun 2006 23:08:25 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FrRAd-0002Mf-BB for Logs-devel@nongnu.org; Fri, 16 Jun 2006 23:08:24 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FrRAd-0002Mc-5o for Logs-devel@nongnu.org; Fri, 16 Jun 2006 23:08:23 -0400 Received: from [66.249.82.206] (helo=wx-out-0102.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FrRKO-00044W-6s for Logs-devel@nongnu.org; Fri, 16 Jun 2006 23:18:28 -0400 Received: by wx-out-0102.google.com with SMTP id s18so533794wxc for ; Fri, 16 Jun 2006 20:08:22 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Xn+xZX2g8gg7wLWY/DIe7dnV7EmNhe1GEVZ/izvf/yBIgOAuxViuiu492geSwplk8EL9bGV19z5Fo/ny5NugQEXrkBfbLrSiH4xAgZB+NIDmMfaK5Zyr86CKNVUsEa95e5yUuQ0FfuxeVDPYRdpbCkBYh8KXDHw91VlaSO1U/lY= Received: by 10.70.132.14 with SMTP id f14mr5041920wxd; Fri, 16 Jun 2006 20:08:22 -0700 (PDT) Received: by 10.70.108.15 with HTTP; Fri, 16 Jun 2006 20:08:22 -0700 (PDT) Message-ID: <5dd3d61b0606162008h5962ca79pf8e4c798c6de6c01@mail.gmail.com> Date: Fri, 16 Jun 2006 22:08:22 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] rename Data_Sources files? In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2006 03:08:25 -0000 On 6/16/06, Jim Prewett wrote: > > Hello, > > I'm thinking we should move files named things like > File-Follower_CMUCL.low.lisp to things like File-Follower_CMUCL_low.lisp > > Any objections? I was thinking that myself. I read somewhere (probably c.l.l.) that generally all implementation dependant code is written in one single file and the methods are used across the rest of the application as an API. > p.s. why doesn't CVS have a 'rename' feature?! I think you've hit the spot on one of CVS's failures :-) Thanks Jim Vijay > James E. Prewett Jim@Prewett.org download@hpc.unm.edu > Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ > Designated Security Officer OpenPGP key: pub 1024D/31816D93 > HPC Systems Engineer III UNM HPC 505.277.8210 > > > _______________________________________________ > LoGS-devel mailing list > LoGS-devel@nongnu.org > http://lists.nongnu.org/mailman/listinfo/logs-devel > From MAILER-DAEMON Sat Jun 17 07:58:28 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FrZRc-0000CA-5f for mharc-logs-devel@gnu.org; Sat, 17 Jun 2006 07:58:28 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FrZRa-00009D-G9 for logs-devel@nongnu.org; Sat, 17 Jun 2006 07:58:26 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FrZRY-000070-Ps for Logs-devel@nongnu.org; Sat, 17 Jun 2006 07:58:25 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FrZRY-00006m-JK for Logs-devel@nongnu.org; Sat, 17 Jun 2006 07:58:24 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FrZbO-0001rv-ID for Logs-devel@nongnu.org; Sat, 17 Jun 2006 08:08:34 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FrZRW-0001kn-KJ; Sat, 17 Jun 2006 05:58:22 -0600 Date: Sat, 17 Jun 2006 05:56:59 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rename Data_Sources files? In-Reply-To: <5dd3d61b0606162008h5962ca79pf8e4c798c6de6c01@mail.gmail.com> Message-ID: References: <5dd3d61b0606162008h5962ca79pf8e4c798c6de6c01@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2006 11:58:26 -0000 > > I'm thinking we should move files named things like > > File-Follower_CMUCL.low.lisp to things like File-Follower_CMUCL_low.lisp > > > > Any objections? > > I was thinking that myself. I read somewhere (probably c.l.l.) that > generally all implementation dependant code is written in one single > file and the methods are used across the rest of the application as an > API. Right, thats generally what I've been trying to do. :) I'm mostly trying to fix the problem that Suresh Madhu pointed out with regaurds to using defsystem (defsystem apparently doesn't like files with two '.' characters in their names). > > p.s. why doesn't CVS have a 'rename' feature?! > > I think you've hit the spot on one of CVS's failures :-) Yes, most definantly! :) I've been looking at the possible solutions to this 'problem' (renaming in CVS) and think that the one in which original-filename is removed from the repository and new-filename is then added to the repository with a note that it used to be called original-filename. The other alternatives sound significantly more time intensive and error-prone and still have their draw-backs! Do you have any opinions? (now is the time to say 'oh, please don't do *THAT*!' ;) Jim (the laptop slayer) From MAILER-DAEMON Sat Jun 17 21:26:28 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Frm3Y-0008Ny-1l for mharc-logs-devel@gnu.org; Sat, 17 Jun 2006 21:26:28 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Frm3V-0008Ky-UP for logs-devel@nongnu.org; Sat, 17 Jun 2006 21:26:26 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Frm3U-0008Il-8Z for Logs-devel@nongnu.org; Sat, 17 Jun 2006 21:26:25 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Frm3U-0008IW-3M for Logs-devel@nongnu.org; Sat, 17 Jun 2006 21:26:24 -0400 Received: from [66.249.82.204] (helo=wx-out-0102.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FrmDS-0005Xp-4Y for Logs-devel@nongnu.org; Sat, 17 Jun 2006 21:36:42 -0400 Received: by wx-out-0102.google.com with SMTP id s18so631479wxc for ; Sat, 17 Jun 2006 18:26:23 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=jObBCxT++8DBzv7XfxxFoc9qUTAofPMQ1zylfmzH9OXTigLAtoRBYTSndGrYs2CZ/ox9YG2bf6FhekqeZ4kRutO8mh/YinqXoAXCHw9bVVHKQ9fh6svzq2XkyfRoKF2LMWr3s4vsH15/71yW09Apcr1cgvKLdGdTH2YEZP3JOhM= Received: by 10.70.40.11 with SMTP id n11mr6379535wxn; Sat, 17 Jun 2006 18:26:23 -0700 (PDT) Received: by 10.70.108.15 with HTTP; Sat, 17 Jun 2006 18:26:23 -0700 (PDT) Message-ID: <5dd3d61b0606171826o577c5062n291cbd1d4c6233ab@mail.gmail.com> Date: Sat, 17 Jun 2006 20:26:23 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] rename Data_Sources files? In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606162008h5962ca79pf8e4c798c6de6c01@mail.gmail.com> Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Jun 2006 01:26:26 -0000 On 6/17/06, Jim Prewett wrote: > I'm mostly trying to fix the problem that Suresh Madhu pointed out with > regaurds to using defsystem (defsystem apparently doesn't like files with > two '.' characters in their names). > > > > p.s. why doesn't CVS have a 'rename' feature?! > > > > I think you've hit the spot on one of CVS's failures :-) > > Yes, most definantly! :) > > I've been looking at the possible solutions to this 'problem' (renaming in > CVS) and think that the one in which original-filename is removed from the > repository and new-filename is then added to the repository with a note > that it used to be called original-filename. The other alternatives sound > significantly more time intensive and error-prone and still have their > draw-backs! > > Do you have any opinions? (now is the time to say 'oh, please don't do > *THAT*!' ;) It is a big of a snag that we don't have version control for renamed files. Here's a solution I think is effective if not good. We don't delete the old .low.lisp files. Let them remain. We move all the necessary code to a new file (impl-depend.lisp?). In logs.asd and everywhere else we just pretend those files don't exist and don't bother compiling them. This will lead to those files downloading all the time but let that be. If and when we find a good working solution to the problem and see that those files are really obsolete, we could delete them. > Jim (the laptop slayer) What? Another? From MAILER-DAEMON Sun Jun 18 07:45:40 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Frvim-0000Cb-82 for mharc-logs-devel@gnu.org; Sun, 18 Jun 2006 07:45:40 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Frvik-0000C8-Fa for logs-devel@nongnu.org; Sun, 18 Jun 2006 07:45:38 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Frvih-0000Bj-N8 for Logs-devel@nongnu.org; Sun, 18 Jun 2006 07:45:38 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Frvih-0000Bg-Ho for Logs-devel@nongnu.org; Sun, 18 Jun 2006 07:45:35 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1Frvsl-0001eL-Da for Logs-devel@nongnu.org; Sun, 18 Jun 2006 07:55:59 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FrviZ-0004eN-IZ; Sun, 18 Jun 2006 05:45:27 -0600 Date: Sun, 18 Jun 2006 05:45:24 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rename Data_Sources files? In-Reply-To: <5dd3d61b0606171826o577c5062n291cbd1d4c6233ab@mail.gmail.com> Message-ID: References: <5dd3d61b0606162008h5962ca79pf8e4c798c6de6c01@mail.gmail.com> <5dd3d61b0606171826o577c5062n291cbd1d4c6233ab@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Jun 2006 11:45:38 -0000 > It is a big of a snag that we don't have version control for renamed > files. Here's a solution I think is effective if not good. We don't > delete the old .low.lisp files. Let them remain. We move all the > necessary code to a new file (impl-depend.lisp?). In logs.asd and > everywhere else we just pretend those files don't exist and don't > bother compiling them. This will lead to those files downloading all > the time but let that be. If and when we find a good working solution > to the problem and see that those files are really obsolete, we could > delete them. That sounds like a relatively good solution. Lets do that :) > > Jim (the laptop slayer) > > What? Another? yep! I'm getting good at this. To be fair, its only been since I met you ;) Jim From MAILER-DAEMON Mon Jun 19 09:07:35 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FsJTb-0002qX-BA for mharc-logs-devel@gnu.org; Mon, 19 Jun 2006 09:07:35 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FsJTZ-0002n0-L3 for logs-devel@nongnu.org; Mon, 19 Jun 2006 09:07:33 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FsJTY-0002js-DG for Logs-devel@nongnu.org; Mon, 19 Jun 2006 09:07:32 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FsJTY-0002jT-85 for Logs-devel@nongnu.org; Mon, 19 Jun 2006 09:07:32 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FsJdq-00062A-GK for Logs-devel@nongnu.org; Mon, 19 Jun 2006 09:18:10 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FsJTP-0000Tw-UE; Mon, 19 Jun 2006 07:07:24 -0600 Date: Mon, 19 Jun 2006 07:07:19 -0600 (MDT) From: Jim Prewett To: l.iyer.vijay@gmail.com Subject: Re: [Logs-devel] rename Data_Sources files? In-Reply-To: <5dd3d61b0606171826o577c5062n291cbd1d4c6233ab@mail.gmail.com> Message-ID: References: <5dd3d61b0606162008h5962ca79pf8e4c798c6de6c01@mail.gmail.com> <5dd3d61b0606171826o577c5062n291cbd1d4c6233ab@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Suresh Madhu , Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 13:07:34 -0000 Hi Vijay, I just copied those files from File-Follower_IMPL.low.lisp to File-Follower_IMPL_low.lisp so as to resolve the problem with Defsystem + LoGS that Suresh Madhu pointed out (Thanks Suresh!). I modified the asdf and load-LoGS.lisp files to reflect the new names. Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 On Sat, 17 Jun 2006, Vijay Lakshminarayanan wrote: > On 6/17/06, Jim Prewett wrote: > > I'm mostly trying to fix the problem that Suresh Madhu pointed out with > > regaurds to using defsystem (defsystem apparently doesn't like files with > > two '.' characters in their names). > > > > > > p.s. why doesn't CVS have a 'rename' feature?! > > > > > > I think you've hit the spot on one of CVS's failures :-) > > > > Yes, most definantly! :) > > > > I've been looking at the possible solutions to this 'problem' (renaming in > > CVS) and think that the one in which original-filename is removed from the > > repository and new-filename is then added to the repository with a note > > that it used to be called original-filename. The other alternatives sound > > significantly more time intensive and error-prone and still have their > > draw-backs! > > > > Do you have any opinions? (now is the time to say 'oh, please don't do > > *THAT*!' ;) > > It is a big of a snag that we don't have version control for renamed > files. Here's a solution I think is effective if not good. We don't > delete the old .low.lisp files. Let them remain. We move all the > necessary code to a new file (impl-depend.lisp?). In logs.asd and > everywhere else we just pretend those files don't exist and don't > bother compiling them. This will lead to those files downloading all > the time but let that be. If and when we find a good working solution > to the problem and see that those files are really obsolete, we could > delete them. > > > Jim (the laptop slayer) > > What? Another? > From MAILER-DAEMON Mon Jun 19 22:44:11 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FsWDr-0007Dy-TV for mharc-logs-devel@gnu.org; Mon, 19 Jun 2006 22:44:11 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FsWDr-0007Ds-Ag for logs-devel@nongnu.org; Mon, 19 Jun 2006 22:44:11 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FsWDp-0007DY-RF for Logs-devel@nongnu.org; Mon, 19 Jun 2006 22:44:11 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FsWDp-0007DV-Cj for Logs-devel@nongnu.org; Mon, 19 Jun 2006 22:44:09 -0400 Received: from [66.249.82.201] (helo=wx-out-0102.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FsWOG-0001q5-0n for Logs-devel@nongnu.org; Mon, 19 Jun 2006 22:54:56 -0400 Received: by wx-out-0102.google.com with SMTP id s8so338756wxc for ; Mon, 19 Jun 2006 19:44:08 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=qQ5EIZU4RpqeLcqRhzW7KDclDLFjRTeyw3525ZLA2S8hgHWun1sLJ+pCGnHV42ArvG0ZuJ6DqLFUJk8SZ1ulgNemxG8cmr1q7ISFmjwM0rAyr2m0aWYh6BrfqysctM0Yul0pFYv8h1dN8Sg476QUq4asdZJxtiMEu9ofgm2j1Ww= Received: by 10.70.100.8 with SMTP id x8mr9596516wxb; Mon, 19 Jun 2006 19:44:08 -0700 (PDT) Received: by 10.70.108.15 with HTTP; Mon, 19 Jun 2006 19:44:08 -0700 (PDT) Message-ID: <5dd3d61b0606191944k11289bcg3c2569e074e0724c@mail.gmail.com> Date: Mon, 19 Jun 2006 21:44:08 -0500 From: "Vijay Lakshminarayanan" To: Logs-devel@nongnu.org, "Jim Prewett" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: [Logs-devel] rdl on hold X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 02:44:11 -0000 Jim, I have temporarily put RDL on hold. I am going to finally get my hands dirty in plain LoGS before thinking of a solid RDL. I'm going to write a complete set of rules/contexts/rulesets (I'll explain my idea later), identify the patterns and abstract everything out. As I see it, we use only one file to describe the rules for a log analysis. And what I'd like is for our RDL to encompass that whole file rather than just make rule description easy. It must be able to manipulate the ruleset, contexts and create other rules and a large set of options to do stuff. In short, a LoGS dialect. Here is what I plan to write. I will need a lot of help from your side often. I'll ask when I get stuck (after I start, that is). I'm no sysadmin and I don't know what kind of messages the kernel spits out. So I wrote a root-rule which reads a log-line and based on the contents, creates a rule that scans for the regex that describes that line. Say the line is: Jun 1 02:08:02 balrog kernel: [4294667.296000] BIOS-provided physical RAM map: The regexp generated would be: "\\[\\d+\\.\\d\\] BIOS-provided physical RAM map" I ignore the first few words until "kernel:" because I alreayd know that all lines start with that and I ignore anything withing parens/brackets/braces With that regexp, I'd create a new rule which scans for this line, saves the matched message into a context (this new context is also created) and this rule has (:continuep 'NIL) As a better example for the regexp/rule generator would be: Jun 1 02:08:02 balrog kernel: [4294667.296000] BIOS-e820: 000000003f7d3800 - 0000000040000000 (reserved) \[\d+\.\d\] BIOS-e\d+: \d+f\d+d\d+ - \d+ \([a-z]+\) All numbers change to \d+ and all values within parens are generalized and escaped. After doing this whole thing for some time, the idea is that I dump the rules and contexts out to one or many files (I haven't decided). All this is just an idea I've had. I think this can be quite easily done with LoGS. I know it has several flaws like similar messages will probably end up in different contexts and so will messages that are actually related and should be in a context together. However, I believe it does have its uses in finding a class of messages to later look for in a log file or give me an idea. Any thoughts? Suggestions? Comments? I'll keep updating you with how things go. Thanks Jim Vijay From MAILER-DAEMON Sun Jun 25 11:30:05 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FuWYn-0007JR-D3 for mharc-logs-devel@gnu.org; Sun, 25 Jun 2006 11:30:05 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FuWYl-0007JE-8n for logs-devel@nongnu.org; Sun, 25 Jun 2006 11:30:03 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FuWYf-0007Iy-Hn for Logs-devel@nongnu.org; Sun, 25 Jun 2006 11:30:02 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FuWYf-0007Iv-Cn for Logs-devel@nongnu.org; Sun, 25 Jun 2006 11:29:57 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FuWkL-0002qy-8v for Logs-devel@nongnu.org; Sun, 25 Jun 2006 11:42:01 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FuWYX-0007A2-MB for Logs-devel@nongnu.org; Sun, 25 Jun 2006 09:29:50 -0600 Date: Sun, 25 Jun 2006 09:29:47 -0600 (MDT) From: Jim Prewett To: Logs-devel@nongnu.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Subject: [Logs-devel] cl-cli proposal -- defopt X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2006 15:30:03 -0000 Hello, I'm thinking a more elegant way to specify a command line option might be a macro I'm working on called defopt. Here is an example usage: (defopt ;; the name of the flag is "file" "file" ;; the flag takes a filename parameter and optional position ;; parameter (filename &optional position) ;; expression (with parameters bound) ;; to set up things properly for that flag ;; (this is the body of the lambda function) (let ((ff (make-instance 'org.prewett.LoGS::File-Follower :FileName filename))) ;; if position is specified, start there (when position (when (not ff) (error "no ff~%")) (org.prewett.LoGS::set-file-follower-position ff (read-from-string position))) (push ff *file-list*)) ;; a list of (optional) aliases for this flag :aliases '("f") ;; the description of this flag, used in the help text :description "file to read from") Does this look somewhat reasonable? I'm thinking that by specifying an option this way, the user only has to specify the argument list once, and in a much more lispy way IMO. Thanks, Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 From MAILER-DAEMON Sun Jun 25 12:13:10 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FuXEU-0000X7-7L for mharc-logs-devel@gnu.org; Sun, 25 Jun 2006 12:13:10 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FuXES-0000WR-O6 for logs-devel@nongnu.org; Sun, 25 Jun 2006 12:13:08 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FuXER-0000VL-KL for Logs-devel@nongnu.org; Sun, 25 Jun 2006 12:13:08 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FuXER-0000V5-Fc for Logs-devel@nongnu.org; Sun, 25 Jun 2006 12:13:07 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FuXQ7-0006UZ-Hk for Logs-devel@nongnu.org; Sun, 25 Jun 2006 12:25:11 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FuXEP-0007Tp-Fl for Logs-devel@nongnu.org; Sun, 25 Jun 2006 10:13:05 -0600 Date: Sun, 25 Jun 2006 10:13:03 -0600 (MDT) From: Jim Prewett To: Logs-devel@nongnu.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: Subject: [Logs-devel] options under DOS/Windoze? X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2006 16:13:09 -0000 Hi All, In working on CL-CLI, I'm wondering if we should try to support DOS-style options as well as UNIX style options. I think that DOS uses /'s where UNIX would use -'s. Does anyone know if that is right? I haven't used M$ anything in a long time :) Are there any other types of options that should be covered? I'd like CL-CLI to be useful outside of the context of LoGS. Also, I'd like LoGS to be useful outside of the UNIX world (despite me not wanting to go outside the UNIX world, unless you count Plan 9 or the HURD as non-UNIX :) Jim James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210 From MAILER-DAEMON Sun Jun 25 19:53:14 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FuePh-000745-Vh for mharc-logs-devel@gnu.org; Sun, 25 Jun 2006 19:53:14 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FuePg-00073s-Rm for logs-devel@nongnu.org; Sun, 25 Jun 2006 19:53:12 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FuePf-00073g-6v for Logs-devel@nongnu.org; Sun, 25 Jun 2006 19:53:11 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FuePf-00073d-1i for Logs-devel@nongnu.org; Sun, 25 Jun 2006 19:53:11 -0400 Received: from [66.249.82.197] (helo=wx-out-0102.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1FuebP-0005Bn-N7 for Logs-devel@nongnu.org; Sun, 25 Jun 2006 20:05:19 -0400 Received: by wx-out-0102.google.com with SMTP id t4so766493wxc for ; Sun, 25 Jun 2006 16:53:09 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=b3MazDfjBlpAXFZWFl2zvmjATUc1b+Kxqz4bJpVXgtTBzefZw0+E9Wcy/VJpB/e+Pbp3vJo0GC67eYzJ4XNn87fj8qL8lLAzr0eGogsLFGB2O/qE5UK95d9TvY7pmiCJ2c3ONjDoUxvq1L+SGFf7w0T+1j132ba1RnH7IfmN4do= Received: by 10.70.78.5 with SMTP id a5mr7692062wxb; Sun, 25 Jun 2006 16:53:09 -0700 (PDT) Received: by 10.70.94.19 with HTTP; Sun, 25 Jun 2006 16:53:09 -0700 (PDT) Message-ID: <5dd3d61b0606251653l43578571pce288df9dd38a3c7@mail.gmail.com> Date: Sun, 25 Jun 2006 18:53:09 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] cl-cli proposal -- defopt In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jun 2006 23:53:13 -0000 On 6/25/06, Jim Prewett wrote: > > Hello, > > I'm thinking a more elegant way to specify a command line option might be > a macro I'm working on called defopt. Here is an example usage: > > (defopt > ;; the name of the flag is "file" > "file" > ;; the flag takes a filename parameter and optional position > ;; parameter > (filename &optional position) > ;; expression (with parameters bound) > ;; to set up things properly for that flag > ;; (this is the body of the lambda function) > (let ((ff > (make-instance > 'org.prewett.LoGS::File-Follower > :FileName > filename))) > ;; if position is specified, start there > (when position > (when (not ff) (error "no ff~%")) > (org.prewett.LoGS::set-file-follower-position > ff > (read-from-string position))) > (push ff *file-list*)) > ;; a list of (optional) aliases for this flag > :aliases '("f") > ;; the description of this flag, used in the help text > :description "file to read from") > > Does this look somewhat reasonable? > > I'm thinking that by specifying an option this way, the user only has to > specify the argument list once, and in a much more lispy way IMO. It is better than the old scheme :-) but I don't like many things. Also, why not have defopt accept the parameters as keyword arguments? It makes things clearer and reduces the need for comments. Something like: (defopt :name "file" :alias '("f" "ff") :params (filename &optional position) :action (list filename position)) There are several problems with :action First, you cannot have recursive functions. All actions must start with an explicit progn or a let or block. Take the case where an action must start with a DECLARE form. Also we cannot reuse functions with this syntax. I think we should keep the original syntax but instead of strings use sexps. It doesn't matter that we are forced to repeat because of the benefits and freedoms we get as a consequence. Here are a few examples: :parameters (one two &optional three) :actions #'list :parameters (one two three) :actions (lambda (&rest args) Here's an implementation of option-lengths that uses symbols: (defun option-lengths (option) (let ((min-args 0) ; minimum arguments to this flag (max-args 0) ; maximum arguments to this flag (seen-optional-arg nil) ; has an optional argument been seen? (multiargs nil) ; are an infinite number of arguments possible? ) (flet ((specialp (arg) (and (symbolp arg) (char= (elt (symbol-name arg) 0) #\&))) (special (arg) (let ((arg (intern (symbol-name arg)))) (ecase arg (&optional (setq seen-optional-arg t)) (&rest (setq multiargs t)) ((&body &key) (error "Cannot support ~a for command line." arg)))))) (loop as arg in (arguments option) if (specialp arg) do (special arg) else do (incf max-args) and unless (or seen-optional-arg multiargs) do (incf min-args) finally (return (if (or multiargs seen-optional-arg) (values min-args max-args) (values min-args ()))))))) While writing this, it occured to me that you could very easily parse the options that took single options using: (destructuring-bind (&key foo bar baz) (get-application-args) &body) IMHO there will be places where using a different lambda-list for the action and the explicit arguments might make life easier for the programmer. At the same time, as a lisper, I do think life can be made simpler with having to specify the arglist only in one place since that is the majority of how anyone would like it. Duplicating the code is also error-prone, I admit. But I would recommend that we keep the arguments and actions parameters independant of each other. > Jim Thanks Vijay From MAILER-DAEMON Mon Jun 26 13:25:11 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fuupj-0002MI-HH for mharc-logs-devel@gnu.org; Mon, 26 Jun 2006 13:25:11 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fuuph-0002Lw-VG for logs-devel@nongnu.org; Mon, 26 Jun 2006 13:25:09 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fuuph-0002La-BH for Logs-devel@nongnu.org; Mon, 26 Jun 2006 13:25:09 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fuuph-0002LX-5s for Logs-devel@nongnu.org; Mon, 26 Jun 2006 13:25:09 -0400 Received: from [66.249.82.206] (helo=wx-out-0102.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Fuv1c-0000SX-3K for Logs-devel@nongnu.org; Mon, 26 Jun 2006 13:37:28 -0400 Received: by wx-out-0102.google.com with SMTP id t4so864340wxc for ; Mon, 26 Jun 2006 10:25:08 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=kRH6tcux22TDUXhtaXnQ9yoPrLosJn8JEk4YAI4PfX4d9tOMzFrM2M29Rffv1DgmDV1V6tSYrWnHYi6dLhDHzc2lCoDtzgUXe6SGkHFimFem8k+fEVa39FSy7CkfvQRmIYdV+VwSfC7ne7zYRRZCYP1ikn8vM3l33t2JIgmGnFs= Received: by 10.70.48.17 with SMTP id v17mr8853701wxv; Mon, 26 Jun 2006 10:25:07 -0700 (PDT) Received: by 10.70.94.19 with HTTP; Mon, 26 Jun 2006 10:25:07 -0700 (PDT) Message-ID: <5dd3d61b0606261025x4bf285f1n1dcf4911341b1f4a@mail.gmail.com> Date: Mon, 26 Jun 2006 12:25:07 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] cl-cli proposal -- defopt In-Reply-To: <5dd3d61b0606251653l43578571pce288df9dd38a3c7@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606251653l43578571pce288df9dd38a3c7@mail.gmail.com> Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jun 2006 17:25:10 -0000 Hi Jim I also think that PROCESS-OPTIONS is better written as (defun process-options (options-list args) (declare (ignorable options-list)) (macrolet ((nextarg () '(loop with seenflag = 'nil as arg in args if (not seenflag) do (setq seenflag t) else if (optionp arg) do (loop-finish) collect (pop args)))) (loop initially (loop as cdarg on args ;; Removes the elements that don't start ;; with an option. So ;; ("lisp" "LoGS.lisp" "--help") ;; will become ("--help") until (optionp (car cdarg)) finally (setq args cdarg)) as nextopt = (nextarg) while nextopt collect nextopt))) (defun optionp (string) "Is STRING an option?" (or (starts-with "-" string) (starts-with "--" string))) (defun starts-with-p (string1 string2 &key (test #'string=)) "Does STRING1 start with STRING2?" (and (>= (length string1) (length string2)) (funcall test string1 string2 :end1 (length string2)))) The macrolet is so simple now that maybe we could just make it a flet. I really love the LOOP macro :-) Thanks Jim Vijay On 6/25/06, Vijay Lakshminarayanan wrote: > On 6/25/06, Jim Prewett wrote: > > > > Hello, > > > > I'm thinking a more elegant way to specify a command line option might be > > a macro I'm working on called defopt. Here is an example usage: > > > > (defopt > > ;; the name of the flag is "file" > > "file" > > ;; the flag takes a filename parameter and optional position > > ;; parameter > > (filename &optional position) > > ;; expression (with parameters bound) > > ;; to set up things properly for that flag > > ;; (this is the body of the lambda function) > > (let ((ff > > (make-instance > > 'org.prewett.LoGS::File-Follower > > :FileName > > filename))) > > ;; if position is specified, start there > > (when position > > (when (not ff) (error "no ff~%")) > > (org.prewett.LoGS::set-file-follower-position > > ff > > (read-from-string position))) > > (push ff *file-list*)) > > ;; a list of (optional) aliases for this flag > > :aliases '("f") > > ;; the description of this flag, used in the help text > > :description "file to read from") > > > > Does this look somewhat reasonable? > > > > I'm thinking that by specifying an option this way, the user only has to > > specify the argument list once, and in a much more lispy way IMO. > > It is better than the old scheme :-) but I don't like many things. > > Also, why not have defopt accept the parameters as keyword arguments? > It makes things clearer and reduces the need for comments. > > Something like: > (defopt :name "file" :alias '("f" "ff") > :params (filename &optional position) > :action (list filename position)) > > There are several problems with :action > > First, you cannot have recursive functions. All actions must start > with an explicit progn or a let or block. Take the case where an > action must start with a DECLARE form. > > Also we cannot reuse functions with this syntax. I think we should > keep the original syntax but instead of strings use sexps. It doesn't > matter that we are forced to repeat because of the benefits and > freedoms we get as a consequence. > > Here are a few examples: > > :parameters (one two &optional three) > :actions #'list > > :parameters (one two three) > :actions (lambda (&rest args) > > Here's an implementation of option-lengths that uses symbols: > > (defun option-lengths (option) > (let ((min-args 0) ; minimum arguments to this flag > (max-args 0) ; maximum arguments to this flag > (seen-optional-arg nil) ; has an optional argument been seen? > (multiargs nil) ; are an infinite number of arguments possible? > ) > (flet ((specialp (arg) > (and (symbolp arg) (char= (elt (symbol-name arg) 0) #\&))) > (special (arg) > (let ((arg (intern (symbol-name arg)))) > (ecase arg > (&optional (setq seen-optional-arg t)) > (&rest (setq multiargs t)) > ((&body &key) > (error "Cannot support ~a for command line." arg)))))) > (loop as arg in (arguments option) > if (specialp arg) > do (special arg) > else > do (incf max-args) > and unless (or seen-optional-arg multiargs) > do (incf min-args) > finally (return (if (or multiargs seen-optional-arg) > (values min-args max-args) > (values min-args ()))))))) > > While writing this, it occured to me that you could very easily parse > the options that took single options using: > (destructuring-bind (&key foo bar baz) (get-application-args) > &body) > > IMHO there will be places where using a different lambda-list for the > action and the explicit arguments might make life easier for the > programmer. At the same time, as a lisper, I do think life can be > made simpler with having to specify the arglist only in one place > since that is the majority of how anyone would like it. Duplicating > the code is also error-prone, I admit. > > But I would recommend that we keep the arguments and actions > parameters independant of each other. > > > Jim > Thanks > Vijay > From MAILER-DAEMON Mon Jun 26 20:36:26 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Fv1Z4-0001Dg-7Z for mharc-logs-devel@gnu.org; Mon, 26 Jun 2006 20:36:26 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Fv1Z2-0001Cn-Dq for logs-devel@nongnu.org; Mon, 26 Jun 2006 20:36:24 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Fv1Yz-0001Bk-H6 for Logs-devel@nongnu.org; Mon, 26 Jun 2006 20:36:24 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Fv1Yz-0001Bf-Bz for Logs-devel@nongnu.org; Mon, 26 Jun 2006 20:36:21 -0400 Received: from [66.249.82.207] (helo=wx-out-0102.google.com) by monty-python.gnu.org with esmtp (Exim 4.52) id 1Fv1ky-0001Rz-Bl for Logs-devel@nongnu.org; Mon, 26 Jun 2006 20:48:44 -0400 Received: by wx-out-0102.google.com with SMTP id t4so914318wxc for ; Mon, 26 Jun 2006 17:36:20 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=qeYM5IqHBiuJR64ZZ2ZBDqle9MlfySWAAzhVHFkLLVpMElbP4/CTNjjacZrJp023CHBC1XRnsc9mHBzaLT6RBfMy9vUlkp07yM4p9+gbSjiXzpaMOelkVQSrPGLX8tFCYmX9/c4VkxgESpEoEMru11oMi93YgBBBbeRmTk/Sn7g= Received: by 10.70.71.9 with SMTP id t9mr9443600wxa; Mon, 26 Jun 2006 17:36:19 -0700 (PDT) Received: by 10.70.94.19 with HTTP; Mon, 26 Jun 2006 17:36:19 -0700 (PDT) Message-ID: <5dd3d61b0606261736j4a5480bdxd4b9a5658a52d678@mail.gmail.com> Date: Mon, 26 Jun 2006 19:36:19 -0500 From: "Vijay Lakshminarayanan" To: "Jim Prewett" Subject: Re: [Logs-devel] cl-cli proposal -- defopt In-Reply-To: <5dd3d61b0606261025x4bf285f1n1dcf4911341b1f4a@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <5dd3d61b0606251653l43578571pce288df9dd38a3c7@mail.gmail.com> <5dd3d61b0606261025x4bf285f1n1dcf4911341b1f4a@mail.gmail.com> Cc: Logs-devel@nongnu.org X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: l.iyer.vijay@gmail.com List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2006 00:36:24 -0000 Hi Jim, Here are the modified cl-cli functions: ;; process all options given on the command line (defun process-options (options-list args) (declare (ignorable options-list)) (flet ((nextarg () (loop with seenflag = 'nil as arg in args if (not seenflag) do (setq seenflag t) else if (optionp arg) do (loop-finish) collect (pop args)))) (loop as cdr on args until (optionp (car cdr)) finally (setq args cdr)) (loop as nextopt = (nextarg) while nextopt do (process-switch nextopt options-list)))) (defun optionp (string) "Is STRING a command line option?" (or (starts-with-p string "--") (starts-with-p string "-"))) (defun starts-with-p (string1 string2 &key (test #'string=)) "Does STRING1 start with STRING2?" (and (>= (length string1) (length string2)) (funcall test string1 string2 :end1 (length string2)))) (defun same-name-p (string name) "Is NAME the same as STRING in terms of command line options?" (or (equal name (subseq string 1)) (equal name (subseq string 2)))) ;; process an individual switch (defun process-switch (list options-list) (let* ((switch (pop list)) (opt (car (member switch options-list :test #'(lambda (a b) (same-name-p a (name b))))))) (when opt (let ((numargs (length list))) (multiple-value-bind (minargs maxargs) (option-lengths opt) (cond ((< numargs minargs) (error "too few arguments for flag: ~A expecting ~A to ~A, given ~A~%" (name opt) minargs maxargs numargs)) ((and (numberp maxargs) (> numargs maxargs)) (error "too many arguments for flag: ~A expecting ~A to ~A, given ~A~%" (name opt) minargs maxargs numargs)) ((action opt) (apply (action opt) list)))))))) ;; pull the next flag & its args off the args list (defun option-lengths (option) (let ((min-args 0) ; minimum arguments to this flag (max-args 0) ; maximum arguments to this flag (seen-optional-arg nil) ; has an optional argument been seen? (multiargs nil) ; are an infinite number of arguments possible? ) (flet ((specialp (arg) (and (symbolp arg) (char= (elt (symbol-name arg) 0) #\&))) (special (arg) (let ((arg (intern (symbol-name arg)))) (ecase arg (&optional (setq seen-optional-arg t)) (&rest (setq multiargs t)) ((&body &key) (error "Cannot support ~a for command line." arg)))))) (loop as arg in (arguments option) if (specialp arg) do (special arg) else do (incf max-args) and unless (or seen-optional-arg multiargs) do (incf min-args) finally (return (if multiargs (values min-args ()) (values min-args max-args))))))) tell me if you'd rather have the diff file. I tested it using SBCL on my machine (sbcl allows #! scripts) Here are the contents of my cli.lisp file: #!/usr/bin/sbcl --noinform (require 'cl-cli) (use-package :cl-cli) (defun print-file (filename) (with-open-file (file filename) (loop as line = (read-line file nil nil) while line do (format *standard-output* "~a~%" line)))) (defvar *opts* NIL) (defmacro defopt (&key name arguments action description) (let ((opt (gensym))) `(let ((,opt (make-instance 'cli-opt :name ,name :arguments ',arguments :action ,action :description ,description))) (setq *opts* (nconc *opts* (list ,opt))) ,opt))) (defopt :name "sum" :arguments (&rest args) :action #'(lambda (&rest args) (loop as i in args summing (parse-integer i) into sum finally (format *standard-output* "~a~%" sum))) :description "Adds the arguments it receives") (defopt :name "cat" :arguments (filename) :action #'print-file :description "Does the equivalent of the UNIX `cat' command.") (eval-when (:execute) (let ((args (cl-cli:get-application-args))) (cl-cli:process-options *opts* args))) ;;;;; And here is my bash interactions: [vijay@balrog foo]$ ./cli.lisp --sum 12 23 34 45 114 [vijay@balrog foo]$ ./cli.lisp --sum 10 12 --cat ~/.sbclrc 22 ;;; -*- Mode: Lisp; -*- ;;; If the first user-processable command-line argument is a filename, ;;; disable the debugger, load the file handling shebang-line and quit. (let ((script (and (second *posix-argv*) (probe-file (second *posix-argv*))))) (when script ;; Handle shebang-line (set-dispatch-macro-character #\# #\! (lambda (stream char arg) (declare (ignore char arg)) (read-line stream))) ;; Disable debugger (setf *invoke-debugger-hook* (lambda (condition hook) (declare (ignore hook)) ;; Uncomment to get backtraces on errors ;; (sb-debug:backtrace 20) (format *error-output* "Error: ~A~%" condition) (quit))) (load script) (quit))) (require '#:asdf) [vijay@balrog foo]$ Thanks Vijay On 6/26/06, Vijay Lakshminarayanan wrote: > Hi Jim > > I also think that PROCESS-OPTIONS is better written as > > (defun process-options (options-list args) > (declare (ignorable options-list)) > (macrolet ((nextarg () > '(loop with seenflag = 'nil > as arg in args > if (not seenflag) > do (setq seenflag t) > else if (optionp arg) > do (loop-finish) > collect (pop args)))) > (loop initially (loop as cdarg on args > ;; Removes the elements that don't start > ;; with an option. So > ;; ("lisp" "LoGS.lisp" "--help") > ;; will become ("--help") > until (optionp (car cdarg)) > finally (setq args cdarg)) > as nextopt = (nextarg) > while nextopt > collect nextopt))) > > (defun optionp (string) > "Is STRING an option?" > (or (starts-with "-" string) (starts-with "--" string))) > > (defun starts-with-p (string1 string2 &key (test #'string=)) > "Does STRING1 start with STRING2?" > (and (>= (length string1) (length string2)) > (funcall test string1 string2 :end1 (length string2)))) > > The macrolet is so simple now that maybe we could just make it a flet. > > I really love the LOOP macro :-) > > Thanks Jim > Vijay > > On 6/25/06, Vijay Lakshminarayanan wrote: > > On 6/25/06, Jim Prewett wrote: > > > > > > Hello, > > > > > > I'm thinking a more elegant way to specify a command line option might be > > > a macro I'm working on called defopt. Here is an example usage: > > > > > > (defopt > > > ;; the name of the flag is "file" > > > "file" > > > ;; the flag takes a filename parameter and optional position > > > ;; parameter > > > (filename &optional position) > > > ;; expression (with parameters bound) > > > ;; to set up things properly for that flag > > > ;; (this is the body of the lambda function) > > > (let ((ff > > > (make-instance > > > 'org.prewett.LoGS::File-Follower > > > :FileName > > > filename))) > > > ;; if position is specified, start there > > > (when position > > > (when (not ff) (error "no ff~%")) > > > (org.prewett.LoGS::set-file-follower-position > > > ff > > > (read-from-string position))) > > > (push ff *file-list*)) > > > ;; a list of (optional) aliases for this flag > > > :aliases '("f") > > > ;; the description of this flag, used in the help text > > > :description "file to read from") > > > > > > Does this look somewhat reasonable? > > > > > > I'm thinking that by specifying an option this way, the user only has to > > > specify the argument list once, and in a much more lispy way IMO. > > > > It is better than the old scheme :-) but I don't like many things. > > > > Also, why not have defopt accept the parameters as keyword arguments? > > It makes things clearer and reduces the need for comments. > > > > Something like: > > (defopt :name "file" :alias '("f" "ff") > > :params (filename &optional position) > > :action (list filename position)) > > > > There are several problems with :action > > > > First, you cannot have recursive functions. All actions must start > > with an explicit progn or a let or block. Take the case where an > > action must start with a DECLARE form. > > > > Also we cannot reuse functions with this syntax. I think we should > > keep the original syntax but instead of strings use sexps. It doesn't > > matter that we are forced to repeat because of the benefits and > > freedoms we get as a consequence. > > > > Here are a few examples: > > > > :parameters (one two &optional three) > > :actions #'list > > > > :parameters (one two three) > > :actions (lambda (&rest args) > > > > Here's an implementation of option-lengths that uses symbols: > > > > (defun option-lengths (option) > > (let ((min-args 0) ; minimum arguments to this flag > > (max-args 0) ; maximum arguments to this flag > > (seen-optional-arg nil) ; has an optional argument been seen? > > (multiargs nil) ; are an infinite number of arguments possible? > > ) > > (flet ((specialp (arg) > > (and (symbolp arg) (char= (elt (symbol-name arg) 0) #\&))) > > (special (arg) > > (let ((arg (intern (symbol-name arg)))) > > (ecase arg > > (&optional (setq seen-optional-arg t)) > > (&rest (setq multiargs t)) > > ((&body &key) > > (error "Cannot support ~a for command line." arg)))))) > > (loop as arg in (arguments option) > > if (specialp arg) > > do (special arg) > > else > > do (incf max-args) > > and unless (or seen-optional-arg multiargs) > > do (incf min-args) > > finally (return (if (or multiargs seen-optional-arg) > > (values min-args max-args) > > (values min-args ()))))))) > > > > While writing this, it occured to me that you could very easily parse > > the options that took single options using: > > (destructuring-bind (&key foo bar baz) (get-application-args) > > &body) > > > > IMHO there will be places where using a different lambda-list for the > > action and the explicit arguments might make life easier for the > > programmer. At the same time, as a lisper, I do think life can be > > made simpler with having to specify the arglist only in one place > > since that is the majority of how anyone would like it. Duplicating > > the code is also error-prone, I admit. > > > > But I would recommend that we keep the arguments and actions > > parameters independant of each other. > > > > > Jim > > Thanks > > Vijay > > > From MAILER-DAEMON Tue Jun 27 08:21:43 2006 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1FvCZb-0001F2-1r for mharc-logs-devel@gnu.org; Tue, 27 Jun 2006 08:21:43 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FvCZZ-0001Ex-Nw for logs-devel@nongnu.org; Tue, 27 Jun 2006 08:21:41 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FvCZY-0001Ec-Gh for Logs-devel@nongnu.org; Tue, 27 Jun 2006 08:21:40 -0400 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FvCZY-0001EZ-D7 for Logs-devel@nongnu.org; Tue, 27 Jun 2006 08:21:40 -0400 Received: from [129.24.245.111] (helo=mailer.hpc.unm.edu) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FvCle-0006EW-C9 for Logs-devel@nongnu.org; Tue, 27 Jun 2006 08:34:10 -0400 Received: from trucha01.hpc.unm.edu ([129.24.245.21] ident=download) by mailer.hpc.unm.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1FvCZQ-0006dA-Kl; Tue, 27 Jun 2006 06:21:32 -0600 Date: Tue, 27 Jun 2006 06:21:30 -0600 (MDT) From: Jim Prewett To: Logs-devel@nongnu.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Score: -4.3 (----) Cc: David Worth , Robert Ballance , nnett , "Timothy L. Thomas" Subject: [Logs-devel] LoGS' 3rd Birthday! X-BeenThere: logs-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: logs-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Jun 2006 12:21:41 -0000 All, I realized this morning that today is LoGS' 3rd birthday! It has been 3 years to the day since I wrote the first '(' :) I believe that paren is the only thing I've not thrown away so far ;) "Happy Birthday to LoGS, Happy Birthday to LoGS... "! Mostly LoGS is as far along as it is because of the help of people like you guys :) Thanks! I'm off to go find the astrology section of the paper ;) Jim p.s. LoGS was concieved in January 2003 when Dave and I started talking about the idea of representing Logsurfer rules with s-expressions. James E. Prewett Jim@Prewett.org download@hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210