Re: [Phpgroupware-developers] hello to Sieve

[Dan Kuykendall (Seek3r)]

Sieve looks like a good plan. I have read a bit thru the html summary
and it looks like it should work.

Im looking forward to this very much.

Seek3r

Tony (Angles) Puglisi wrote:
> 
> Welcome to your new proposed filtering scheme: Sieve.
> (no code committed yet - don't worry :)
> 
> Sieve has been in development for at least 6 years and describes a common 
> scheme for
> filtering mail. I don't  like to write docs nor reinvent the wheel, so I 
> choose
> Sieve, as much has already been written about it (ex. rfc3028).
> 
> Grabbing headers, unfolding them, and examining them is relatively easy, even 
> the
> regex is not that big of a deal. Writing a hard coded filter system is, 
> therefor,
> not that hard. However, writing something that is portable, flexible, and
> standardized is hard.
> 
> Sieve scripts can be used as is on the server side if Sieve is supported 
> there, such
> as with Cyrus Imap. Additionally, since it is standardized, it should provide 
> as
> some sort of lingua franca that can be translated into other filtering 
> schemes,
> server side or not (maildrop, procmail, etc...).
> 
> Sieve is extensible, as my summary shows my intent to include php's POSIX and 
> perl
> regex in our implementation.
> 
> I'll probably start with a small subset of Sieve, so that if you'all decide 
> that
> Sieve blows, the code will still be applicable. For example, instead of 
> "discard"
> I'll probably make you "fileto" some sort of trash folder, we don't want to 
> delete
> important mail during the beta phase, do we :)
> 
> Sieve docs are long on words and short on "action" so I paste my summary 
> below. If
> you like to read, you may find some Sieve docs at:
> 
> Sieve Home Page
> http://www.cyrusoft.com/sieve
> 
> Sieve Whitepaper
> http://www.cyrusoft.com/sieve/sievewhitepaper.pdf
> 
> RFC 3028
> http://www.ietf.org/rfc/rfc3028.txt
> 
> BEGIN SUMMARY: (you may wish to use the "view unformatted" option for this)
> 
> =====  String Comparison  =====
> 
> Three kinds
>         :is                     exact match
> 
>         :contains       substring match
>                 :comparator "i;ascii-casemap"   case-insensitive comparison
>                         (which treats uppercase and lowercase characters in
>                         the ASCII subset of UTF-8 as the same)
> 
>                 :comparator "i;octet"           case-sensitive comparison
>                         (simply compares octets)
> 
>                         NOTES:
>                         - default comparator if unspecified is 
> "i;ascii-casemap"
>                         - when specifying a name of a header line (ex. 
> "From"),
>                                 the comparator is always "i;ascii-casemap" 
> case insensitive
>                         - Comparators other than i;octet and i;ascii-casemap 
> must
>                                 be declared with require, as they are 
> extensions.
> 
>         :matches        wildcard glob-style match
>                 using the characters "*" and "?".  "*" matches zero or more
>                 characters, and "?" matches a single character.
> 
>         ANGLES PLANNED EXTNSIONS:
>         :ereg           php POSIX regex
>         :preg           php Perl regex
> 
> =====  Addresses Comparisons  =====
> 
>         :localpart              acts on the local-part (left-side of the @ 
> sign)
>         :domain                 acts on the domain part (right-side of the @ 
> sign)
>         :all                    acts on the whole address
> 
> =====  Blocks  =====
> 
> Blocks are sets of commands enclosed within curly braces.  Blocks are
> supplied to commands so that the commands can implement control
> commands.
> 
> A control structure is a command that happens to take a test and a
> block as one of its arguments; depending on the result of the test
> supplied as another argument, it runs the code in the block some
> number of times.
> 
> With the commands supplied in this memo, there are no loops.  The
> control structures supplied--if, elsif, and else--run a block either
> once or not at all.  So there are two arguments, the test and the
> block.
> 
> =====  Control Structures: If, Require, Stop  =====
> 
> Syntax:   if
> Syntax:   elsif
> Syntax:   else
> 
>         There is no sub if-then blocks, just linear if--else(if)*--(end)
> 
> Syntax:   stop
> 
>    The "stop" action ends all processing.  If no actions have been
>    executed, then the keep action is taken.
> 
> Syntax:   require
> 
>    The require command, if present, MUST be used before anything other
>    than a require can be used.  An error occurs if a require appears
>    after a command other than require.
> 
>    Example:  require ["fileinto", "reject"];
> 
>    Example:  require "fileinto";
>              require "vacation";
> 
> =====  Action Commands  =====
> 
> Five available: keep, fileinto, redirect, reject, discard
> 
> Syntax:   keep  =========================================
> 
>    The "keep" action is whatever action is taken in lieu of all other
>    actions, if no filtering happens at all; generally, this simply means
>    to file the message into the user's main mailbox.  This command
>    provides a way to execute this action without needing to know the
>    name of the user's main mailbox, providing a way to call it without
>    needing to understand the user's setup, or the underlying mail
>    system.
> 
> Syntax:   fileinto   =========================================
> 
>    The "fileinto" action delivers the message into the specified folder.
>    Implementations SHOULD support fileinto, but in some environments
>    this may be impossible.
> 
>    The capability string for use with the require command is "fileinto".
> 
>    In the following script, message A is filed into folder
>    "INBOX.harassment".
> 
>    Example:  require "fileinto";
>              if header :contains ["from"] "coyote" {
>                 fileinto "INBOX.harassment";
>              }
> 
> Syntax:   redirect   =========================================
> 
>    The "redirect" action is used to send the message to another user at
>    a supplied address, as a mail forwarding feature does.  The
>    "redirect" action makes no changes to the message body or existing
>    headers, but it may add new headers.  The "redirect" modifies the
>    envelope recipient.
> 
>    The redirect command performs an MTA-style "forward"--that is, what
>    you get from a .forward file using sendmail under UNIX.  The address
>    on the SMTP envelope is replaced with the one on the redirect command
>    and the message is sent back out.  (This is not an MUA-style forward,
>    which creates a new message with a different sender and message ID,
>    wrapping the old message in a new one.)
> 
>    A simple script can be used for redirecting all mail:
> 
>    Example:  redirect "address@hidden";
> 
>    Implementations SHOULD take measures to implement loop control,
>    possibly including adding headers to the message or counting received
>    headers.  If an implementation detects a loop, it causes an error.
> 
> Syntax:   reject  =========================================
> 
>    The optional "reject" action refuses delivery of a message by sending
>    back an [MDN] to the sender.  It resends the message to the sender,
>    wrapping it in a "reject" form, noting that it was rejected by the
>    recipient.  In the following script, message A is rejected and
>    returned to the sender.
> 
>    Example:  if header :contains "from" "address@hidden" {
>                 reject "I am not taking mail from you, and I don't want
>                 your birdseed, either!";
>              }
> 
>    A reject message MUST take the form of a failure MDN as specified  by
>    [MDN].    The  human-readable  portion  of  the  message,  the  first
>    component of the MDN, contains the human readable message  describing
>    the  error,  and  it  SHOULD  contain  additional  text  alerting the
>    original sender that mail was refused by a filter.  This part of  the
>    MDN might appear as follows:
> 
>    ------------------------------------------------------------
>    Message was refused by recipient's mail filtering program.  Reason
>    given was as follows:
> 
>    I am not taking mail from you, and I don't want your birdseed,
>    either!
>    ------------------------------------------------------------
> 
>    The MDN action-value field as defined in the MDN specification MUST
>    be "deleted" and MUST have the MDN-sent-automatically and automatic-
>    action modes set.
> 
>    Because some implementations can not or will not implement the reject
>    command, it is optional.  The capability string to be used with the
>    require command is "reject".
> 
> Syntax:   discard  =========================================
> 
>    Discard is used to silently throw away the message.  It does so by
>    simply canceling the implicit keep.  If discard is used with other
>    actions, the other actions still happen.  Discard is compatible with
>    all other actions.  (For instance fileinto+discard is equivalent to
>    fileinto.)
> 
>    Discard MUST be silent; that is, it MUST NOT return a non-delivery
>    notification of any kind ([DSN], [MDN], or otherwise).
> 
>    In the following script, any mail from "address@hidden" is thrown
>    out.
> 
>    Example:  if header :contains ["from"] ["address@hidden"] {
>                 discard;
>              }
> 
>    While an important part of this language, "discard" has the potential
>    to create serious problems for users: Students who leave themselves
>    logged in to an unattended machine in a public computer lab may find
>    their script changed to just "discard".  In order to protect users in
>    this situation (along with similar situations), implementations MAY
>    keep messages destroyed by a script for an indefinite period, and MAY
>    disallow scripts that throw out all mail.
> 
> =====  Test Commands  =====
> 
>    Tests are used in conditionals to decide which part(s) of the
>    conditional to execute.
> 
> Available tests:
>         address
>         allof
>         anyof
>         envelope
>         exists
>         false
>         header
>         not
>         size
>         true
> 
> Syntax:   address [ADDRESS-PART] [COMPARATOR] [MATCH-TYPE]
> 
>    The address test matches Internet addresses in structured headers
>    that contain addresses.  It returns true if any header contains any
>    key in the specified part of the address, as modified by the
>    comparator and the match keyword.
> 
> Syntax:   allof   =========================================
> 
>    The allof test performs a logical AND on the tests supplied to it.
> 
>    Example:  allof (false, false)  => false
>              allof (false, true)   => false
>              allof (true,  true)   => true
> 
>    The allof test takes as its argument a test-list.
> 
> Syntax:   anyof   =========================================
> 
>    The anyof test performs a logical OR on the tests supplied to it.
> 
>    Example:  anyof (false, false)  => false
>              anyof (false, true)   => true
>              anyof (true,  true)   => true
> 
> Syntax:   envelope [COMPARATOR] [ADDRESS-PART] [MATCH-TYPE]
> 
>    The "envelope" test is true if the specified part of the SMTP (or
>    equivalent) envelope matches the specified key.
> 
>    If one of the envelope-part strings is (case insensitive) "from",
>    then matching occurs against the FROM address used in the SMTP MAIL
>    command.
> 
> NOTE (ANGLES):
>         I THINK THIS IS ONLY AVAILABLE IN CERTAIN SITUATIONS,
>         SUCH AS DURING AN SMTP TRANSACTION, THIS IS TH ONLY
>         TIME WHEN THIS SPECIFIC "RCPT TO" DATUM IS EXPOSED,
>         I.E. A CLIENT FEEDING AN OUTGOING MESSAGE TO A SMTP SERVER,
>         PERHAPS OTHER OCCASIONS SUCH AS DURING SMTP DESTINATION
>         FINAL DELIVERY. I MAY BE WORNG, THOUGH.
> 
> Syntax:   exists   =========================================
> 
>    The "exists" test is true if the headers listed in the header-names
>    argument exist within the message.  All of the headers must exist or
>    the test is false.
> 
>    The following example throws out mail that doesn't have a From header
>    and a Date header.
> 
>    Example:  if not exists ["From","Date"] {
>                 discard;
>              }
> 
> Syntax:   false   =========================================
> 
>    The "false" test always evaluates to false.
> 
> Syntax:   header [COMPARATOR] [MATCH-TYPE]   =============
> 
>    The "header" test evaluates to true if any header name matches any
>    key.  The type of match is specified by the optional match argument,
>    which defaults to ":is" if not specified, as specified in section
>    2.6.
> 
>    Like address and envelope, this test returns true if any combination
>    of the string-list and key-list arguments match.
> 
>    If a header listed in the header-names argument exists, it contains
>    the null key ("").  However, if the named header is not present, it
>    does not contain the null key.  So if a message contained the header
> 
>            X-Caffeine: C8H10N4O2
> 
>    these tests on that header evaluate as follows:
> 
>            header :is ["X-Caffeine"] [""]         => false
>            header :contains ["X-Caffeine"] [""]   => true
> 
> Syntax:   not    =========================================
> 
>    The "not" test takes some other test as an argument, and yields the
>    opposite result.  "not false" evaluates to "true" and "not true"
>    evaluates to "false".
> 
> Syntax:   size <":over" / ":under">    =============================
> 
>    The "size" test deals with the size of a message.  It takes either a
>    tagged argument of ":over" or ":under", followed by a number
>    representing the size of the message.
> 
>    If the argument is ":over", and the size of the message is greater
>    than the number provided, the test is true; otherwise, it is false.
> 
>    If the argument is ":under", and the size of the message is less than
>    the number provided, the test is true; otherwise, it is false.
> 
>    Exactly one of ":over" or ":under" must be specified, and anything
>    else is an error.
> 
>    The size of a message is defined to be the number of octets from the
>    initial header until the last character in the message body.
> 
>    Note that for a message that is exactly 4,000 octets, the message is
>    neither ":over" 4000 octets or ":under" 4000 octets.
> 
> Syntax:   true   =========================================
> 
>    The "true" test always evaluates to true.
> 
> ======   Extended Example   ======
> 
>    The following is an extended example of a Sieve script.  Note that it
>    does not make use of the implicit keep.
> 
>     #
>     # Example Sieve Filter
>     # Declare any optional features or extension used by the script
>     #
>     require ["fileinto", "reject"];
> 
>     #
>     # Reject any large messages (note that the four leading dots get
>     # "stuffed" to three)
>     #
>     if size :over 1M
>             {
>             reject text:
>     Please do not send me large attachments.
>     Put your file on a server and send me the URL.
>     Thank you.
>     .... Fred
>     .
>     ;
>             stop;
>             }
>     #
>     # Handle messages from known mailing lists
>     # Move messages from IETF filter discussion list to filter folder
>     #
>     if header :is "Sender" "address@hidden"
>             {
>             fileinto "filter";  # move to "filter" folder
>             }
>     #
>     # Keep all messages to or from people in my company
>     #
>     elsif address :domain :is ["From", "To"] "example.com"
>             {
>             keep;               # keep in "In" folder
>             }
> 
>     #
>     # Try and catch unsolicited email.  If a message is not to me,
>     # or it contains a subject known to be spam, file it away.
>     #
>     elsif anyof (not address :all :contains
>                    ["To", "Cc", "Bcc"] "address@hidden",
>                  header :matches "subject"
>                    ["*make*money*fast*", "*university*dipl*mas*"])
>             {
>             # If message header does not contain my address,
>             # it's from a list.
>             fileinto "spam";   # move to "spam" folder
>             }
>     else
>             {
>             # Move all other (non-company) mail to "personal"
>             # folder.
>             fileinto "personal";
>             }
> 
> ======  Example: hillen-sieve-script-001.txt   ======
> 
> require "fileinto";
>         if header :is "X-Mailinglist" "suse-linux" {
>                 fileinto "INBOX.Listen.suse-linux";}
>         elsif header :contains "Mailing-List" "reiserfs" {
>                 fileinto "INBOX.Listen.reiserfs";}
>         elsif address :contains :all ["to", "cc", "bcc"] "free-clim" {
>                 fileinto "INBOX.Listen.free-clim";}
>         elsif header :contains "List-Id" "gnupg-users.gnupg.org" {
>                 fileinto "INBOX.Listen.gnupg";}
>         elsif header :is "X-loop" "isdn4linux" {
>                 fileinto "INBOX.Listen.isdn4linux";}
>         elsif header :contains  "Mailing-list" "address@hidden"{
>                 fileinto "INBOX.Listen.qmail";}
>         elsif allof (header :contains "Sender" "address@hidden",
>                      address :contains :localpart ["to", "cc", "bcc"] 
> "info-cyrus"){
>                 fileinto "INBOX.Listen.info-cyrus";}
>         elsif header :contains "Sender" "address@hidden"{
>                 fileinto "INBOX.Listen.ntbugtraq";}
>         elsif header :is "list-id" "<ietf-mta-filters.imc.org>"{
>                 fileinto "INBOX.Listen.sieve";}
>         elsif header :contains "From" "address@hidden"{
>                 fileinto "INBOX.Newsletter.securityportal";}
>         elsif address :contains :all ["from"] "address@hidden"{
>                 fileinto "INBOX.Newsletter.ebay";}
>         elsif address :contains :all ["to", "cc", "bcc"] "address@hidden"{
>                         fileinto "INBOX.Listen.allegro-cl";}
>         elsif address :contains :all ["to", "cc", "bcc"] "address@hidden"{
>                        fileinto "INBOX.Listen.plob";}
>                 else {
>                          fileinto "INBOX";
> 
> ======  Example: maro-sieve-script-001.txt   ======
> 
> ##############################################
> # Submitted by Tony Maro
> # http://www.maro.net/tony
> # Use and abuse this script as you will
> # I'm not responsible for what it does...
> #
> # Save this script in your home directory.
> # Install this script with the following command,
> # replacing "thisfile" and "yourname" with the appropriate
> # information:
> #
> # installsieve -i thisfile -m PLAIN -u yourname localhost
> #
> #
> require "fileinto";
> require "reject";
> #
> # Circle MUD mailing list list
> # All mail from the list has "[CIRCLE]" in the subject
> # Place all these in the "Circle List" folder
> # I could filter on the mail list senders e-mail as it's always
> # the same, but this way I even catch personal replies that come
> # directly from a user to me
> if header :contains "Subject" "[CIRCLE]" {
>         fileinto "INBOX.Circle List";
> }
> #
> # "shockwave" e-mail virus - just reject it
> #
> if header :contains "Subject" "A great Shockwave flash movie" {
>         reject "Possible virus?  Check your system!";
> }
> #
> # Get a lot of junk from dial-up uu.net accounts
> # Make sure you create a Junk folder under your INBOX
> # I like this one because it catches them even if they
> # relay their crap through some international open
> # mail relay
> #
> if header :contains "Received" ".da.uu.net" {
>         fileinto "INBOX.Junk";
> }
> #
> # If the mail is listed as TO someone at bigfoot.com
> # Then just reject it because it's spam (my experience anyway)
> #
> if header :contains "To:" "@bigfoot.com" {
>         reject "Yeah, right.  Bugoff, hosier!";
> }
> #
> # If the mail is not directed to me put in the junk folder
> # be sure to replace address@hidden with the
> # appropriate information
> # Took me a while to figure out how to do NOT statements... :-}
> #
> if anyof ( not address :all :contains ["To", "Cc", "Bcc"] "address@hidden" ) {
>         fileinto "INBOX.Junk";
> }
> #
> # Everything that makes it to here ends up in the INBOX
> ########################################################
> --
> that's "angle" as in geometry
> 
> _______________________________________________
> Phpgroupware-developers mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/phpgroupware-developers