diff --git a/doc/rfc/Makefile.am b/doc/rfc/Makefile.am
index 386287a50f3b5ff2399ad5c5973b9a38e2adccd6..948b32fd719ae7f4f1e18dcc81fc2df144c54909 100644
--- a/doc/rfc/Makefile.am
+++ b/doc/rfc/Makefile.am
@@ -2,16 +2,18 @@ docfiles = \
body.rfc5173.txt \
collation.rfc4790.txt \
copy.rfc3894.txt \
+ date-index.rfc5260.txt \
draft-degener-sieve-multiscript-00.txt \
draft-duerst-mailto-bis-05.txt \
- draft-ietf-appsawg-sieve-duplicate-03.txt \
draft-murchison-sieve-regex-07.txt \
+ duplicate.rfc7352.txt \
editheader.rfc5293.txt \
environment.rfc5183.txt \
ihave.rfc5463.txt \
imail.rfc2822.txt \
imap4flags.rfc5232.txt \
include.rfc6609.txt \
+ mailbox-metadata.rfc5490.txt \
mailto.rfc2368.txt \
managesieve.rfc5804.txt \
notify-mailto.rfc5436.txt \
@@ -21,7 +23,6 @@ docfiles = \
sieve.rfc5228.txt \
spamvirustest.rfc5235.txt \
spec-bosch-sieve-debug.txt \
- spec-bosch-sieve-duplicate.txt \
spec-bosch-sieve-extprograms.txt \
subaddress.rfc5233.txt \
uri.rfc3986.txt \
@@ -30,6 +31,7 @@ docfiles = \
vacation-seconds.rfc6131.txt \
variables.rfc5229.txt
+
EXTRA_DIST = \
$(docfiles)
diff --git a/doc/rfc/date-index.rfc5260.txt b/doc/rfc/date-index.rfc5260.txt
new file mode 100644
index 0000000000000000000000000000000000000000..0baf5f6e48f4f42140b5c6d641e389c46c99c786
--- /dev/null
+++ b/doc/rfc/date-index.rfc5260.txt
@@ -0,0 +1,731 @@
+
+
+
+
+
+
+Network Working Group N. Freed
+Request for Comments: 5260 Sun Microsystems
+Category: Standards Track July 2008
+
+
+ Sieve Email Filtering: Date and Index Extensions
+
+Status of This Memo
+
+ This document specifies an Internet standards track protocol for the
+ Internet community, and requests discussion and suggestions for
+ improvements. Please refer to the current edition of the "Internet
+ Official Protocol Standards" (STD 1) for the standardization state
+ and status of this protocol. Distribution of this memo is unlimited.
+
+Abstract
+
+ This document describes the "date" and "index" extensions to the
+ Sieve email filtering language. The "date" extension gives Sieve the
+ ability to test date and time values in various ways. The "index"
+ extension provides a means to limit header and address tests to
+ specific instances of header fields when header fields are repeated.
+
+Table of Contents
+
+ 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
+ 2. Conventions Used in This Document . . . . . . . . . . . . . . 2
+ 3. Capability Identifiers . . . . . . . . . . . . . . . . . . . . 3
+ 4. Date Test . . . . . . . . . . . . . . . . . . . . . . . . . . 3
+ 4.1. Zone and Originalzone Arguments . . . . . . . . . . . . . 4
+ 4.2. Date-part Argument . . . . . . . . . . . . . . . . . . . . 4
+ 4.3. Comparator Interactions with Date-part Arguments . . . . . 5
+ 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 6
+ 5. Currentdate Test . . . . . . . . . . . . . . . . . . . . . . . 6
+ 5.1. Examples . . . . . . . . . . . . . . . . . . . . . . . . . 6
+ 6. Index Extension . . . . . . . . . . . . . . . . . . . . . . . 7
+ 6.1. Example . . . . . . . . . . . . . . . . . . . . . . . . . 8
+ 7. Security Considerations . . . . . . . . . . . . . . . . . . . 8
+ 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9
+ 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9
+ 9.1. Normative References . . . . . . . . . . . . . . . . . . . 9
+ 9.2. Informative References . . . . . . . . . . . . . . . . . . 10
+ Appendix A. Julian Date Conversions . . . . . . . . . . . . . . . 11
+ Appendix B. Acknowledgements . . . . . . . . . . . . . . . . . . 12
+
+
+
+
+
+
+
+Freed Standards Track [Page 1]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+1. Introduction
+
+ Sieve [RFC5228] is a language for filtering email messages at or
+ around the time of final delivery. It is designed to be
+ implementable on either a mail client or mail server. It is meant to
+ be extensible, simple, and independent of access protocol, mail
+ architecture, and operating system. It is suitable for running on a
+ mail server where users may not be allowed to execute arbitrary
+ programs, such as on black box Internet Message Access Protocol
+ [RFC3501] servers, as it does not have user-controlled loops or the
+ ability to run external programs.
+
+ The "date" extension provides a new date test to extract and match
+ date/time information from structured header fields. The date test
+ is similar in concept to the address test specified in [RFC5228],
+ which performs similar operations on addresses in header fields.
+
+ The "date" extension also provides a currentdate test that operates
+ on the date and time when the Sieve script is executed.
+
+ Some header fields containing date/time information, e.g., Received:,
+ naturally occur more than once in a single header. In such cases it
+ is useful to be able to restrict the date test to some subset of the
+ fields that are present. For example, it may be useful to apply a
+ date test to the last (earliest) Received: field. Additionally, it
+ may also be useful to apply similar restrictions to either the header
+ or address tests specified in [RFC5228].
+
+ For this reason, this specification also defines an "index"
+ extension. This extension adds two additional tagged arguments
+ :index and :last to the header, address, and date tests. If present,
+ these arguments specify which occurrence of the named header field is
+ to be tested.
+
+2. Conventions Used in This Document
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in RFC 2119 [RFC2119].
+
+ The terms used to describe the various components of the Sieve
+ language are taken from Section 1.1 of [RFC5228]. Section 2 of the
+ same document describes basic Sieve language syntax and semantics.
+ The date-time syntactic element defined using ABNF notation [RFC5234]
+ in [RFC3339] is also used here.
+
+
+
+
+
+
+Freed Standards Track [Page 2]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+3. Capability Identifiers
+
+ The capability strings associated with the two extensions defined in
+ this document are "date" and "index".
+
+4. Date Test
+
+ Usage: date [<":zone" <time-zone: string>> / ":originalzone"]
+ [COMPARATOR] [MATCH-TYPE] <header-name: string>
+ <date-part: string> <key-list: string-list>
+
+ The date test matches date/time information derived from headers
+ containing [RFC2822] date-time values. The date/time information is
+ extracted from the header, shifted to the specified time zone, and
+ the value of the given date-part is determined. The test returns
+ true if the resulting string matches any of the strings specified in
+ the key-list, as controlled by the comparator and match keywords.
+ The date test returns false unconditionally if the specified header
+ field does not exist, the field exists but does not contain a
+ syntactically valid date-time specification, the date-time isn't
+ valid according to the rules of the calendar system (e.g., January
+ 32nd, February 29 in a non-leap year), or the resulting string fails
+ to match any key-list value.
+
+ The type of match defaults to ":is" and the default comparator is
+ "i;ascii-casemap".
+
+ Unlike the header and address tests, the date test can only be
+ applied to a single header field at a time. If multiple header
+ fields with the same name are present, only the first field that is
+ found is used. (Note, however, that this behavior can be modified
+ with the "index" extension defined below.) These restrictions
+ simplify the test and keep the meaning clear.
+
+ The "relational" extension [RFC5231] adds a match type called
+ ":count". The count of a date test is 1 if the specified field
+ exists and contains a valid date; 0, otherwise.
+
+ Implementations MUST support extraction of RFC 2822 date-time
+ information that either makes up the entire header field (e.g., as it
+ does in a standard Date: header field) or appears at the end of a
+ header field following a semicolon (e.g., as it does in a standard
+ Received: header field). Implementations MAY support extraction of
+ date and time information in RFC2822 or other formats that appears in
+ other positions in header field content. In the case of a field
+ containing more than one date or time value, the last one that
+ appears SHOULD be used.
+
+
+
+
+Freed Standards Track [Page 3]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+4.1. Zone and Originalzone Arguments
+
+ The :originalzone argument specifies that the time zone offset
+ originally in the extracted date-time value should be retained. The
+ :zone argument specifies a specific time zone offset that the date-
+ time value is to be shifted to prior to testing. It is an error to
+ specify both :zone and :originalzone.
+
+ The value of time-zone MUST be an offset relative to UTC with the
+ following syntax:
+
+ time-zone = ( "+" / "-" ) 4DIGIT
+
+ The "+" or "-" indicates whether the time-of-day is ahead of (i.e.,
+ east of) or behind (i.e., west of) UTC. The first two digits
+ indicate the number of hours difference from Universal Time, and the
+ last two digits indicate the number of minutes difference from
+ Universal Time. Note that this agrees with the RFC 2822 format for
+ time zone offsets, not the ISO 8601 format.
+
+ If both the :zone and :originalzone arguments are omitted, the local
+ time zone MUST be used.
+
+4.2. Date-part Argument
+
+ The date-part argument specifies a particular part of the resulting
+ date/time value to match against the key-list. Possible case-
+ insensitive values are:
+
+ "year" => the year, "0000" .. "9999".
+ "month" => the month, "01" .. "12".
+ "day" => the day, "01" .. "31".
+ "date" => the date in "yyyy-mm-dd" format.
+ "julian" => the Modified Julian Day, that is, the date
+ expressed as an integer number of days since
+ 00:00 UTC on November 17, 1858 (using the Gregorian
+ calendar). This corresponds to the regular
+ Julian Day minus 2400000.5. Sample routines to
+ convert to and from modified Julian dates are
+ given in Appendix A.
+ "hour" => the hour, "00" .. "23".
+ "minute" => the minute, "00" .. "59".
+ "second" => the second, "00" .. "60".
+ "time" => the time in "hh:mm:ss" format.
+ "iso8601" => the date and time in restricted ISO 8601 format.
+ "std11" => the date and time in a format appropriate
+ for use in a Date: header field [RFC2822].
+
+
+
+
+Freed Standards Track [Page 4]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+ "zone" => the time zone in use. If the user specified a
+ time zone with ":zone", "zone" will
+ contain that value. If :originalzone is specified
+ this value will be the original zone specified
+ in the date-time value. If neither argument is
+ specified the value will be the server's default
+ time zone in offset format "+hhmm" or "-hhmm". An
+ offset of 0 (Zulu) always has a positive sign.
+ "weekday" => the day of the week expressed as an integer between
+ "0" and "6". "0" is Sunday, "1" is Monday, etc.
+
+ The restricted ISO 8601 format is specified by the date-time ABNF
+ production given in [RFC3339], Section 5.6, with the added
+ restrictions that the letters "T" and "Z" MUST be in upper case, and
+ a time zone offset of zero MUST be represented by "Z" and not
+ "+00:00".
+
+4.3. Comparator Interactions with Date-part Arguments
+
+ Not all comparators are suitable with all date-part arguments. In
+ general, the date-parts can be compared and tested for equality with
+ either "i;ascii-casemap" (the default) or "i;octet", but there are
+ two exceptions:
+
+ julian This is an integer, and may or may not have leading zeros.
+ As such, "i;ascii-numeric" is almost certainly the best
+ comparator to use with it.
+
+ std11 This is provided as a means to obtain date/time values in a
+ format appropriate for inclusion in email header fields. The
+ wide range of possible syntaxes for a std11 date/time --
+ which implementations of this extension are free to use when
+ composing a std11 string -- makes this format a poor choice
+ for comparisons. Nevertheless, if a comparison must be
+ performed, this is case-insensitive, and therefore "i;ascii-
+ casemap" needs to be used.
+
+ "year", "month", "day", "hour", "minute", "second" and "weekday" all
+ use fixed-width string representations of integers, and can therefore
+ be compared with "i;octet", "i;ascii-casemap", and "i;ascii-numeric"
+ with equivalent results.
+
+ "date" and "time" also use fixed-width string representations of
+ integers, and can therefore be compared with "i;octet" and "i;ascii-
+ casemap"; however, "i;ascii-numeric" can't be used with it, as
+ "i;ascii-numeric" doesn't allow for non-digit characters.
+
+
+
+
+
+Freed Standards Track [Page 5]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+4.4. Examples
+
+ The Date: field can be checked to test when the sender claims to have
+ created the message and act accordingly:
+
+ require ["date", "relational", "fileinto"];
+ if allof(header :is "from" "boss@example.com",
+ date :value "ge" :originalzone "date" "hour" "09",
+ date :value "lt" :originalzone "date" "hour" "17")
+ { fileinto "urgent"; }
+
+ Testing the initial Received: field can provide an indication of when
+ a message was actually received by the local system:
+
+ require ["date", "relational", "fileinto"];
+ if anyof(date :is "received" "weekday" "0",
+ date :is "received" "weekday" "6")
+ { fileinto "weekend"; }
+
+5. Currentdate Test
+
+ Usage: currentdate [":zone" <time-zone: string>]
+ [COMPARATOR] [MATCH-TYPE]
+ <date-part: string>
+ <key-list: string-list>
+
+ The currentdate test is similar to the date test, except that it
+ operates on the current date/time rather than a value extracted from
+ the message header. In particular, the ":zone" and date-part
+ arguments are the same as those in the date test.
+
+ All currentdate tests in a single Sieve script MUST refer to the same
+ point in time during execution of the script.
+
+ The :count value of a currentdate test is always 1.
+
+5.1. Examples
+
+ The simplest use of currentdate is to have an action that only
+ operates at certain times. For example, a user might want to have
+ messages redirected to their pager after business hours and on
+ weekends:
+
+
+
+
+
+
+
+
+
+Freed Standards Track [Page 6]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+ require ["date", "relational"];
+ if anyof(currentdate :is "weekday" "0",
+ currentdate :is "weekday" "6",
+ currentdate :value "lt" "hour" "09",
+ currentdate :value "ge" "hour" "17")
+ { redirect "pager@example.com"; }
+
+ Currentdate can be used to set up vacation [RFC5230] responses in
+ advance and to stop response generation automatically:
+
+ require ["date", "relational", "vacation"];
+ if allof(currentdate :value "ge" "date" "2007-06-30",
+ currentdate :value "le" "date" "2007-07-07")
+ { vacation :days 7 "I'm away during the first week in July."; }
+
+ Currentdate may also be used in conjunction with the variables
+ extension to pass time-dependent arguments to other tests and
+ actions. The following Sieve places messages in a folder named
+ according to the current month and year:
+
+ require ["date", "variables", "fileinto"];
+ if currentdate :matches "month" "*" { set "month" "${1}"; }
+ if currentdate :matches "year" "*" { set "year" "${1}"; }
+ fileinto "${month}-${year}";
+
+ Finally, currentdate can be used in conjunction with the editheader
+ extension to insert a header-field containing date/time information:
+
+ require ["variables", "date", "editheader"];
+ if currentdate :matches "std11" "*"
+ {addheader "Processing-date" "${0}";}
+
+6. Index Extension
+
+ The "index" extension, if specified, adds optional :index and :last
+ arguments to the header, address, and date tests as follows:
+
+ Syntax: date [":index" <fieldno: number> [":last"]]
+ [<":zone" <time-zone: string>> / ":originalzone"]
+ [COMPARATOR] [MATCH-TYPE] <header-name: string>
+ <date-part: string> <key-list: string-list>
+
+
+ Syntax: header [":index" <fieldno: number> [":last"]]
+ [COMPARATOR] [MATCH-TYPE]
+ <header-names: string-list> <key-list: string-list>
+
+
+
+
+
+Freed Standards Track [Page 7]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+ Syntax: address [":index" <fieldno: number> [":last"]]
+ [ADDRESS-PART] [COMPARATOR] [MATCH-TYPE]
+ <header-list: string-list> <key-list: string-list>
+
+ If :index <fieldno> is specified, the attempts to match a value are
+ limited to the header field fieldno (beginning at 1, the first named
+ header field). If :last is also specified, the count is backwards; 1
+ denotes the last named header field, 2 the second to last, and so on.
+ Specifying :last without :index is an error.
+
+ :index only counts separate header fields, not multiple occurrences
+ within a single field. In particular, :index cannot be used to test
+ a specific address in an address list contained within a single
+ header field.
+
+ Both header and address allow the specification of more than one
+ header field name. If more than one header field name is specified,
+ all the named header fields are counted in the order specified by the
+ header-list.
+
+6.1. Example
+
+ Mail delivery may involve multiple hops, resulting in the Received:
+ field containing information about when a message first entered the
+ local administrative domain being the second or subsequent field in
+ the message. As long as the field offset is consistent, it can be
+ tested:
+
+ # Implement the Internet-Draft cutoff date check assuming the
+ # second Received: field specifies when the message first
+ # entered the local email infrastructure.
+ require ["date", "relational", "index"];
+ if date :value "gt" :index 2 :zone "-0500" "received"
+ "iso8601" "2007-02-26T09:00:00-05:00",
+ { redirect "aftercutoff@example.org"; }
+
+7. Security Considerations
+
+ The facilities defined here, like the facilities in the base Sieve
+ specification, operate on message header information that can easily
+ be forged. Note, however, that some fields are inherently more
+ reliable than others. For example, the Date: field is typically
+ inserted by the message sender and can be altered at any point. By
+ contrast, the uppermost Received: field is typically inserted by the
+ local mail system and is therefore difficult for the sender or an
+ intermediary to falsify.
+
+
+
+
+
+Freed Standards Track [Page 8]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+ Use of the currentdate test makes script behavior inherently less
+ predictable and harder to analyze. This may have consequences for
+ systems that use script analysis to try and spot problematic scripts.
+
+ All of the security considerations given in the base Sieve
+ specification also apply to these extensions.
+
+8. IANA Considerations
+
+ The following templates specify the IANA registrations of the two
+ Sieve extensions specified in this document:
+
+ To: iana@iana.org
+ Subject: Registration of new Sieve extensions
+
+ Capability name: date
+ Description: The "date" extension gives Sieve the ability
+ to test date and time values.
+ RFC number: RFC 5260
+ Contact address: Sieve discussion list <ietf-mta-filters@imc.org>
+
+ Capability name: index
+ Description: The "index" extension provides a means to
+ limit header and address tests to specific
+ instances when more than one field of a
+ given type is present.
+ RFC number: RFC 5260
+ Contact address: Sieve discussion list <ietf-mta-filters@imc.org>
+
+9. References
+
+9.1. Normative References
+
+ [CALGO199] Tantzen, R., "Algorithm 199: Conversions Between Calendar
+ Date and Julian Day Number", Collected Algorithms from
+ CACM 199.
+
+ [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+ [RFC2822] Resnick, P., "Internet Message Format", RFC 2822,
+ April 2001.
+
+ [RFC3339] Klyne, G., Ed. and C. Newman, "Date and Time on the
+ Internet: Timestamps", RFC 3339, July 2002.
+
+ [RFC5228] Guenther, P. and T. Showalter, "Sieve: An Email Filtering
+ Language", RFC 5228, January 2008.
+
+
+
+Freed Standards Track [Page 9]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+ [RFC5231] Segmuller, W. and B. Leiba, "Sieve Email Filtering:
+ Relational Extension", RFC 5231, January 2008.
+
+ [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
+ Specifications: ABNF", STD 68, RFC 5234, January 2008.
+
+9.2. Informative References
+
+ [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
+ 4rev1", RFC 3501, March 2003.
+
+ [RFC5230] Showalter, T. and N. Freed, "Sieve Email Filtering:
+ Vacation Extension", RFC 5230, January 2008.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Freed Standards Track [Page 10]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+Appendix A. Julian Date Conversions
+
+ The following C routines show how to translate day/month/year
+ information to and from modified Julian dates. These routines are
+ straightforward translations of the Algol routines specified in CACM
+ Algorithm 199 [CALGO199].
+
+ Given the day, month, and year, jday returns the modified Julian
+ date.
+
+ int jday(int year, int month, int day)
+ {
+ int j, c, ya;
+
+ if (month > 2)
+ month -= 3;
+ else
+ {
+ month += 9;
+ year--;
+ }
+ c = year / 100;
+ ya = year - c * 100;
+ return (c * 146097 / 4 + ya * 1461 / 4 + (month * 153 + 2) / 5 +
+ day + 1721119);
+ }
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Freed Standards Track [Page 11]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+ Given j, the modified Julian date, jdate returns the day, month, and
+ year.
+
+ void jdate(int j, int *year, int *month, int *day)
+ {
+ int y, m, d;
+
+ j -= 1721119;
+ y = (j * 4 - 1) / 146097;
+ j = j * 4 - y * 146097 - 1;
+ d = j / 4;
+ j = (d * 4 + 3) / 1461;
+ d = d * 4 - j * 1461 + 3;
+ d = (d + 4) / 4;
+ m = (d * 5 - 3) / 153;
+ d = d * 5 - m * 153 - 3;
+ *day = (d + 5) / 5;
+ *year = y * 100 + j;
+ if (m < 10)
+ *month = m + 3;
+ else
+ {
+ *month = m - 9;
+ *year += 1;
+ }
+ }
+
+Appendix B. Acknowledgements
+
+ Dave Cridland contributed the text describing the proper comparators
+ to use with different date-parts. Cyrus Daboo, Frank Ellerman,
+ Alexey Melnikov, Chris Newman, Dilyan Palauzov, and Aaron Stone
+ provided helpful suggestions and corrections.
+
+Author's Address
+
+ Ned Freed
+ Sun Microsystems
+ 800 Royal Oaks
+ Monrovia, CA 91016-6347
+ USA
+
+ Phone: +1 909 457 4293
+ EMail: ned.freed@mrochek.com
+
+
+
+
+
+
+
+Freed Standards Track [Page 12]
+
+RFC 5260 Sieve Date and Index Extensions July 2008
+
+
+Full Copyright Statement
+
+ Copyright (C) The IETF Trust (2008).
+
+ This document is subject to the rights, licenses and restrictions
+ contained in BCP 78, and except as set forth therein, the authors
+ retain all their rights.
+
+ This document and the information contained herein are provided on an
+ "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
+ OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
+ THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
+ OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
+ THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
+ WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
+
+Intellectual Property
+
+ The IETF takes no position regarding the validity or scope of any
+ Intellectual Property Rights or other rights that might be claimed to
+ pertain to the implementation or use of the technology described in
+ this document or the extent to which any license under such rights
+ might or might not be available; nor does it represent that it has
+ made any independent effort to identify any such rights. Information
+ on the procedures with respect to rights in RFC documents can be
+ found in BCP 78 and BCP 79.
+
+ Copies of IPR disclosures made to the IETF Secretariat and any
+ assurances of licenses to be made available, or the result of an
+ attempt made to obtain a general license or permission for the use of
+ such proprietary rights by implementers or users of this
+ specification can be obtained from the IETF on-line IPR repository at
+ http://www.ietf.org/ipr.
+
+ The IETF invites any interested party to bring to its attention any
+ copyrights, patents or patent applications, or other proprietary
+ rights that may cover technology that may be required to implement
+ this standard. Please address the information to the IETF at
+ ietf-ipr@ietf.org.
+
+
+
+
+
+
+
+
+
+
+
+
+Freed Standards Track [Page 13]
+
diff --git a/doc/rfc/draft-ietf-appsawg-sieve-duplicate-03.txt b/doc/rfc/draft-ietf-appsawg-sieve-duplicate-03.txt
deleted file mode 100644
index 1100934c7fd5967445590acf7ca761ff2ac42612..0000000000000000000000000000000000000000
--- a/doc/rfc/draft-ietf-appsawg-sieve-duplicate-03.txt
+++ /dev/null
@@ -1,728 +0,0 @@
-
-
-
-APPSAWG S. Bosch
-Internet-Draft March 3, 2014
-Intended status: Standards Track
-Expires: September 4, 2014
-
-
- Sieve Email Filtering: Detecting Duplicate Deliveries
- draft-ietf-appsawg-sieve-duplicate-03
-
-Abstract
-
- This document defines a new test command "duplicate" for the "Sieve"
- email filtering language. This test adds the ability to detect
- duplications. The main application for this new test is handling
- duplicate deliveries commonly caused by mailing list subscriptions or
- redirected mail addresses. The detection is normally performed by
- matching the message ID to an internal list of message IDs from
- previously delivered messages. For more complex applications, the
- "duplicate" test can also use the content of a specific header or
- other parts of the message.
-
-Status of this Memo
-
- This Internet-Draft is submitted in full conformance with the
- provisions of BCP 78 and BCP 79.
-
- Internet-Drafts are working documents of the Internet Engineering
- Task Force (IETF). Note that other groups may also distribute
- working documents as Internet-Drafts. The list of current Internet-
- Drafts is at http://datatracker.ietf.org/drafts/current/.
-
- Internet-Drafts are draft documents valid for a maximum of six months
- and may be updated, replaced, or obsoleted by other documents at any
- time. It is inappropriate to use Internet-Drafts as reference
- material or to cite them other than as "work in progress."
-
- This Internet-Draft will expire on September 4, 2014.
-
-Copyright Notice
-
- Copyright (c) 2014 IETF Trust and the persons identified as the
- document authors. All rights reserved.
-
- This document is subject to BCP 78 and the IETF Trust's Legal
- Provisions Relating to IETF Documents
- (http://trustee.ietf.org/license-info) in effect on the date of
- publication of this document. Please review these documents
- carefully, as they describe your rights and restrictions with respect
-
-
-
-Bosch Expires September 4, 2014 [Page 1]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- to this document. Code Components extracted from this document must
- include Simplified BSD License text as described in Section 4.e of
- the Trust Legal Provisions and are provided without warranty as
- described in the Simplified BSD License.
-
-
-Table of Contents
-
- 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
- 2. Conventions Used in This Document . . . . . . . . . . . . . . 3
- 3. Test "duplicate" . . . . . . . . . . . . . . . . . . . . . . . 3
- 3.1. Interaction with Other Sieve Extensions . . . . . . . . . 8
- 4. Sieve Capability Strings . . . . . . . . . . . . . . . . . . . 8
- 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
- 5.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . . . 8
- 5.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . . . 8
- 5.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . . . 9
- 5.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . . . 10
- 6. Security Considerations . . . . . . . . . . . . . . . . . . . 11
- 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11
- 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
- 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12
- 9.1. Normative References . . . . . . . . . . . . . . . . . . . 12
- 9.2. Informative References . . . . . . . . . . . . . . . . . . 12
- Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 13
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Bosch Expires September 4, 2014 [Page 2]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
-1. Introduction
-
- This document specifies an extension to the Sieve filtering language
- defined by RFC 5228 [SIEVE]. It adds a test to track whether or not
- a text string was seen before by the delivery agent in an earlier
- execution of the Sieve script. This can be used to detect and handle
- duplicate message deliveries.
-
- Duplicate deliveries are a common side-effect of being subscribed to
- a mailing list. For example, if a member of the list decides to
- reply to both the user and the mailing list itself, the user will
- often get one copy of the message directly and another through the
- mailing list. Also, if someone cross-posts over several mailing
- lists to which the user is subscribed, the user will likely receive a
- copy from each of those lists. In another scenario, the user has
- several redirected mail addresses all pointing to his main mail
- account. If one of the user's contacts sends the message to more
- than one of those addresses, the user will likely receive more than a
- single copy. Using the "duplicate" extension, users have the means
- to detect and handle such duplicates, e.g. by discarding them,
- marking them as "seen", or putting them in a special folder.
-
- Duplicate messages are normally detected using the Message-ID header
- field, which is required to be unique for each message. However, the
- "duplicate" test is flexible enough to use different criteria for
- defining what makes a message a duplicate, for example on the subject
- line or parts of the message body. Other applications of this new
- test command are also possible, as long as the tracked unique value
- is a string.
-
-
-2. Conventions Used in This Document
-
- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
- "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
- document are to be interpreted as described in [KEYWORDS].
-
- Conventions for notations are as in [SIEVE] Section 1.1, including
- use of the "Usage:" label for the definition of action and tagged
- arguments syntax.
-
-
-3. Test "duplicate"
-
- Usage: "duplicate" [":handle" <handle: string>]
- [":header" <header-name: string> /
- ":uniqueid" <value: string>]
- [":seconds" <timeout: number>] [":last"]
-
-
-
-Bosch Expires September 4, 2014 [Page 3]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- In its basic form, the "duplicate" test keeps track of which messages
- were seen before by this test during an earlier Sieve execution.
- Messages are by default identified by their message ID as contained
- in the Message-ID header. The "duplicate" test evaluates to "true"
- when the message was seen before and it evaluates to "false" when it
- was not.
-
- As a side-effect, the "duplicate" test adds the message ID to an
- internal duplicate tracking list once the Sieve execution finishes
- successfully. This way, the same test will evaluate to "true" during
- the next Sieve execution. Note that this side-effect is performed
- only when the "duplicate" test is actually evaluated. If the
- "duplicate" test is nested in a control structure or it is not the
- first item of an "allof" or "anyof" test list, its evaluation depends
- on the result of preceding tests, which may produce unexpected
- results.
-
- Implementations MUST only update the internal duplicate tracking list
- when the Sieve script execution finishes successfully. If failing
- script executions add the message ID to the duplicate tracking list,
- all "duplicate" tests in the Sieve script would erroneously yield
- "true" for the next delivery attempt of the same message, which can
- -- depending on the action taken for a duplicate -- easily lead to
- discarding the message without further notice.
-
- However, deferring the definitive modification of the tracking list
- to the end of a successful Sieve script execution is not without
- problems. It can cause a race condition when a duplicate message is
- delivered in parallel before the tracking list is updated. This way,
- a duplicate message could be missed by the "duplicate" test. More
- complex implementations could use a locking mechanism to prevent this
- problem. But, irrespective of what implementation is chosen,
- situations in which the "duplicate" test erroneously yields "true"
- MUST be prevented.
-
- The "duplicate" test MUST only check for duplicates amongst message
- ID values encountered in previous executions of the Sieve script; it
- MUST NOT consider ID values encountered earlier in the current Sieve
- script execution as potential duplicates. This means that all
- "duplicate" tests in a Sieve script execution, including those
- located in scripts included using the "include" [INCLUDE] extension,
- MUST always yield the same result if the arguments are identical.
-
- Implementations SHOULD limit the number of entries in the duplicate
- tracking list. When limiting the number of entries, implementations
- SHOULD discard the oldest ones first.
-
- Also, implementations SHOULD let entries in the tracking list expire
-
-
-
-Bosch Expires September 4, 2014 [Page 4]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- after a short period of time. The user can explicitly control the
- length of this expiration time by means of the ":seconds" argument,
- which accepts an integer value specifying the timeout value in
- seconds. If the ":seconds" argument is omitted, an appropriate
- default value MUST be used. A default expiration time of around 7
- days is usually appropriate. Sites SHOULD impose a maximum limit on
- the expiration time. If that limit is exceeded by the ":seconds"
- argument, the maximum value MUST silently be substituted; exceeding
- the limit MUST NOT produce an error. If the ":seconds" argument is
- zero, the "duplicate" test MUST yield "false" unconditionally.
-
- When the ":last" argument is omitted, the expiration time for entries
- in the duplicate tracking list MUST be measured relative to the
- moment at which the entry was first created; i.e., at the end of the
- successful script execution during which "duplicate" test returned
- "false" for a message with that particular message ID value. This
- means that subsequent duplicate messages have no influence on the
- time at which the entry in the duplicate tracking list finally
- expires.
-
- In contrast, when the ":last" argument is specified, the expiration
- time MUST be measured relative to the last script execution during
- which the "duplicate" test was used to check the entry's message ID
- value. This effectively means that the entry in the duplicate
- tracking will not expire while duplicate messages with the
- corresponding message ID keep being delivered within intervals
- smaller than the expiration time.
-
- By default, the content of the message's Message-ID header field is
- used as the unique ID for duplicate tracking. For more complex
- applications, the "duplicate" test can also be used to detect
- duplicate deliveries based on other message text. Then, the tracked
- unique ID can be an arbitrary string value extracted from the
- message. By adding the ":header" argument with a message header
- field name, the content of the specified header field can be used as
- the tracked unique ID instead of the default Message-ID header.
- Alternatively, the tracked unique ID can be specified explicitly
- using the ":uniqueid" argument. The ":header" and ":uniqueid"
- arguments are mutually exclusive and specifying both for a single
- "duplicate" test command MUST trigger an error.
-
- The syntax rules for the header name parameter of the ":header"
- argument are specified in Section 2.4.2.2 of RFC 5228 [SIEVE]. Note
- that implementations MUST NOT trigger an error for an invalid header
- name. Instead, the "duplicate" test MUST yield "false"
- unconditionally in this case. The parameter of the ":uniqueid"
- argument can be any string.
-
-
-
-
-Bosch Expires September 4, 2014 [Page 5]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- The Messsage-ID header field is assumed to be globally unique as
- required in Section 3.6.4 of RFC 5322 [IMAIL]. In practice, this
- assumption may not aways prove to be true. The "duplicate" tests
- does not deal with this situation implicitly, which means that false
- duplicates may be detected in this case. However, the user can
- address such situations by specifying an alternative means of message
- identification using the ":header" or the ":uniqueid" argument.
-
- If the tracked unique ID value is extracted directly from a message
- header field, i.e., when the ":uniqueid" argument is not used, the
- following operations MUST be performed before the actual duplicate
- verification:
-
- o Unfold the header line as described in [IMAIL] Section 2.2.3. (see
- also Section 2.4.2.2 of RFC 5228 [SIEVE]).
-
- o If possible, convert the header value to Unicode, encoded as UTF-8
- (see Section 2.7.2 of RFC 5228 [SIEVE]). If conversion is not
- possible, the value is left unchanged.
-
- o Trim leading and trailing whitespace from the header value (see
- Section 2.2 of RFC 5228 [SIEVE]).
-
- Note that these rules also apply to the Message-ID header field used
- by the basic "duplicate" test without a ":header" or ":uniqueid"
- argument. When the ":uniqueid" argument is used, such normalization
- concerns are the responsibility of the user.
-
- If the header field specified using the ":header" argument exists
- multiple times in the message, only the first occurrence MUST be used
- for duplicate tracking. If the specified header field is not present
- in the message, the "duplicate" test MUST yield "false"
- unconditionally. In that case the duplicate tracking list is left
- unmodified by this test, since no unique ID value is available. The
- same rules apply with respect to the Message-ID header field for the
- basic "duplicate" test without a ":header" or ":uniqueid" argument,
- since that header field could also be missing or occur multiple
- times.
-
- The string parameter of the ":uniqueid" argument can be composed from
- arbitrary text extracted from the message using the "variables"
- [VARIABLES] extension. To extract text from the message body, the
- "foreverypart" and "extracttext" [SIEVE-MIME] extensions need to be
- used as well. This provides the user with detailed control over what
- identifies a message as a duplicate.
-
- The tracked unique ID value MUST be matched case-sensitively,
- irrespective of whether it originates from a header or is specified
-
-
-
-Bosch Expires September 4, 2014 [Page 6]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- explicitly using the ":uniqueid" argument. To achieve case-
- insensitive behavior, the "set" command added by the "variables"
- [VARIABLES] extension can be used in combination with the ":uniqueid"
- argument to normalize the tracked unique ID value to upper or lower
- case.
-
- The "duplicate" test MUST track a unique ID value independent of its
- source. This means that it does not matter whether values are
- obtained from the message ID header, from an arbitrary header
- specified using the ":header" argument or explicitly from the
- ":uniqueid" argument. For example, the following three examples are
- equivalent and match the same entry in the duplicate tracking list:
-
- require "duplicate";
- if duplicate {
- discard;
- }
-
-
- require "duplicate";
- if duplicate :header "message-id" {
- discard;
- }
-
-
- require ["duplicate", "variables"];
- if header :matches "message-id" "*" {
- if duplicate :uniqueid "${0}" {
- discard;
- }
- }
-
- The ":handle" argument can be used to override this default behavior.
- The ":handle" argument separates a "duplicate" test from other
- duplicate tests with a different or omitted ":handle" argument.
- Using the ":handle" argument, unrelated "duplicate" tests can be
- prevented from interfering with each other: a message is only
- recognized as a duplicate when the tracked unique ID was seen before
- in an earlier script execution by a "duplicate" test with the same
- ":handle" argument.
-
- NOTE: The necessary mechanism to track duplicate messages is very
- similar to the mechanism that is needed for tracking duplicate
- responses for the "vacation" [VACATION] action. One way to implement
- the necessary mechanism for the "duplicate" test is therefore to
- store a hash of the tracked unique ID and, if provided, the ":handle"
- argument.
-
-
-
-
-Bosch Expires September 4, 2014 [Page 7]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
-3.1. Interaction with Other Sieve Extensions
-
- The "duplicate" test does not support either the "index"
- [DATE-INDEX], or "mime" [SIEVE-MIME] extensions directly, meaning
- that none of the ":index", ":mime" or associated arguments are added
- to the "duplicate" test when these extensions are active. The
- ":uniqueid" argument can be used in combination with the "variables"
- [VARIABLES] extension to achieve the same result indirectly.
-
- Normally, Sieve scripts are executed at final delivery. However,
- with the "imapsieve" [IMAPSIEVE] extension, Sieve scripts are invoked
- when the IMAP [IMAP] server performs operations on the message store,
- e.g. when messages are uploaded, flagged, or moved to another
- location. The "duplicate" test is devised for use at final delivery
- and the semantics in the "imapsieve" context are left undefined.
- Therefore it is NOT RECOMMENDED to allow the "duplicate" test to be
- used in the context of "imapsieve".
-
-
-4. Sieve Capability Strings
-
- A Sieve implementation that defines the "duplicate" test command will
- advertise the capability string "duplicate".
-
-
-5. Examples
-
-5.1. Example 1
-
- In this basic example, message duplicates are detected by tracking
- the Message-ID header. Duplicate deliveries are stored in a special
- folder contained in the user's Trash folder. If the folder does not
- exist, it is created automatically using the "mailbox" [MAILBOX]
- extension. This way, the user has a chance to recover messages when
- necessary. Messages that are not recognized as duplicates are stored
- in the user's inbox as normal.
-
- require ["duplicate", "fileinto", "mailbox"];
-
- if duplicate {
- fileinto :create "Trash/Duplicate";
- }
-
-5.2. Example 2
-
- This example shows a more complex use of the "duplicate" test. The
- user gets network alerts from a set of remote automated monitoring
- systems. Several notifications can be received about the same event
-
-
-
-Bosch Expires September 4, 2014 [Page 8]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- from different monitoring systems. The Message-ID of these messages
- is different, because these are all distinct messages from different
- senders. To avoid being notified more than a single time about the
- same event the user writes the following script:
-
- require ["duplicate", "variables", "imap4flags",
- "fileinto"];
-
- if header :matches "subject" "ALERT: *" {
- if duplicate :seconds 60 :uniqueid "${1}" {
- setflag "\\seen";
- }
- fileinto "Alerts";
- }
-
- The subjects of the notification message are structured with a
- predictable pattern which includes a description of the event. In
- the script above, the "duplicate" test is used to detect duplicate
- alert events. The message subject is matched against a pattern and
- the event description is extracted using the "variables" [VARIABLES]
- extension. If a message with that event in the subject was received
- before, but more than a minute ago, it is not detected as a duplicate
- due to the specified ":seconds" argument. In the the event of a
- duplicate, the message is marked as "seen" using the "imap4flags"
- [IMAP4FLAGS] extension. All alert messages are put into the "Alerts"
- mailbox irrespective of whether those messages are duplicates or not.
-
-5.3. Example 3
-
- This example shows how the "duplicate" test can be used to limit the
- frequency of notifications sent using the "enotify" [NOTIFY]
- extension. Consider the following scenario: a mail user receives
- XMPP notifications [NOTIFY-XMPP] about new mail through Sieve, but
- sometimes a single contact sends many messages in a short period of
- time. Now the user wants to prevent being notified of all of those
- messages. The user wants to be notified about messages from each
- person at most once per 30 minutes and writes the following script:
-
- require ["variables", "envelope", "enotify", "duplicate"];
-
- if envelope :matches "from" "*" { set "sender" "${1}"; }
- if header :matches "subject" "*" { set "subject" "${1}"; }
-
- if not duplicate :seconds 1800 :uniqueid "${sender}"
- {
- notify :message "[SIEVE] ${sender}: ${subject}"
- "xmpp:user@im.example.com";
- }
-
-
-
-Bosch Expires September 4, 2014 [Page 9]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- The example shown above uses the message envelope sender rather than
- the Message-ID header as the unique ID for duplicate tracking.
-
- The example can be extended to allow more messages from the same
- sender in close succession as long as the discussed subject is
- different. This can be achieved as follows:
-
- require ["variables", "envelope", "enotify", "duplicate"];
-
- if envelope :matches "from" "*" { set "sender" "${1}"; }
- if header :matches "subject" "*" { set "subject" "${1}"; }
-
- # account for 'Re:' prefix
- if string :comparator "i;ascii-casemap"
- :matches "${subject}" "Re:*"
- {
- set "subject" "${1}";
- }
- if not duplicate :seconds 1800
- :uniqueid "${sender} ${subject}"
- {
- notify :message "[SIEVE] ${sender}: ${subject}"
- "xmpp:user@im.example.com";
- }
-
- This uses a combination of the message envelope sender and the
- subject of the message as the unique ID for duplicate tracking.
-
-5.4. Example 4
-
- For this example, the mail user uses the "duplicate" test for two
- separate applications: for discarding duplicate events from a
- notification system and to mark certain follow-up messages in a
- software support mailing as "seen" using the "imap4flags"
- [IMAP4FLAGS] extension.
-
- The two "duplicate" tests in the following example each use a
- different header to identify messages. However, these "X-Event-ID"
- and "X-Ticket-ID headers can have similar values in this case (e.g.
- both based on a time stamp), meaning that one "duplicate" test can
- erroneously detect duplicates based on ID values tracked by the
- other. Therefore, the user wants to prevent the second "duplicate"
- test from matching ID values tracked by the first "duplicate" test
- and vice versa. This is achieved by specifying different ":handle"
- arguments for these tests.
-
-
-
-
-
-
-Bosch Expires September 4, 2014 [Page 10]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- require ["duplicate", "imap4flags"];
-
- if duplicate :header "X-Event-ID" :handle "notifier" {
- discard;
- }
- if allof (
- duplicate :header "X-Ticket-ID" :handle "support",
- address "to" "support@example.com",
- header :contains "subject" "fileserver")
- {
- setflag "\\seen";
- }
-
-
-6. Security Considerations
-
- A flood of unique messages could cause the list of tracked message ID
- values to grow indefinitely. Implementations SHOULD apply limits on
- the number and lifespan of entries in that list.
-
-
-7. IANA Considerations
-
- The following template specifies the IANA registration of the Sieve
- extension specified in this document:
-
- To: iana@iana.org
- Subject: Registration of new Sieve extension
-
- Capability name: duplicate
- Description: Adds test 'duplicate' that can be used to test
- whether a particular message is a duplicate;
- i.e., whether a copy of it was seen before by the
- delivery agent that is executing the Sieve
- script.
- RFC number: this RFC
- Contact address: Sieve mailing list <sieve@ietf.org>
-
- This information should be added to the list of sieve extensions
- given on http://www.iana.org/assignments/sieve-extensions.
-
-
-8. Acknowledgements
-
- Thanks to Cyrus Daboo, Arnt Gulbrandsen, Tony Hansen, Kristin Hubner,
- Alexey Melnikov, Subramanian Moonesamy, Tom Petch, Hector Santos,
- Robert Sparks, and Aaron Stone for reviews and suggestions. With
- special thanks to Ned Freed for his guidance and support.
-
-
-
-Bosch Expires September 4, 2014 [Page 11]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
-9. References
-
-9.1. Normative References
-
- [DATE-INDEX]
- Freed, N., "Sieve Email Filtering: Date and Index
- Extensions", RFC 5260, July 2008.
-
- [IMAIL] Resnick, P., Ed., "Internet Message Format", RFC 5322,
- October 2008.
-
- [IMAPSIEVE]
- Leiba, B., "Support for Internet Message Access Protocol
- (IMAP) Events in Sieve", RFC 6785, November 2012.
-
- [INCLUDE] Daboo, C. and A. Stone, "Sieve Email Filtering: Include
- Extension", RFC 6609, May 2012.
-
- [KEYWORDS]
- Bradner, S., "Key words for use in RFCs to Indicate
- Requirement Levels", BCP 14, RFC 2119, March 1997.
-
- [SIEVE] Guenther, P. and T. Showalter, "Sieve: An Email Filtering
- Language", RFC 5228, January 2008.
-
- [SIEVE-MIME]
- Hansen, T. and C. Daboo, "Sieve Email Filtering: MIME Part
- Tests, Iteration, Extraction, Replacement, and Enclosure",
- RFC 5703, October 2009.
-
- [VARIABLES]
- Homme, K., "Sieve Email Filtering: Variables Extension",
- RFC 5229, January 2008.
-
-9.2. Informative References
-
- [IMAP] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
- 4rev1", RFC 3501, March 2003.
-
- [IMAP4FLAGS]
- Melnikov, A., "Sieve Email Filtering: Imap4flags
- Extension", RFC 5232, January 2008.
-
- [MAILBOX] Melnikov, A., "The Sieve Mail-Filtering Language --
- Extensions for Checking Mailbox Status and Accessing
- Mailbox Metadata", RFC 5490, March 2009.
-
- [NOTIFY] Melnikov, A., Leiba, B., Segmuller, W., and T. Martin,
-
-
-
-Bosch Expires September 4, 2014 [Page 12]
-
-Internet-Draft Sieve: Detecting Duplicate Deliveries March 2014
-
-
- "Sieve Email Filtering: Extension for Notifications",
- RFC 5435, January 2009.
-
- [NOTIFY-XMPP]
- Saint-Andre, P. and A. Melnikov, "Sieve Notification
- Mechanism: Extensible Messaging and Presence Protocol
- (XMPP)", RFC 5437, January 2009.
-
- [VACATION]
- Showalter, T. and N. Freed, "Sieve Email Filtering:
- Vacation Extension", RFC 5230, January 2008.
-
-
-Author's Address
-
- Stephan Bosch
- Enschede
- NL
-
- Email: stephan@rename-it.nl
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Bosch Expires September 4, 2014 [Page 13]
-
diff --git a/doc/rfc/duplicate.rfc7352.txt b/doc/rfc/duplicate.rfc7352.txt
new file mode 100644
index 0000000000000000000000000000000000000000..8cbd2a7a8970230266964bc9fc685a226c7173d8
--- /dev/null
+++ b/doc/rfc/duplicate.rfc7352.txt
@@ -0,0 +1,843 @@
+
+
+
+
+
+
+Internet Engineering Task Force (IETF) S. Bosch
+Request for Comments: 7352 September 2014
+Category: Standards Track
+ISSN: 2070-1721
+
+
+ Sieve Email Filtering: Detecting Duplicate Deliveries
+
+Abstract
+
+ This document defines a new test command, "duplicate", for the Sieve
+ email filtering language. This test adds the ability to detect
+ duplications. The main application for this new test is handling
+ duplicate deliveries commonly caused by mailing list subscriptions or
+ redirected mail addresses. The detection is normally performed by
+ matching the message ID to an internal list of message IDs from
+ previously delivered messages. For more complex applications, the
+ "duplicate" test can also use the content of a specific header field
+ or other parts of the message.
+
+Status of This Memo
+
+ This is an Internet Standards Track document.
+
+ This document is a product of the Internet Engineering Task Force
+ (IETF). It represents the consensus of the IETF community. It has
+ received public review and has been approved for publication by the
+ Internet Engineering Steering Group (IESG). Further information on
+ Internet Standards is available in Section 2 of RFC 5741.
+
+ Information about the current status of this document, any errata,
+ and how to provide feedback on it may be obtained at
+ http://www.rfc-editor.org/info/rfc7352.
+
+Copyright Notice
+
+ Copyright (c) 2014 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents
+ (http://trustee.ietf.org/license-info) in effect on the date of
+ publication of this document. Please review these documents
+ carefully, as they describe your rights and restrictions with respect
+ to this document. Code Components extracted from this document must
+ include Simplified BSD License text as described in Section 4.e of
+ the Trust Legal Provisions and are provided without warranty as
+ described in the Simplified BSD License.
+
+
+
+Bosch Standards Track [Page 1]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+Table of Contents
+
+ 1. Introduction ....................................................2
+ 2. Conventions Used in This Document ...............................3
+ 3. Test "duplicate" ................................................3
+ 3.1. Arguments ":header" and ":uniqueid" ........................5
+ 3.2. Argument ":handle" .........................................7
+ 3.3. Arguments ":seconds" and ":last" ...........................8
+ 3.4. Interaction with Other Sieve Extensions ....................9
+ 4. Sieve Capability Strings ........................................9
+ 5. Examples ........................................................9
+ 5.1. Example 1 ..................................................9
+ 5.2. Example 2 .................................................10
+ 5.3. Example 3 .................................................11
+ 5.4. Example 4 .................................................12
+ 6. Security Considerations ........................................12
+ 7. IANA Considerations ............................................13
+ 8. Acknowledgements ...............................................14
+ 9. References .....................................................14
+ 9.1. Normative References ......................................14
+ 9.2. Informative References ....................................15
+
+1. Introduction
+
+ This document specifies an extension to the Sieve filtering language
+ defined by RFC 5228 [SIEVE]. It adds a test to track whether or not
+ a text string was seen before by the delivery agent in an earlier
+ execution of the Sieve script. This can be used to detect and handle
+ duplicate message deliveries.
+
+ Duplicate deliveries are a common side effect of being subscribed to
+ a mailing list. For example, if a member of the list decides to
+ reply to both the user and the mailing list itself, the user will
+ often get one copy of the message directly and another through the
+ mailing list. Also, if someone crossposts over several mailing lists
+ to which the user is subscribed, the user will likely receive a copy
+ from each of those lists. In another scenario, the user has several
+ redirected mail addresses all pointing to his main mail account. If
+ one of the user's contacts sends the message to more than one of
+ those addresses, the user will likely receive more than a single
+ copy. Using the "duplicate" extension, users have the means to
+ detect and handle such duplicates (e.g., by discarding them, marking
+ them as "seen", or putting them in a special folder).
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 2]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+ Duplicate messages are normally detected using the Message-ID header
+ field, which is required to be unique for each message. However, the
+ "duplicate" test is flexible enough to use different criteria for
+ defining what makes a message a duplicate (e.g., using the subject
+ line or parts of the message body). Other applications of this new
+ test command are also possible, as long as the tracked unique value
+ is a string.
+
+2. Conventions Used in This Document
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in [KEYWORDS].
+
+ Conventions for notations are as in Section 1.1 of [SIEVE], including
+ use of the "Usage:" label for the definition of action and tagged
+ arguments syntax.
+
+3. Test "duplicate"
+
+ Usage: "duplicate" [":handle" <handle: string>]
+ [":header" <header-name: string> /
+ ":uniqueid" <value: string>]
+ [":seconds" <timeout: number>] [":last"]
+
+ The "duplicate" test identifies the message by a "unique ID" and,
+ using that unique ID, keeps track of which messages were seen by a
+ "duplicate" test during an earlier Sieve execution. In its basic
+ form, the test gets the unique ID from the content of the message's
+ Message-ID header field. The "duplicate" test evaluates to "true"
+ if the message was seen before, and it evaluates to "false" if it
+ was not.
+
+ As a side effect, the "duplicate" test adds the unique ID to an
+ internal duplicate-tracking list once the Sieve execution finishes
+ successfully. The first time a particular unique ID is seen, the
+ message is not a duplicate, and the unique ID is added to the
+ tracking list. If a future Sieve execution sees a message whose
+ unique ID appears in the tracking list, that test will evaluate to
+ "true", and that message will be considered a duplicate.
+
+ Note that this side effect is performed only when the "duplicate"
+ test is actually evaluated. If the "duplicate" test is nested in a
+ control structure or if it is not the first item of an "allof" or
+ "anyof" test list, its evaluation depends on the result of preceding
+ tests, which may produce unexpected results.
+
+
+
+
+
+Bosch Standards Track [Page 3]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+ Implementations MUST only update the internal duplicate-tracking list
+ when the Sieve script execution finishes successfully. If failing
+ script executions add the unique ID to the duplicate-tracking list,
+ all "duplicate" tests in the Sieve script would erroneously yield
+ "true" for the next delivery attempt of the same message. This
+ can -- depending on the action taken for a duplicate -- easily lead
+ to discarding the message without further notice.
+
+ However, deferring the definitive modification of the tracking list
+ to the end of a successful Sieve script execution is not without
+ problems. It can cause a race condition when a duplicate message is
+ delivered in parallel before the tracking list is updated. This way,
+ a duplicate message could be missed by the "duplicate" test. More
+ complex implementations could use a locking mechanism to prevent this
+ problem. But, irrespective of what implementation is chosen,
+ situations in which the "duplicate" test erroneously yields "true"
+ MUST be prevented.
+
+ The "duplicate" test MUST only check for duplicates amongst unique ID
+ values encountered in previous executions of the Sieve script; it
+ MUST NOT consider ID values encountered earlier in the current Sieve
+ script execution as potential duplicates. This means that all
+ "duplicate" tests in a Sieve script execution, including those
+ located in scripts included using the "include" [INCLUDE] extension,
+ MUST always yield the same result if the arguments are identical.
+
+ The Message-ID header field is assumed to be globally unique as
+ required in Section 3.6.4 of RFC 5322 [IMAIL]. In practice, this
+ assumption may not always prove to be true. The "duplicate" test
+ does not deal with this situation, which means that false duplicates
+ may be detected in this case. However, the user can address such
+ situations by specifying an alternative means of message
+ identification using the ":header" or the ":uniqueid" argument, as
+ described in the next section.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 4]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+3.1. Arguments ":header" and ":uniqueid"
+
+ Duplicate tracking involves determining the unique ID for a
+ given message and checking whether that unique ID is in the
+ duplicate-tracking list. The unique ID for a message is
+ determined as follows:
+
+ o When neither the ":header" argument nor the ":uniqueid" argument
+ is used, the unique ID is the content of the message's Message-ID
+ header field.
+
+ o When the ":header" argument is used, the unique ID is the content
+ of the specified header field in the message. The header field
+ name is not part of the resulting unique ID; it consists only of
+ the field value.
+
+ o When the ":uniqueid" argument is used, the unique ID is the string
+ parameter that is specified with the argument.
+
+ The ":header" and ":uniqueid" arguments are mutually exclusive;
+ specifying both for a single "duplicate" test command MUST trigger an
+ error.
+
+ The syntax rules for the header name parameter of the ":header"
+ argument are specified in Section 2.4.2.2 of RFC 5228 [SIEVE]. Note
+ that implementations MUST NOT trigger an error for an invalid header
+ name. Instead, the "duplicate" test MUST yield "false"
+ unconditionally in this case. The parameter of the ":uniqueid"
+ argument can be any string.
+
+ If the tracked unique ID value is extracted directly from a message
+ header field (i.e., when the ":uniqueid" argument is not used), the
+ following operations MUST be performed before the actual duplicate
+ verification:
+
+ o Unfold the header line as described in Section 2.2.3 of RFC 5322
+ [IMAIL] (see also Section 2.4.2.2 of RFC 5228 [SIEVE]).
+
+ o If possible, convert the header value to Unicode, encoded as UTF-8
+ (see Section 2.7.2 of RFC 5228 [SIEVE]). If conversion is not
+ possible, the value is left unchanged.
+
+ o Trim leading and trailing whitespace from the header value (see
+ Section 2.2 of RFC 5228 [SIEVE]).
+
+
+
+
+
+
+
+Bosch Standards Track [Page 5]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+ Note that these rules also apply to the Message-ID header field used
+ by the basic "duplicate" test without a ":header" or ":uniqueid"
+ argument. When the ":uniqueid" argument is used, any normalization
+ needs to be done in the Sieve script itself as the unique ID is
+ created.
+
+ If the header field specified using the ":header" argument exists
+ multiple times in the message, extraction of the unique ID MUST use
+ only the first occurrence. This is true whether or not multiple
+ occurrences are allowed by Section 3.6 of RFC 5322 [IMAIL]. If the
+ specified header field is not present in the message, the "duplicate"
+ test MUST yield "false" unconditionally. In that case, the
+ duplicate-tracking list is left unmodified by this test, since no
+ unique ID value is available. The same rules apply with respect to
+ the Message-ID header field for the basic "duplicate" test without a
+ ":header" or ":uniqueid" argument, since that header field could also
+ be missing or occur multiple times.
+
+ The string parameter of the ":uniqueid" argument can be composed from
+ arbitrary text extracted from the message using the "variables"
+ [VARIABLES] extension. To extract text from the message body, the
+ "foreverypart" and "extracttext" [SIEVE-MIME] extensions need to be
+ used as well. This provides the user with detailed control over how
+ the message's unique ID is created.
+
+ The unique ID MUST be matched case-sensitively with the contents of
+ the duplicate-tracking list, irrespective of how the unique ID was
+ determined. To achieve case-insensitive behavior when the
+ ":uniqueid" argument is used, the "set" command added by the
+ "variables" [VARIABLES] extension can be used to normalize the unique
+ ID value to upper or lower case.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 6]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+3.2. Argument ":handle"
+
+ The "duplicate" test MUST track a unique ID value independent of its
+ source. This means that all values in the duplicate-tracking list
+ should be used for duplicate testing, regardless of whether they were
+ obtained from the Message-ID header field, from an arbitrary header
+ specified using the ":header" argument, or explicitly from the
+ ":uniqueid" argument. The following three examples are equivalent
+ and match the same entry in the duplicate-tracking list:
+
+ require "duplicate";
+ if duplicate {
+ discard;
+ }
+
+ require "duplicate";
+ if duplicate :header "message-id" {
+ discard;
+ }
+
+ require ["duplicate", "variables"];
+ if header :matches "message-id" "*" {
+ if duplicate :uniqueid "${0}" {
+ discard;
+ }
+ }
+
+ The ":handle" argument can be used to override this default behavior.
+ The ":handle" argument separates a "duplicate" test from other
+ "duplicate" tests with a different or omitted ":handle" argument.
+ Using the ":handle" argument, unrelated "duplicate" tests can be
+ prevented from interfering with each other: a message is only
+ recognized as a duplicate when the tracked unique ID was seen before
+ in an earlier script execution by a "duplicate" test with the same
+ ":handle" argument.
+
+ NOTE: The necessary mechanism to track duplicate messages is very
+ similar to the mechanism that is needed for tracking duplicate
+ responses for the "vacation" action [VACATION]. One way to
+ implement the necessary mechanism for the "duplicate" test is
+ therefore to store a hash of the tracked unique ID and, if
+ provided, the ":handle" argument.
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 7]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+3.3. Arguments ":seconds" and ":last"
+
+ Implementations SHOULD let entries in the tracking list expire after
+ a short period of time. The user can explicitly control the length
+ of this expiration time by means of the ":seconds" argument, which
+ accepts an integer value specifying the timeout value in seconds. If
+ the ":seconds" argument is omitted, an appropriate default value MUST
+ be used. A default expiration time of around 7 days is usually
+ appropriate. Sites SHOULD impose a maximum limit on the expiration
+ time. If that limit is exceeded by the ":seconds" argument, the
+ maximum value MUST be silently substituted; exceeding the limit MUST
+ NOT produce an error. If the ":seconds" argument is zero, the
+ "duplicate" test MUST yield "false" unconditionally.
+
+ When the ":last" argument is omitted, the expiration time for entries
+ in the duplicate-tracking list MUST be measured relative to the
+ moment at which the entry was first created (i.e., at the end of the
+ successful script execution during which the "duplicate" test
+ returned "false" for a message with that particular unique ID value).
+ This means that subsequent duplicate messages have no influence on
+ the time at which the entry in the duplicate-tracking list finally
+ expires.
+
+ In contrast, when the ":last" argument is specified, the expiration
+ time MUST be measured relative to the last script execution during
+ which the "duplicate" test was used to check the entry's unique ID
+ value. This effectively means that the entry in the duplicate-
+ tracking list will not expire while duplicate messages with the
+ corresponding unique ID keep being delivered within intervals smaller
+ than the expiration time.
+
+ It is possible to write Sieve scripts where, during a single
+ execution, more than one "duplicate" test is evaluated with the same
+ unique ID value and ":handle" argument but different ":seconds" or
+ ":last" arguments. The resulting behavior is left undefined by this
+ specification, so such constructs should be avoided. Implementations
+ MAY choose to use the ":seconds" and ":last" arguments from the
+ "duplicate" test that was evaluated last.
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 8]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+3.4. Interaction with Other Sieve Extensions
+
+ The "duplicate" test does not support either the "index" [DATE-INDEX]
+ or "mime" [SIEVE-MIME] extensions directly, meaning that none of the
+ ":index", ":mime", or associated arguments are added to the
+ "duplicate" test when these extensions are active. The ":uniqueid"
+ argument can be used in combination with the "variables" [VARIABLES]
+ extension to achieve the same result indirectly.
+
+ Normally, Sieve scripts are executed at final delivery. However,
+ with the "imapsieve" [IMAPSIEVE] extension, Sieve scripts are invoked
+ when the IMAP [IMAP] server performs operations on the message store
+ (e.g., when messages are uploaded, flagged, or moved to another
+ location). The "duplicate" test is devised for use at final
+ delivery, and the semantics in the "imapsieve" context are left
+ undefined. Therefore, implementations SHOULD NOT allow the
+ "duplicate" test to be used in the context of "imapsieve".
+
+4. Sieve Capability Strings
+
+ A Sieve implementation that defines the "duplicate" test command will
+ advertise the capability string "duplicate".
+
+5. Examples
+
+5.1. Example 1
+
+ In this basic example, message duplicates are detected by tracking
+ the Message-ID header field. Duplicate deliveries are stored in a
+ special folder contained in the user's Trash folder. If the folder
+ does not exist, it is created automatically using the "mailbox"
+ [MAILBOX] extension. This way, the user has a chance to recover
+ messages when necessary. Messages that are not recognized as
+ duplicates are stored in the user's inbox as normal.
+
+ require ["duplicate", "fileinto", "mailbox"];
+
+ if duplicate {
+ fileinto :create "Trash/Duplicate";
+ }
+
+
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 9]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+5.2. Example 2
+
+ This example shows a more complex use of the "duplicate" test. The
+ user gets network alerts from a set of remote automated monitoring
+ systems. Several notifications can be received about the same event
+ from different monitoring systems. The Message-ID header field of
+ these messages is different, because these are all distinct messages
+ from different senders. To avoid being notified more than a single
+ time about the same event, the user writes the following script:
+
+ require ["duplicate", "variables", "imap4flags",
+ "fileinto"];
+
+ if header :matches "subject" "ALERT: *" {
+ if duplicate :seconds 60 :uniqueid "${1}" {
+ setflag "\\seen";
+ }
+ fileinto "Alerts";
+ }
+
+ The subjects of the notification message are structured with a
+ predictable pattern that includes a description of the event. In the
+ script above, the "duplicate" test is used to detect duplicate alert
+ events. The message subject is matched against a pattern, and the
+ event description is extracted using the "variables" [VARIABLES]
+ extension. If a message with that event in the subject was received
+ before, but more than a minute ago, it is not detected as a duplicate
+ due to the specified ":seconds" argument. In the event of a
+ duplicate, the message is marked as "seen" using the "imap4flags"
+ [IMAP4FLAGS] extension. All alert messages are put into the "Alerts"
+ mailbox, irrespective of whether those messages are duplicates
+ or not.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 10]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+5.3. Example 3
+
+ This example shows how the "duplicate" test can be used to limit the
+ frequency of notifications sent using the "enotify" [NOTIFY]
+ extension. Consider the following scenario: a mail user receives
+ Extensible Messaging and Presence Protocol (XMPP) notifications
+ [NOTIFY-XMPP] about new mail through Sieve, but sometimes a single
+ contact sends many messages in a short period of time. Now the user
+ wants to prevent being notified of all of those messages. The user
+ wants to be notified about messages from each person at most once per
+ 30 minutes and writes the following script:
+
+ require ["variables", "envelope", "enotify", "duplicate"];
+
+ if envelope :matches "from" "*" { set "sender" "${1}"; }
+ if header :matches "subject" "*" { set "subject" "${1}"; }
+
+ if not duplicate :seconds 1800 :uniqueid "${sender}"
+ {
+ notify :message "[SIEVE] ${sender}: ${subject}"
+ "xmpp:user@im.example.com";
+ }
+
+ The example shown above uses the message envelope sender rather than
+ the Message-ID header field as the unique ID for duplicate tracking.
+
+ The example can be extended to allow more messages from the same
+ sender in close succession as long as the discussed subject is
+ different. This can be achieved as follows:
+
+ require ["variables", "envelope", "enotify", "duplicate"];
+
+ if envelope :matches "from" "*" { set "sender" "${1}"; }
+ if header :matches "subject" "*" { set "subject" "${1}"; }
+
+ # account for 'Re:' prefix
+ if string :comparator "i;ascii-casemap"
+ :matches "${subject}" "Re:*"
+ {
+ set "subject" "${1}";
+ }
+ if not duplicate :seconds 1800
+ :uniqueid "${sender} ${subject}"
+ {
+ notify :message "[SIEVE] ${sender}: ${subject}"
+ "xmpp:user@im.example.com";
+ }
+
+
+
+
+Bosch Standards Track [Page 11]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+ This uses a combination of the message envelope sender and the
+ subject of the message as the unique ID for duplicate tracking.
+
+5.4. Example 4
+
+ For this example, the mail user uses the "duplicate" test for two
+ separate applications: for discarding duplicate events from a
+ notification system and for marking certain follow-up messages in a
+ software support mailing as "seen" using the "imap4flags"
+ [IMAP4FLAGS] extension.
+
+ The two "duplicate" tests in the following example each use a
+ different header to identify messages. However, these "X-Event-ID"
+ and "X-Ticket-ID" headers can have similar values in this case (e.g.,
+ both based on a time stamp), meaning that one "duplicate" test can
+ erroneously detect duplicates based on ID values tracked by the
+ other. Therefore, the user wants to prevent the second "duplicate"
+ test from matching ID values tracked by the first "duplicate" test
+ and vice versa. This is achieved by specifying different ":handle"
+ arguments for these tests.
+
+ require ["duplicate", "imap4flags"];
+
+ if duplicate :header "X-Event-ID" :handle "notifier" {
+ discard;
+ }
+ if allof (
+ duplicate :header "X-Ticket-ID" :handle "support",
+ address "to" "support@example.com",
+ header :contains "subject" "fileserver")
+ {
+ setflag "\\seen";
+ }
+
+6. Security Considerations
+
+ A flood of unique messages could cause the duplicate-tracking list to
+ grow indefinitely. Therefore, implementations SHOULD limit the
+ number of entries in the duplicate-tracking list. When limiting the
+ number of entries, implementations SHOULD discard the oldest ones
+ first.
+
+ Scripts using the "duplicate" test evaluation should be aware that
+ message IDs are not necessarily unique, either through the fault of
+ benign generators or attackers injecting a message with the
+ properties used by the duplicate Sieve filter at some point prior to
+
+
+
+
+
+Bosch Standards Track [Page 12]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+ the Sieve filter. Therefore, scripts are well advised to be
+ conservative with respect to actions taken when duplicate messages
+ are identified only by message ID.
+
+ The list of unique IDs used for duplicate tracking can include
+ privacy-sensitive information, such as message ID values, content of
+ subject lines, and content extracted from message bodies.
+ Implementations SHOULD protect that information by obscuring it
+ through hashing (see the note at the end of Section 3.2) and/or by
+ storing it with a level of access control equivalent to that of the
+ messages themselves.
+
+ These measures will not prevent an entity that has access to the
+ duplicate-tracking list from querying whether messages with certain
+ unique ID values were received. As this operation is the essence of
+ the "duplicate" test, this cannot be prevented and may violate the
+ expectations of the user. For example, a user who deletes a message
+ from the server may expect that no record of it remains on the
+ server, but that will not be true if its message ID is persisted on
+ the server in the duplicate-tracking list.
+
+ It's notable, however, that server logs will often store the
+ information present on the duplicate-tracking list anyway and
+ probably would expose plaintext message IDs for a much longer period
+ than this mechanism would. Users of email services that
+ intentionally delete such logs with the intent of limiting
+ traceability should be made aware that use of the duplicate-tracking
+ mechanism re-exposes this information for the duration of the expiry
+ interval. Therefore, a shorter default expiry interval may be
+ appropriate in those situations.
+
+7. IANA Considerations
+
+ The following template specifies the IANA registration of the Sieve
+ extension specified in this document:
+
+ To: iana@iana.org
+ Subject: Registration of new Sieve extension
+
+ Capability name: duplicate
+ Description: Adds test 'duplicate' that can be used to test
+ whether a particular message is a duplicate,
+ i.e., whether a copy of it was seen before by
+ the delivery agent that is executing the Sieve
+ script.
+ RFC number: RFC 7352
+ Contact address: Sieve mailing list <sieve@ietf.org>
+
+
+
+
+Bosch Standards Track [Page 13]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+ This information has been added to the list of Sieve extensions given
+ on <http://www.iana.org/assignments/sieve-extensions>.
+
+8. Acknowledgements
+
+ Thanks to Brian Carpenter, Cyrus Daboo, Arnt Gulbrandsen, Tony
+ Hansen, Kristin Hubner, Barry Leiba, Alexey Melnikov, Subramanian
+ Moonesamy, Tom Petch, Hector Santos, Robert Sparks, Aaron Stone, and
+ Stefan Winter for reviews and suggestions. Special thanks to Ned
+ Freed for his guidance and support.
+
+9. References
+
+9.1. Normative References
+
+ [DATE-INDEX]
+ Freed, N., "Sieve Email Filtering: Date and Index
+ Extensions", RFC 5260, July 2008.
+
+ [IMAIL] Resnick, P., Ed., "Internet Message Format", RFC 5322,
+ October 2008.
+
+ [IMAPSIEVE]
+ Leiba, B., "Support for Internet Message Access Protocol
+ (IMAP) Events in Sieve", RFC 6785, November 2012.
+
+ [INCLUDE] Daboo, C. and A. Stone, "Sieve Email Filtering: Include
+ Extension", RFC 6609, May 2012.
+
+ [KEYWORDS]
+ Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+ [SIEVE] Guenther, P. and T. Showalter, "Sieve: An Email Filtering
+ Language", RFC 5228, January 2008.
+
+ [SIEVE-MIME]
+ Hansen, T. and C. Daboo, "Sieve Email Filtering: MIME Part
+ Tests, Iteration, Extraction, Replacement, and Enclosure",
+ RFC 5703, October 2009.
+
+ [VARIABLES]
+ Homme, K., "Sieve Email Filtering: Variables Extension",
+ RFC 5229, January 2008.
+
+
+
+
+
+
+
+Bosch Standards Track [Page 14]
+
+RFC 7352 Sieve: Detecting Duplicate Deliveries September 2014
+
+
+9.2. Informative References
+
+ [IMAP] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL -
+ VERSION 4rev1", RFC 3501, March 2003.
+
+ [IMAP4FLAGS]
+ Melnikov, A., "Sieve Email Filtering: Imap4flags
+ Extension", RFC 5232, January 2008.
+
+ [MAILBOX] Melnikov, A., "The Sieve Mail-Filtering Language --
+ Extensions for Checking Mailbox Status and Accessing
+ Mailbox Metadata", RFC 5490, March 2009.
+
+ [NOTIFY] Melnikov, A., Leiba, B., Segmuller, W., and T. Martin,
+ "Sieve Email Filtering: Extension for Notifications",
+ RFC 5435, January 2009.
+
+ [NOTIFY-XMPP]
+ Saint-Andre, P. and A. Melnikov, "Sieve Notification
+ Mechanism: Extensible Messaging and Presence Protocol
+ (XMPP)", RFC 5437, January 2009.
+
+ [VACATION] Showalter, T. and N. Freed, "Sieve Email Filtering:
+ Vacation Extension", RFC 5230, January 2008.
+
+Author's Address
+
+ Stephan Bosch
+ Enschede
+ NL
+
+ EMail: stephan@rename-it.nl
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Bosch Standards Track [Page 15]
+
diff --git a/doc/rfc/mailbox-metadata.rfc5490.txt b/doc/rfc/mailbox-metadata.rfc5490.txt
new file mode 100644
index 0000000000000000000000000000000000000000..8466a449ddf1680dbbf8383f54112b017fa8dfc5
--- /dev/null
+++ b/doc/rfc/mailbox-metadata.rfc5490.txt
@@ -0,0 +1,451 @@
+
+
+
+
+
+
+Network Working Group A. Melnikov
+Request for Comments: 5490 Isode Limited
+Category: Standards Track March 2009
+
+
+ The Sieve Mail-Filtering Language --
+Extensions for Checking Mailbox Status and Accessing Mailbox Metadata
+
+Status of This Memo
+
+ This document specifies an Internet standards track protocol for the
+ Internet community, and requests discussion and suggestions for
+ improvements. Please refer to the current edition of the "Internet
+ Official Protocol Standards" (STD 1) for the standardization state
+ and status of this protocol. Distribution of this memo is unlimited.
+
+Copyright Notice
+
+ Copyright (c) 2009 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents in effect on the date of
+ publication of this document (http://trustee.ietf.org/license-info).
+ Please review these documents carefully, as they describe your rights
+ and restrictions with respect to this document.
+
+ This document may contain material from IETF Documents or IETF
+ Contributions published or made publicly available before November
+ 10, 2008. The person(s) controlling the copyright in some of this
+ material may not have granted the IETF Trust the right to allow
+ modifications of such material outside the IETF Standards Process.
+ Without obtaining an adequate license from the person(s) controlling
+ the copyright in such materials, this document may not be modified
+ outside the IETF Standards Process, and derivative works of it may
+ not be created outside the IETF Standards Process, except to format
+ it for publication as an RFC or to translate it into languages other
+ than English.
+
+Abstract
+
+ This memo defines an extension to the Sieve mail filtering language
+ (RFC 5228) for accessing mailbox and server annotations, checking for
+ mailbox existence, and controlling mailbox creation on "fileinto"
+ action.
+
+
+
+
+
+
+Melnikov Standards Track [Page 1]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+Table of Contents
+
+ 1. Introduction ....................................................2
+ 2. Conventions Used in This Document ...............................2
+ 3. "mailbox" and "mboxmetadata" Extensions .........................2
+ 3.1. Test "mailboxexists" .......................................2
+ 3.2. ":create" Argument to "fileinto" Command ...................3
+ 3.3. Test "metadata" ............................................4
+ 3.4. Test "metadataexists" ......................................4
+ 4. "servermetadata" Extension ......................................5
+ 4.1. Test "servermetadata" ......................................5
+ 4.2. Test "servermetadataexists" ................................6
+ 5. Security Considerations .........................................6
+ 6. IANA Considerations .............................................7
+ 7. Acknowledgements ................................................7
+ 8. References ......................................................8
+ 8.1. Normative References .......................................8
+ 8.2. Informative References .....................................8
+
+1. Introduction
+
+ This memo defines an extension to the Sieve mail filtering language
+ [SIEVE] for accessing mailbox and server annotations. This allows
+ for customization of the Sieve engine behaviour based on variables
+ set using [METADATA].
+
+ This document also defines an extension for checking for mailbox
+ existence and for controlling mailbox creation on "fileinto" action.
+
+2. Conventions Used in This Document
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in [KEYWORDS].
+
+ Conventions for notations are as in [SIEVE] Section 1.1, including
+ the use of [ABNF].
+
+ This document is written with an assumption that readers are familiar
+ with the data model and terms defined in Section 3 of [METADATA].
+
+3. "mailbox" and "mboxmetadata" Extensions
+
+3.1. Test "mailboxexists"
+
+ Usage: mailboxexists <mailbox-names: string-list>
+
+
+
+
+
+Melnikov Standards Track [Page 2]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+ The "mailboxexists" test is true if all mailboxes listed in the
+ "mailbox-names" argument exist in the mailstore, and each allows the
+ user in whose context the Sieve script runs to "deliver" messages
+ into it. When the mailstore is an IMAP server, "delivery" of
+ messages is possible if:
+
+ a. the READ-WRITE response code is present for the mailbox (see
+ Section 7.1 of [IMAP]), if IMAP Access Control List (ACL)
+ [IMAPACL] is not supported by the server, or
+
+ b. the user has 'p' or 'i' rights for the mailbox (see Section 5.2
+ of [IMAPACL]).
+
+ Note that a successful "mailboxexists" test for a mailbox doesn't
+ necessarily mean that a "fileinto" action on this mailbox would
+ succeed. For example, the "fileinto" action might put user over
+ quota. The "mailboxexists" only verifies existence of the mailbox
+ and whether the user in whose context the Sieve script runs has
+ permissions to execute "fileinto" on it.
+
+ The capability string for use with the require command is "mailbox".
+
+ Example: The following example assumes that the Sieve engine also
+ supports "reject" [REJECT] and "fileinto" [SIEVE]. However, these
+ extensions are not required in order to implement the "mailbox"
+ extension.
+
+ require ["fileinto", "reject", "mailbox"];
+ if mailboxexists "Partners" {
+ fileinto "Partners";
+ } else {
+ reject "This message was not accepted by the Mailstore";
+ }
+
+3.2. ":create" Argument to "fileinto" Command
+
+ Usage: fileinto [:create] <mailbox: string>
+
+ If the optional ":create" argument is specified with "fileinto", it
+ instructs the Sieve interpreter to create the specified mailbox, if
+ needed, before attempting to deliver the message into the specified
+ mailbox. If the mailbox already exists, this argument is ignored.
+ Failure to create the specified mailbox is considered to be an error.
+
+ The capability string for use with the ":create" parameter is
+ "mailbox".
+
+
+
+
+
+Melnikov Standards Track [Page 3]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+3.3. Test "metadata"
+
+ Usage: metadata [MATCH-TYPE] [COMPARATOR]
+ <mailbox: string>
+ <annotation-name: string> <key-list: string-list>
+
+ This test retrieves the value of the mailbox annotation "annotation-
+ name" for the mailbox "mailbox" [METADATA]. The retrieved value is
+ compared to the "key-list". The test returns true if the annotation
+ exists and its value matches any of the keys.
+
+ The default match type is ":is" [SIEVE]. The default comparator is
+ "i;ascii-casemap" [SIEVE].
+
+ The capability string for use with the require command is
+ "mboxmetadata".
+
+ Annotations MUST be accessed with the permissions of the user in
+ whose context the Sieve script runs, and annotations starting with
+ the "/private" prefix MUST be those of the user in whose context the
+ Sieve script runs.
+
+ Example: The following example assumes that the Sieve engine also
+ supports the "vacation" [VACATION] extension. However, this
+ extension is not required in order to implement the "mboxmetadata"
+ extension.
+
+ require ["mboxmetadata", "vacation"];
+
+ if metadata :is "INBOX"
+ "/private/vendor/vendor.isode/auto-replies" "on" {
+
+ vacation text:
+ I'm away on holidays till March 2009.
+ Expect a delay.
+ .
+ }
+
+3.4. Test "metadataexists"
+
+ Usage: metadataexists <mailbox: string> <annotation-names: string-
+ list>
+
+ The "metadataexists" test is true if all of the annotations listed in
+ the "annotation-names" argument exist (i.e., have non-NIL values) for
+ the specified mailbox.
+
+
+
+
+
+Melnikov Standards Track [Page 4]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+ The capability string for use with the require command is
+ "mboxmetadata".
+
+4. "servermetadata" Extension
+
+4.1. Test "servermetadata"
+
+ Usage: servermetadata [MATCH-TYPE] [COMPARATOR]
+ <annotation-name: string> <key-list: string-list>
+
+ This test retrieves the value of the server annotation "annotation-
+ name" [METADATA]. The retrieved value is compared to the "key-list".
+ The test returns true if the annotation exists and its value matches
+ any of the keys.
+
+ The default match type is ":is". The default comparator is "i;ascii-
+ casemap".
+
+ The capability string for use with the require command is
+ "servermetadata".
+
+ Annotations MUST be accessed with the permissions of the user in
+ whose context the Sieve script runs, and annotations starting with
+ the "/private" prefix MUST be those of the user in whose context the
+ Sieve script runs.
+
+ Example: The following example assumes that the Sieve engine also
+ supports "variables" [VARIABLES], "enotify" [NOTIFY], and "envelope"
+ [SIEVE] extensions. However, these extensions are not required in
+ order to implement the "servermetadata" extension.
+
+ require ["enotify", "servermetadata", "variables", "envelope"];
+
+ if servermetadata :matches
+ "/private/vendor/vendor.isode/notification-uri" "*" {
+ set "notif_uri" "${0}";
+ }
+
+ if not string :is "${notif_uri}" "none" {
+ # :matches is used to get the MAIL FROM address
+ if envelope :all :matches "from" "*" {
+ set "env_from" " [really: ${1}]";
+ }
+
+ # :matches is used to get the value of the Subject header
+ if header :matches "Subject" "*" {
+ set "subject" "${1}";
+ }
+
+
+
+Melnikov Standards Track [Page 5]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+ # :matches is used to get the address from the From header
+ if address :matches :all "from" "*" {
+ set "from_addr" "${1}";
+ }
+
+ notify :message "${from_addr}${env_from}: ${subject}"
+ "${notif_uri}";
+ }
+
+4.2. Test "servermetadataexists"
+
+ Usage: servermetadataexists
+ <annotation-names: string-list>
+
+ The "servermetadataexists" test is true if all of the server
+ annotations listed in the "annotation-names" argument exist (i.e.,
+ have non-NIL values).
+
+ The capability string for use with the require command is
+ "servermetadata".
+
+5. Security Considerations
+
+ Extensions defined in this document deliberately don't provide a way
+ to modify annotations.
+
+ A failure to retrieve data due to the server storing the annotations
+ being down or otherwise inaccessible may alter the result of Sieve
+ processing. So implementations SHOULD treat a temporary failure to
+ retrieve annotations in the same manner as a temporary failure to
+ retrieve a Sieve script. For example, if the Sieve script is stored
+ in the Lightweight Directory Access Protocol (LDAP) and the script
+ can't be retrieved when a message is processed, then the agent
+ performing Sieve processing can, for example, assume that the script
+ doesn't exist or delay message delivery until the script can be
+ retrieved successfully. Annotations should be treated as if they are
+ a part of the script itself, so a temporary failure to retrieve them
+ should be handled in the same way as a temporary failure to retrieve
+ the Sieve script itself.
+
+ Protocols/APIs used to retrieve annotations MUST provide at least the
+ same level of confidentiality as protocols/APIs used to retrieve
+ Sieve scripts. For example, if Sieve scripts are retrieved using
+ LDAP secured with Transport Layer Security (TLS) encryption, then the
+ protocol used to retrieve annotations must use a comparable mechanism
+ for providing connection confidentiality. In particular, the
+ protocol used to retrieve annotations must not be lacking encryption.
+
+
+
+
+Melnikov Standards Track [Page 6]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+6. IANA Considerations
+
+ IANA has added the following registrations to the list of Sieve
+ extensions:
+
+ To: iana@iana.org
+ Subject: Registration of new Sieve extension
+ Capability name: mailbox
+ Description: adds test for checking for mailbox existence and a new
+ optional argument to fileinto for creating a mailbox
+ before attempting mail delivery.
+ RFC number: this RFC
+ Contact address:
+ The Sieve discussion list <ietf-mta-filters@imc.org>
+
+ Capability name: mboxmetadata
+ Description: adds tests for checking for mailbox metadata item
+ existence and for retrieving of a mailbox metadata
+ value.
+ RFC number: this RFC
+ Contact address:
+ The Sieve discussion list <ietf-mta-filters@imc.org>
+
+ Capability name: servermetadata
+ Description: adds tests for checking for server metadata item
+ existence and for retrieving of a server metadata
+ value.
+ RFC number: this RFC
+ Contact address:
+ The Sieve discussion list <ietf-mta-filters@imc.org>
+
+7. Acknowledgements
+
+ Thanks to Cyrus Daboo for initial motivation for this document.
+
+ Thanks to Barry Leiba, Randall Gellens, and Aaron Stone for helpful
+ comments on this document.
+
+ The author also thanks the Open Mobile Alliance's Mobile Email
+ working group for providing a set of requirements for mobile devices,
+ guiding some of the extensions in this document.
+
+
+
+
+
+
+
+
+
+
+Melnikov Standards Track [Page 7]
+
+RFC 5490 Sieve METADATA March 2009
+
+
+8. References
+
+8.1. Normative References
+
+ [ABNF] Crocker, D. and P. Overell, "Augmented BNF for Syntax
+ Specifications: ABNF", STD 68, RFC 5234, January 2008.
+
+ [IMAP] Crispin, M., "Internet Message Access Protocol - Version
+ 4rev1", RFC 3501, March 2003.
+
+ [IMAPACL] Melnikov, A., "IMAP4 Access Control List (ACL)
+ Extension", RFC 4314, December 2005.
+
+ [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+ [METADATA] Daboo, C., "The IMAP METADATA Extension", RFC 5464,
+ February 2009.
+
+ [SIEVE] Guenther, P. and T. Showalter, "Sieve: An Email
+ Filtering Language", RFC 5228, January 2008.
+
+8.2. Informative References
+
+ [NOTIFY] Melnikov, A., Leiba, B., Segmuller, W., and T. Martin,
+ "Sieve Email Filtering: Extension for Notifications",
+ RFC 5435, January 2009.
+
+ [REJECT] Stone, A., "Sieve Email Filtering: Reject and Extended
+ Reject Extensions", RFC 5429, March 2009.
+
+ [VACATION] Showalter, T. and N. Freed, "Sieve Email Filtering:
+ Vacation Extension", RFC 5230, January 2008.
+
+ [VARIABLES] Homme, K., "Sieve Email Filtering: Variables Extension",
+ RFC 5229, January 2008.
+
+Author's Address
+
+ Alexey Melnikov
+ Isode Limited
+ 5 Castle Business Village
+ 36 Station Road
+ Hampton, Middlesex TW12 2BX
+ UK
+
+ EMail: Alexey.Melnikov@isode.com
+
+
+
+
+Melnikov Standards Track [Page 8]
+
diff --git a/doc/rfc/spec-bosch-sieve-duplicate.txt b/doc/rfc/spec-bosch-sieve-duplicate.txt
deleted file mode 100644
index 246e26b2bf3ac843ef8075d7c54c86cb6750ed31..0000000000000000000000000000000000000000
--- a/doc/rfc/spec-bosch-sieve-duplicate.txt
+++ /dev/null
@@ -1,392 +0,0 @@
-
-
-
-Pigeonhole Project S. Bosch
- October 17, 2012
-
-
- Sieve Email Filtering: Detecting Duplicate Deliveries
-
-Abstract
-
- This document defines a new vendor-defined test command "duplicate"
- for the "Sieve" email filtering language. It can be used to test
- whether a particular string value is a duplicate, i.e. whether it was
- seen before by the delivery agent that is executing the Sieve script.
- The main application for this new test is detecting duplicate message
- deliveries commonly caused by mailing list subscriptions or
- redirected mail addresses.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Bosch [Page 1]
-
- Sieve: Detecting Duplicate Deliveries October 2012
-
-
-Table of Contents
-
- 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
- 2. Conventions Used in This Document . . . . . . . . . . . . . . . 3
- 3. Test "duplicate" . . . . . . . . . . . . . . . . . . . . . . . 4
- 4. Sieve Capability Strings . . . . . . . . . . . . . . . . . . . 5
- 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
- 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 6
- 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 7
- 7.1. Normative References . . . . . . . . . . . . . . . . . . . 7
- 7.2. Informative References . . . . . . . . . . . . . . . . . . 7
- Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 7
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Bosch [Page 2]
-
- Sieve: Detecting Duplicate Deliveries October 2012
-
-
-1. Introduction
-
- This is an extension to the Sieve filtering language defined by RFC
- 5228 [SIEVE]. It adds a test to determine whether a certain string
- value was seen before by the delivery agent in an earlier execution
- of the Sieve script. This can be used to detect and handle duplicate
- message deliveries.
-
- Duplicate deliveries are a common side-effect of being subscribed to
- a mailing list. For example, if a member of the list decides to
- reply to both the user and the mailing list itself, the user will get
- a copy of the message directly and through mailing list. Also, if
- someone cross-posts over several mailing lists to which the user is
- subscribed, the user will receive a copy from each of those lists.
- In another scenario, the user has several redirected mail addresses
- all pointing to his main mail account. If one of the user's contacts
- sends the message to more than one of those addresses, the user will
- receive more than a single copy. Using the "vnd.dovecot.duplicate"
- extension, users have the means to detect and handle such duplicates,
- e.g. by discarding them or putting them in a special folder.
-
- Duplicate messages are normally detected using the Message-ID header
- field, which is required to be unique for each message. However, the
- "duplicate" test is flexible enough to use different (weaker)
- criteria for defining what makes a message a duplicate, for example
- based on the subject line. Also, other applications of this new test
- command are possible, as long as the tracked value is a string.
-
- This extension is specific to the Pigeonhole Sieve implementation for
- the Dovecot Secure IMAP server. It will therefore most likely not be
- supported by web interfaces and GUI-based Sieve editors.
-
-
-2. Conventions Used in This Document
-
- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
- "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
- document are to be interpreted as described in [KEYWORDS].
-
- Conventions for notations are as in [SIEVE] Section 1.1, including
- use of the "Usage:" label for the definition of action and tagged
- arguments syntax.
-
-
-
-
-
-
-
-
-
-Bosch [Page 3]
-
- Sieve: Detecting Duplicate Deliveries October 2012
-
-
-3. Test "duplicate"
-
- Usage: "duplicate" [":seconds" <timeout: number>]
- [":header" <header-name: string> /
- ":value" <value: string>]
- [":handle" <handle: string>]
-
- The "duplicate" test keeps track of which values were seen before by
- this test in an earlier execution of this Sieve script. In its basic
- form, the tested value is the content of the Message-ID header of the
- message. This way, this test can be used to detect duplicate
- deliveries of the same message. It can also detect duplicate
- deliveries based on other message header fields if requested and it
- can even use a user-provided string value, e.g. as composed from text
- extracted from the message using the "variables" [VARIABLES]
- extension.
-
- The "duplicate" test evaluates to "true" when the provided value was
- seen before in an earlier Sieve execution for a previous message
- delivery. If the value was not seen earlier, the test evaluates to
- "false".
-
- As a side-effect, the "duplicate" test adds the evaluated value to an
- internal duplicate tracking list, so that the test will evaluate to
- "true" the next time the Sieve script is executed and the same value
- is encountered. Note that the "duplicate" test MUST only check for
- duplicates amongst values encountered in previous executions of the
- Sieve script; it MUST NOT consider values encountered earlier in the
- current Sieve script execution as potential duplicates. This means
- that all "duplicate" tests in a Sieve script execution, including
- those located in scripts included using the "include" [INCLUDE]
- extension, MUST yield the same result if the arguments are identical.
-
- Implementations MUST prevent adding values to the internal duplicate
- tracking list when the Sieve script execution fails. For example,
- this can be implemented by deferring the definitive modification of
- the tracking list to the end of the Sieve script execution. If
- failed script executions would add values to the duplicate tracking
- list, all "duplicate" tests would erroneously yield "true" for the
- next delivery attempt of the same message, which can -- depending on
- the action taken for a duplicate -- easily lead to discarding the
- message without further notice.
-
- Implementations SHOULD limit the number of values (and thereby
- messages) that are tracked. Also, implementations SHOULD let entries
- in the value tracking list expire after a short period of time. The
- user can explicitly control the length of this expiration time by
- means of the ":seconds" argument. If the ":seconds" argument is
-
-
-
-Bosch [Page 4]
-
- Sieve: Detecting Duplicate Deliveries October 2012
-
-
- omitted, an appropriate default MUST be used. Sites SHOULD impose a
- maximum limit on the expiration time. If that limit is exceeded, the
- maximum value MUST silently be substituted; exceeding the limit MUST
- NOT produce an error.
-
- By default, the tracked value is the content of the message's
- Message-ID header field. For more advanced purposes, the content of
- another header can be chosen for tracking by specifying the ":header"
- argument. The tracked string value can also be specified explicitly
- using the ":value" argument. The ":header" and ":value" arguments
- are mutually exclusive and specifying both for a single "duplicate"
- test command MUST trigger an error at compile time. If the value is
- extracted from a header, i.e. when the ":value" argument is not used,
- leading and trailing whitespace (see Section 2.2 of RFC 5228 [SIEVE])
- MUST first be trimmed from the value before performing the actual
- duplicate verification.
-
- Using the ":handle" argument, the duplicate test can be employed for
- multiple independent purposes. Only when the tracked value was seen
- before in an earlier script execution by a "duplicate" test with the
- same ":handle" argument, it is recognized as a duplicate.
-
- NOTE: The necessary mechanism to track duplicate messages is very
- similar to the mechanism that is needed for tracking duplicate
- responses for the "vacation" [VACATION] action. One way to implement
- the necessary mechanism for the "duplicate" test is therefore to
- store a hash of the tracked value and, if provided, the ":handle"
- argument.
-
-
-4. Sieve Capability Strings
-
- A Sieve implementation that defines the "duplicate" test command will
- advertise the capability string "vnd.dovecot.duplicate".
-
-
-5. Examples
-
- In the following basic example, message duplicates are detected by
- tracking the Message-ID header. Duplicate deliveries are stored in a
- special folder contained in the user's Trash folder. If the folder
- does not exist, it is created automatically using the "mailbox"
- [MAILBOX] extension. This way, the user has a chance to recover
- messages when necessary. Messages that are not recognized as
- duplicates are stored in the user's inbox as normal.
-
-
-
-
-
-
-Bosch [Page 5]
-
- Sieve: Detecting Duplicate Deliveries October 2012
-
-
- require ["vnd.dovecot.duplicate", "fileinto", "mailbox"];
-
- if duplicate {
- fileinto :create "Trash/Duplicate";
- }
-
- The next example shows a more complex use of the "duplicate" test.
- The user gets network alerts from a set of remote automated
- monitoring systems. Multiple notifications can be received about the
- same event from different monitoring systems. The Message-ID of
- these messages is different, because these are all distinct messages
- from different senders. To avoid being notified multiple times about
- the same event the user writes the following script:
-
- require ["vnd.dovecot.duplicate", "variables", "imap4flags",
- "fileinto"];
-
- if header :matches "subject" "ALERT: *" {
- if duplicate :seconds 60 :value "${1}" {
- setflag "\\seen";
- }
- fileinto "Alerts";
- }
-
- The subjects of the notification message are structured with a
- predictable pattern which includes a description of the event. In
- the script above the "duplicate" test is used to detect duplicate
- alert events. The message subject is matched against a pattern and
- the event description is extracted using the "variables" [VARIABLES]
- extension. If a message with that event in the subject was received
- before, but more than a minute ago, it is not detected as a duplicate
- due to the specified ":seconds" argument. In the the event of a
- duplicate, the message is marked as seen using the "imap4flags"
- [IMAP4FLAGS] extension. All alert messages are put into the "Alerts"
- mailbox irrespective of whether those messages are duplicates or not.
-
-
-6. Security Considerations
-
- A flood of unique messages could cause the list of tracked values to
- grow indefinitely. Implementations therefore SHOULD implement limits
- on the number and lifespan of entries in that list.
-
-
-7. References
-
-
-
-
-
-
-Bosch [Page 6]
-
- Sieve: Detecting Duplicate Deliveries October 2012
-
-
-7.1. Normative References
-
- [INCLUDE] Daboo, C. and A. Stone, "Sieve Email Filtering: Include
- Extension", RFC 6609, May 2012.
-
- [KEYWORDS]
- Bradner, S., "Key words for use in RFCs to Indicate
- Requirement Levels", BCP 14, RFC 2119, March 1997.
-
- [SIEVE] Guenther, P. and T. Showalter, "Sieve: An Email Filtering
- Language", RFC 5228, January 2008.
-
-7.2. Informative References
-
- [IMAP4FLAGS]
- Melnikov, A., "Sieve Email Filtering: Imap4flags
- Extension", RFC 5232, January 2008.
-
- [MAILBOX] Melnikov, A., "The Sieve Mail-Filtering Language --
- Extensions for Checking Mailbox Status and Accessing
- Mailbox Metadata", RFC 5490, March 2009.
-
- [VACATION]
- Showalter, T. and N. Freed, "Sieve Email Filtering:
- Vacation Extension", RFC 5230, January 2008.
-
- [VARIABLES]
- Homme, K., "Sieve Email Filtering: Variables Extension",
- RFC 5229, January 2008.
-
-
-Author's Address
-
- Stephan Bosch
- Enschede
- NL
-
- Email: stephan@rename-it.nl
-
-
-
-
-
-
-
-
-
-
-
-
-
-Bosch [Page 7]
-