Pigeonhole for Dovecot v2.0

(NOTE: This is the UNSTABLE development branch)

Introduction
============

This package provides Sieve support for the Dovecot Secure IMAP server. Sieve is
a machine language specifically tailored for internet message filtering. By
writing Sieve scripts, users can customize how messages are delivered, e.g. 
whether they are forwarded or stored in special folders. The Sieve language is 
meant to be simple, extensible and system independent. And, unlike most other 
mail filtering script languages, it does not allow users to execute arbitrary 
programs. This is particularly useful to prevent virtual users from having full 
access to the mail store. The intention of the language is to make it impossible
for users to do anything more complex (and dangerous) than write simple mail
filters.

Support for filtering messages through Sieve scripts is provided at message
delivery. This is implemented as a plugin for Dovecot's Local Delivery Agent
(LDA). 

This package also adds a ManageSieve service to Dovecot, which can be used to
let users manage their Sieve script remotely. It has the following advantages 
over managing these directly via the filesystem:

  * There is no need to let users log in via FTP/SFTP/etc, which could be 
    difficult and undesirable, especially with virtual users.

  * ManageSieve is a standard protocol (although still a draft), so users can
    manage their scripts using (hopefully) user-friendly ManageSieve clients.
    Many webmails already include a ManageSieve client.

  * Scripts are compiled before they are installed, which guarantees that the
    uploaded script is valid, preventing a user from inadvertently installing a
    broken Sieve script.

More information about the Pigeonhole project can be found at:

  http://pigeonhole.dovecot.org

Features
========

  * The Pigeonhole Sieve implementation aims to be admin- and user-friendly. 
    Much like Dovecot itself, common error messages are made as easily 
    understandable as possible. Any crash, no matter how it happened, is 
    considered a bug that will be fixed. The compiler does not bail on the first
    error, but it looks for more script errors to make debugging more efficient.

  * The Pigeonhole Sieve implementation is, much like the Sieve language itself,
    highly extensible with new Sieve capabilities. This includes support for
    third-party plugins. It should eventually provide the necessary 
    infrastructure for at least all currently known relevant (proposed) Sieve
    extensions. The goal is to keep the extension interface provided by the
    Sieve implementation as generic as possible, i.e. without explicit support
    for specific extensions. New similar extensions can then use the same
    interface methods without changes to the Sieve engine code. If an extension
    is not loaded using the require command, the compiler truly does not know of
    its existence.

  * The Pigeonhole Sieve plugin is backwards compatible with the old CMUSieve
    plugin, which provided Sieve support for older versions of Dovecot. All
    Sieve extensions supported by the old plugin are also supported by the 
    Pigeonhole Sieve plugin, including those that are now considered to be 
    deprecated.

  * The Pigeonhole Sieve implementation supports executing multiple Sieve
    scripts sequentially. Using this feature it is possible to execute
    administrator-controlled Sieve scripts before and after the user's personal
    Sieve script, guaranteeing that responses and message deliveries are never
    duplicated. This implementation is based on a draft specification 
    (http://tools.ietf.org/html/draft-degener-sieve-multiscript-00), which 
    defines the Sieve behavior when multiple scripts are executed sequentially
    on the same message.

  * The Pigeonhole Sieve implementation includes a test suite to automatically
    assess whether the compiled Sieve engine works correctly. The test suite is
    an extension to the Sieve language and is therefore easily extended with new
    tests. Currently, the test suite is mostly limited to testing script
    processing. The performed actions are not tested fully yet.

  * The Pigeonhole Sieve implementation supports the new and very useful
    variables extension, which allows maintaining state information throughout
    a Sieve script across subsequent rules.

  * The Pigeonhole Sieve plugin is distributed with a sieve-test tool that 
    simplifies testing Sieve scripts and provides additional debugging 
    facilities. 

Sieve Implementation Status
===========================

The core of the language (as specified in RFC 5228) is fully supported. In
addition to that, this Sieve implementation features various extensions. The
following list outlines the implementation status of each supported extension:

  The language extensions defined in the base specification are fully supported:

    encoded-character (RFC 5228; page 10)
    fileinto (RFC 5228; page 23)
    envelope (RFC 5228; page 27)

  The following Sieve language extensions are also supported:

    copy (RFC 3894): fully supported
    body (RFC 5173): almost fully supported, but the text body-transform
      implementation is simple and some issues make it still not completely
      RFC compliant.
    environment (RFC 5183): basic support is provided (v0.1.5+)
    variables (RFC 5229): fully supported
    vacation (RFC 5230): fully supported
    relational (RFC 5231): fully supported
    imap4flags (RFC 5232): fully supported
    subaddress (RFC 5233): fully supported, but with limited configurability
    date (RFC 5260; page 3): fully supported (v0.1.12+)
    reject (RFC 5429; page 6): fully supported
    enotify (RFC 5435): fully supported (v0.1.3+), but only the mailto
      notification mechanism is available
    mailbox (RFC 5490; page 2): fully supported (v0.1.10+), but
      ACL permissions are not verified for mailboxexists
    include (draft): fully supported, but the interaction with
      ManageSieve is not according to specification.
    regex (draft): almost fully supported, but UTF-8 is not supported.

  The following deprecated extensions are supported for backwards
  compatibility:

    imapflags (obsolete draft): fully backwards compatible (v0.1.3+)
    notify (obsolete draft): fully backwards compatible (v0.1.15+)

    The availability of these deprecated extensions is disabled by default.

  The following extensions are under development:

    ereject (RFC 5429; page 4): implemented, but currently equal to reject
    spamtest and virustest (RFC 5235): implemented, but needs thorough testing

  Many more extensions to the language exist. Not all of these extensions are
  useful for Dovecot in particular, but many of them are. Currently, the
  author has taken notice of the following extensions:

    index (RFC 5260; page 7): planned
    editheader (RFC 5293): planned
    foreverypart, mime, replace, enclose, and extracttext (RFC 5703): planned

    These extensions will be added as soon as the necessary infrastructure is
    available.

Compiling and Configuring
=========================

Refer to INSTALL file.

Sieve Tools
===========

To test the sieve engine outside deliver, it is useful to try the commands that 
exist in the src/sieve-tools/ directory of this package. After installation, 
these are available at your $prefix/bin directory. The following commands are 
installed:

sievec     - Compiles sieve scripts into a binary representation for later 
             execution. Refer to the next section on manually compiling Sieve 
             scripts.

sieve-test - This is a universal Sieve test tool for testing the effect of a
             Sieve script on a particular message. It allows compiling, running 
             and testing Sieve scripts. It can either be used to display the
             actions that would be performed on the provided test message or it
             can be used to test the actual delivery of the message and show the
             messages that would normally be sent through SMTP.

sieved     - Dumps the content of a Sieve binary file for (development) 
             debugging purposes.

When installed, man pages are also available for these commands. In this package
the man pages are present in doc/man and can be viewed before install using 
e.g.: 

man -l doc/man/sieve-test.1

Various example scripts are bundled in the directory 'examples'. These scripts
were downloaded from various locations. View the top comment in the scripts for 
url and author information.

Compiling Sieve Scripts
=======================

When the Sieve plugin executes a script for the first time (or after it has been
changed), it's compiled into into a binary form. Pigeonhole Sieve implementation
uses the .svbin extension to store compiled Sieve scripts (e.g. .dovecot.svbin).
To store the binary, the plugin needs write access in the directory in which the
script is located.

A problem occurs when a global script is encountered by the plugin. For security
reasons, global script directories are not supposed to be writable by the user.
Therefore, the plugin cannot store the binary when the script is first compiled.
Note that this doesn't mean that the old compiled version of the script is used
when the binary cannot be written: it compiles and uses the current script
version. The only real problem is that the plugin will not be able to update
the binary on disk, meaning that the global script needs to be recompiled each
time it needs to be executed, i.e. for every incoming message, which is
inefficient.

To mitigate this problem, the administrator must manually pre-compile global
scripts using the sievec command line tool. For example:

sievec /var/lib/dovecot/sieve/global/

This is necessary for scripts listed in the sieve_global_path, sieve_before and
sieve_after settings. For global scripts that are only included in other scripts
using the include extension, this step is not necessary, since included scripts
are incorporated into the binary produced for the main script.

When manually compiling scripts with sievec, if those scripts use the include
sieve extension and your sieve_dir is not the sieve subfolder of the directory
of the main file, you can specify it by defining the SIEVE_DIR environment
variable (e.g SIEVE_DIR=~/.sieve sievec .dovecot.sieve )

Compile and Runtime Logging
===========================

Log messages produced at runtime by the Sieve plugin are written to two
locations:

  * A log file is written in the same directory as the user's main private 
    script (as specified by the sieve setting). This log file bears the name of
    that script file appended with ".log", e.g. .dovecot.sieve.log. If there are
    errors or warnings in the script, the messages are appended to that log file
    until it eventually grows too large. When that happens, the old log file is
    rotated to a ".log.0" file and an empty log file is started. Informational
    messages are not written to this log file and the log file is not created
    until messages are actually logged, i.e. when an error or warning is
    produced.

  * Messages that could be of interest to the system administrator are also
    written to the Dovecot logging facility (usually syslog). This includes
    informational messages that indicate what actions are executed on incoming
    messages. Compile errors encountered in the user's private script are not
    logged here.

The ManageSieve service reports compile errors and warnings only back to the 
user. System and configuration-related messages are written to the Dovecot 
logging facility. 

Known issues
============

Sieve
-----

Most open issues are outlined in the TODO file. The more generic ones are (re-)
listed here:

* Compile errors are sometimes a bit obscure and long. This needs work. 
  Suggestions for improvement are welcome. 

* The documentation needs work.

ManageSieve
-----------

* Although this ManageSieve server should comply with the draft specification of
  the ManageSieve protocol, quite a few clients don't. This is particularly true
  for the TLS support. However, now that Cyrus' Timsieved has changed its
  behavior towards protocol compliance, all those clients will follow
  eventually.

  Clients known to have TLS issues:
	- Thunderbird Sieve add-on: fixed as per version 0.1.5
	- AvelSieve: patch on the wiki:	http://wiki.dovecot.org/ManageSieve
	- KMail + kio_sieve: TLS broken for old versions. This issue is fixed at
	  least in kmail 1.9.9 / kde 3.5.9.

  Unfortunately, there is no reliable way to provide a workaround for this
  problem. We will have to wait for the authors of these clients to make the
  proper adjustments.

* Other client issues:

	- SmartSieve, WebSieve:
	  These clients are specifically written for Cyrus timsieved and fail on
	  multiple stages of the protocol when connected to Dovecot ManageSieve.

* The ANONYMOUS authentication mechanism is currently not supported and
  explicitly denied.

Authors
=======

Refer to AUTHORS file.

Contact Info
============

Stephan Bosch <stephan at rename-it dot nl>
IRC: Freenode, #dovecot, S[r]us
Web: http://pigeonhole.dovecot.org

Please use the Dovecot mailing list <dovecot at dovecot.org> for questions about 
this package. You can post to the list without subscribing, the mail then waits 
in a moderator queue for a while. See http://dovecot.org/mailinglists.html