WARNING: This sieve implementation is highly experimental. In addition to the 
usual GPL disclaimer I urge you not to use this for any important mail just yet!

Compiling and Configuring
-------------------------

Refer to INSTALL file.

Using
-----

The main purpose of this package is to replace the existing cmusieve plugin that 
is currently available for dovecot's deliver. With this respect it is currently 
not very different from the cmusieve implementation. 

Unlike cmusieve, this sieve module logs runtime errors to <scriptfile>.log if it 
can and not '<scriptfile>.err'. It appends new timestamped log entries to the 
end of the logfile. I currently didn't bother to implement log rotation, so keep 
this in mind. Some development-related debug output is sent to stderr/stdout at 
all times, this will be removed when this project matures. 

To test the sieve engine outside deliver it is useful to execute the binaries 
that exist in the src/sieve-bin/ directory:

--

sievec [-d] <sieve-file> <out-file>

Compiles the script and produces a sieve binary. However, when the -d option is
provided, a code dump is written to the output file. When the output file is '-'
the dump output is written to stdout (only if -d is specified). 

--

sieved <bin-file> [<dump-file>]

Reads a sieve binary and produces a code dump. The optional dump-file parameter
provides the means to specify a file to which the dump is to be written. 
Otherwise, the dump is printed to stdout. 

-- 

sieve-test [-r <recipient address>][-s <envelope sender>]
           [-m <mailbox>][-d <dump filename>] <scriptfile> <mailfile>

Reads mail message from the specified mailfile and executes the specified sieve 
script to produce a verdict. This prints an execution dump with the instructions 
encountered during execution and finally it prints a list of actions that would 
have been performed on this message.

Options:
	-r  envelope recipient address
	-s	envelope sender
	-m	the mailbox where the keep action should store
	-d	causes a dump of the generated code to be written to the specified
		file. Using - as filename causes the dump to be written to stdout

--

sieve-exec

Currently undocumented (will be merged with sieve-test).

--

Various example scripts are bundled in the directory 'sieve'.

Authors
-------

Refer to AUTHORS file.

Features
--------

* Well-structured 3-stage compiler: uses dovecot framework and avoids using
  lex/yucc. Compiler doesn't bail on first error, but tries to find more.
* Highly extendable with new sieve capabilities: This keeps the possibility of 
  plugins in mind. It should eventually provide the necessary infrastructure for 
  at least all currently known (proposed) extensions. The goal is to keep the
  extension interface provided by sieve engine as generic as possible, i.e. 
  without explicit support for specific extensions. New similar extensions can
  then use the same interface methods without changes to the sieve engine code.

What works:
* Scripts can be parsed, the grammar is fully supported. 
* Script validation (contextual analysis) works almost completely. Outgoing mail 
  addresses are not verified yet as required. 
* Script code generation works for all core commands. Comparators, match-types 
  and address-part modifiers work as required.
* Interpreter runs core commands and tests. Comparators, match-types and 
  address-part modifiers have the desired effect. Most test commands work as 
  specified (see list below).
* The interpreter produces a result containing a set of actions to execute.
  Duplicate actions can be avoided and conflicts can be detected. 
* Execution of the result is supported for all core action commands and all
  fully implemented extensions (see list below).  
* This sieve implementation is now available as an alternative plugin to  
  dovecot's deliver. It is not completely useful yet, but hopefully this will 
  soon be able to replace the current cmusieve implementation.

Base tests and their implementation status:
  false, true: trivial, full
  address: full
  header: full 
  exists: full
  size: full 	 
  not, anyof, allof: full

Base commands and their implementation status:
  require: full
  if,elsif,else: full
  discard: full
  keep: full
  redirect: full 
  stop: trivial, full
	
Extensions and their implementation status:

  Base specification (RFC3028):
    fileinto: full
    reject: full
    envelope: full
    encoded-character: full

  Other RFCs/drafts:
    subaddress: full
    comparator-i;ascii-numeric: full
    relational: full 
    copy: full
    regex: full, but suboptimal
    body: full, but text body-transform implementation is simple
    include: full, but needs much more work
    vacation: almost complete, but no support for required References header
    imapflags: flag management works, but flags are not stored 
    variables: skeleton

    notify: planned, first mailto only (- very low priority)

All implemented extensions are like the engine itself currently very much 
experimental. Other extensions will be added as soon as the necessary 
infrastructure is available. Extensions supported by cmu-sieve have priority, 
although variables might be implemented somewhere in between. 

Design
------

The compiler consists of the following stages:

PARSER: sieve-parser.c, sieve-lexer.c
  Parses the scriptfile and produces an abstract syntax tree for it 
  (sieve-ast.c). 

VALIDATOR: sieve-validator.c
  Performs contextual analysis on the ast produced by the parser. This checks 
  for the validity of commands, tests and arguments. Also, the ast is decorated 
  with any context data acquired during the process. This context is used by the 
  last compiler stage. 

GENERATOR: sieve-generator.c
  This last compiler stage uses a visitor pattern to wander through the ast and 
  produces sieve byte code (sieve-binary.c).

The resulting (in-memory) binary can be fed to the interpreter for execution:

INTERPRETER: sieve-interpreter.c 
  The interpreter executes the byte code and produces a sieve_result object. 
  This result is no more than just a collection of actions to be performed. 
  During execution, action commands add actions to the result. Duplates and 
  conflicts between actions are handled in this execution phase.

RESULT: sieve-result.c sieve-actions.c
  When the result is to be executed, it needs no further checking, as the 
  validity of the result was verified during interpretation already. The 
  result's actions are executed in a transaction-like atomic manner. If one of 
  the actions fails, the whole transaction is rolled back meaning that either 
  everything succeeds or everything fails. This is only possible to some extent:
  transmitted responses can of course not be rolled back. However, these are 
  executed in the commit phase, meaning that they will only be performed if all
  other actions were successful.
  
Debugging:

CODE-DUMPER: sieve-code-dumper.c 
  A loaded binary can be dumped to a stream in human-readable form using the 
  code-dumper. It's implementation is similar to the interpreter, with the 
  exception that it performs no actions and just sequentially wanders through 
  the byte code printing instructions along the way. The term human-readable is 
  a bit optimistic though; currently the presented data looks like an assembly
  language. 

TODO
----

Current:
* Implement variables extension.

Next (in order of descending priority/precedence):
* Finish implementing all extensions supported by cmusieve, except notify.
* Limit the maximum number of errors. 
* Verify outgoing mail addresses
* Implement dropping errors inthe user's mailbox as a mail message.
* Make this implementation conform section 2.7.2 of RFC3028 (Comparisons Across
  Character Sets). 
* Get rid of all <stdio.h> printf()s in the library. 
* Full security review. Enforce limits on number of created objects, script 
  size, execution time, etc...
* Full standards compliance review for the engine and all fully implemented 
  sieve extensions.
* Make the sieve plugins true plugins and add a SIEVE_PLUGINS config item to the
  lda-sieve plugin. 
* Make sure cmusieve can be replaced seamlessly with this new plugin.
* Make a few elaborate test scripts to test engine and all implemented extensions 
  a little better. Also include specially crafted e-mail messages that give 
  deterministic and thus testable results.
* ## MAKE A FIRST RELEASE ##
* Resolve code duplication introduced for handling address-parts and match-types
  in different command implementations.
* Resolve code duplication amongst comparator, address-part and match-type 
  support as much as possible.
* Add development documentation, i.e. comment on library functions and document
  the binary and byte-code format. 
* Make the engine and its extensions much more configurable. Possibly this can be
  merged with Dovecot's new master config implementation.
* Implement notify extension with sole support for mailto mechanism. 
* Support mmap'ing a sieve binary upon load. 
* Give the byte code format some more thought, it is currently quite rough and
  to the point. 
* Implement a faster substring search algorithm to make sopport for the body 
  extension a less bad idea.  
* Automate script tests; i.e. build a test suite.
* Try to implement proposed notify mechanisms other than mailto. Currently: xmpp
  and sip