diff --git a/DESIGN b/DESIGN
new file mode 100644
index 0000000000000000000000000000000000000000..09120aff172126aeaa7ceab8f2f45a21cc0b94c3
--- /dev/null
+++ b/DESIGN
@@ -0,0 +1,45 @@
+The compiler consists of the following stages:
+
+PARSER: sieve-parser.c, sieve-lexer.c
+ Parses the scriptfile and produces an abstract syntax tree for it
+ (sieve-ast.c).
+
+VALIDATOR: sieve-validator.c
+ Performs contextual analysis on the ast produced by the parser. This checks
+ for the validity of commands, tests and arguments. Also, the ast is decorated
+ with any context data acquired during the process. This context is used by the
+ last compiler stage.
+
+GENERATOR: sieve-generator.c
+ This last compiler stage uses a visitor pattern to wander through the ast and
+ produces sieve byte code (sieve-binary.c).
+
+The resulting (in-memory) binary can be fed to the interpreter for execution:
+
+INTERPRETER: sieve-interpreter.c
+ The interpreter executes the byte code and produces a sieve_result object.
+ This result is no more than just a collection of actions to be performed.
+ During execution, action commands add actions to the result. Duplates and
+ conflicts between actions are handled in this execution phase.
+
+RESULT: sieve-result.c sieve-actions.c
+ When the result is to be executed, it needs no further checking, as the
+ validity of the result was verified during interpretation already. The
+ result's actions are executed in a transaction-like atomic manner. If one of
+ the actions fails, the whole transaction is rolled back meaning that either
+ everything succeeds or everything fails. This is only possible to some extent:
+ transmitted responses can of course not be rolled back. However, these are
+ executed in the commit phase, meaning that they will only be performed if all
+ other actions were successful.
+
+Debugging:
+
+BINARY-DUMPER: sieve-code-dumper.c sieve-binary-dumper.c
+ A loaded binary can be dumped to a stream in human-readable form using the
+ binary-dumper. The binary-dumper displays information on all the blocks that
+ the binary consists off. Program code blocks are dumped using the code-dumper.
+ It's implementation is similar to the interpreter, with the exception that it
+ performs no actions and just sequentially wanders through the byte code
+ printing instructions along the way. The term human-readable is a bit optimistic
+ though; currently, the presented data looks like an assembly language.
+
diff --git a/INSTALL b/INSTALL
index de343b75862bf9beec8af5e89c94adc6c0bb3ca0..746a9c5225736fc8001427415c05e0478f8f0368 100644
--- a/INSTALL
+++ b/INSTALL
@@ -1,6 +1,11 @@
-WARNING: This sieve implementation is highly experimental. In addition to
-the usual GPL disclaimer I urge you not to use this for any important mail
-just yet!
+################################################################################
+ !! WARNING !!
+
+ This sieve implementation is highly experimental. In addition to the usual GPL
+ disclaimer I urge you not to use this for any important mail just yet!
+
+################################################################################
+
Compiling
---------
diff --git a/Makefile.am b/Makefile.am
index 9fffc44a24379a951a4aa698bc1a70e83bcbda0f..e9b08aa86d392a7608b1b88033bc4c85687b084d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,7 +1,10 @@
SUBDIRS = src
EXTRA_DIST = \
+ sieve/* \
+ doc/* \
COPYING.LGPL \
+ DESIGN \
ChangeLog
if MAINTAINER_MODE
diff --git a/README b/README
index f3bc3b9cdf7d6acc2f9240d9efcf6428f9265310..b7e91ed659c6bd5a31e2a7118481cdf5176cbd92 100644
--- a/README
+++ b/README
@@ -1,5 +1,10 @@
-WARNING: This sieve implementation is highly experimental. In addition to the
-usual GPL disclaimer I urge you not to use this for any important mail just yet!
+################################################################################
+ !! WARNING !!
+
+ This sieve implementation is highly experimental. In addition to the usual GPL
+ disclaimer I urge you not to use this for any important mail just yet!
+
+################################################################################
Compiling and Configuring
-------------------------
@@ -90,17 +95,18 @@ What works:
* Script code generation works for all core commands. Comparators, match-types
and address-part modifiers work as required.
* Interpreter runs core commands and tests. Comparators, match-types and
- address-part modifiers have the desired effect. Most test commands work as
- specified (see list below).
+ address-part modifiers have the desired effect.
* The interpreter produces a result containing a set of actions to execute.
- Duplicate actions can be avoided and conflicts can be detected.
+ Duplicate actions can be avoided and conflicts can be detected. Multiple
+ scripts could be executed in succession on the result object although no
+ support to do so is currently available.
* Execution of the result is supported for all core action commands and all
fully implemented extensions (see list below).
* Compiled binary code can be saved to and loaded back from disk. The binary
is structured as a set of data blocks, which can contain extension-dependent
extra data. For example, the include extension uses this to store the
compiled included scripts (no external inclusion of binaries supported yet).
-* This sieve implementation is now available as an alternative plugin to
+* This sieve implementation is available as an alternative plugin to
dovecot's deliver. It is not completely useful yet, but hopefully this will
soon be able to replace the current cmusieve implementation.
@@ -122,7 +128,7 @@ Base commands and their implementation status:
Extensions and their implementation status:
- Base specification (RFC3028):
+ Base specification (RFC5228):
fileinto: full
reject: full
envelope: full
@@ -133,13 +139,13 @@ Extensions and their implementation status:
comparator-i;ascii-numeric: full
relational: full
copy: full
- regex: full, but suboptimal, no UTF-8
+ regex: full, but suboptimal and no UTF-8
body: full, but text body-transform implementation is simple
- include: almost full; needs more work (no external binaries)
+ include: almost full; needs some more work (no external binaries)
vacation: almost full; no support for required References header
imapflags: flag management works, but flags are not stored (no variables)
- variables: untested core functionality present; need to provide support
- other extensions that depend on this one
+ variables: largely untested core functionality present; need to provide
+ suppor other extensions that depend on this one
Low priority:
notify: planned, first mailto only
@@ -148,116 +154,16 @@ Extensions and their implementation status:
All implemented extensions are like the engine itself currently very much
experimental. A status of 'full' does not mean that the extension is bug-free
-or even RFC-compliant. Other extensions will be added as soon as the necessary
-infrastructure is available. Extensions supported by cmusieve have priority.
+or even fully RFC-compliant. Other extensions will be added as soon as the
+necessary infrastructure is available. Extensions already supported by cmusieve
+have priority.
Design
------
-The compiler consists of the following stages:
-
-PARSER: sieve-parser.c, sieve-lexer.c
- Parses the scriptfile and produces an abstract syntax tree for it
- (sieve-ast.c).
-
-VALIDATOR: sieve-validator.c
- Performs contextual analysis on the ast produced by the parser. This checks
- for the validity of commands, tests and arguments. Also, the ast is decorated
- with any context data acquired during the process. This context is used by the
- last compiler stage.
-
-GENERATOR: sieve-generator.c
- This last compiler stage uses a visitor pattern to wander through the ast and
- produces sieve byte code (sieve-binary.c).
-
-The resulting (in-memory) binary can be fed to the interpreter for execution:
-
-INTERPRETER: sieve-interpreter.c
- The interpreter executes the byte code and produces a sieve_result object.
- This result is no more than just a collection of actions to be performed.
- During execution, action commands add actions to the result. Duplates and
- conflicts between actions are handled in this execution phase.
-
-RESULT: sieve-result.c sieve-actions.c
- When the result is to be executed, it needs no further checking, as the
- validity of the result was verified during interpretation already. The
- result's actions are executed in a transaction-like atomic manner. If one of
- the actions fails, the whole transaction is rolled back meaning that either
- everything succeeds or everything fails. This is only possible to some extent:
- transmitted responses can of course not be rolled back. However, these are
- executed in the commit phase, meaning that they will only be performed if all
- other actions were successful.
-
-Debugging:
-
-CODE-DUMPER: sieve-code-dumper.c
- A loaded binary can be dumped to a stream in human-readable form using the
- code-dumper. It's implementation is similar to the interpreter, with the
- exception that it performs no actions and just sequentially wanders through
- the byte code printing instructions along the way. The term human-readable is
- a bit optimistic though; currently, the presented data looks like an assembly
- language.
+Refer to DESIGN file.
TODO
----
-Current:
-* Implement variables extension
- Implemented:
- -> Core variable substitution works
- -> Accept namespaces (parse, no real support; proper error messages)
- -> Support match variables
- -> Implement variables support for include extension
- Current:
- -> Provide support for extensions that (partly) depend on variables support.
- -> Implement variables support for imapflags extension
-
-Next (in order of descending priority/precedence):
-* Vacation extension accepts duplicate tags (not allowed)
-* Finish implementing all extensions supported by cmusieve, except notify.
-* Limit the maximum number of errors.
-* Verify outgoing mail addresses
-* Add normalize() method to comparators to normalize the string before mathing
- (for efficiency).
-* Implement comparator-i;unicode-casemap
-* Implement dropping errors in the user's mailbox as a mail message.
-* Make this implementation conform section 2.7.2 of RFC3028 (Comparisons Across
- Character Sets).
-* Get rid of all <stdio.h> printf()s in the library; use trace macro instead
-* Use lib/llist.h for the AST implementation.
-* Make the sieve plugins true plugins and add a SIEVE_PLUGINS config item to the
- lda-sieve plugin.
-
-* Full security review. Enforce limits on number of created objects, script
- size, execution time, etc...
-* Full standards compliance review for the engine and all fully implemented
- sieve extensions.
-* Code cleanup
-* Make sure cmusieve can be replaced seamlessly with this new plugin.
-* Make a few elaborate test scripts to test engine and all implemented
- extensions a little better. Also include specially crafted e-mail messages
- that give deterministic and thus testable results.
- (This might become an initial version of the test suite)
-
-* ## MAKE A FIRST RELEASE ##
-
-* Automate script tests; i.e. build a test suite.
-* Use lib/str-find.h for :contains and :matches match types
-* Resolve code duplication introduced for handling address-parts and match-types
- in different command implementations.
-* Resolve code duplication amongst comparator, address-part and match-type
- support as much as possible.
-* Add development documentation, i.e. comment on library functions and document
- the binary and byte-code format.
-* Make the engine and its extensions much more configurable. Possibly this can
- be merged with Dovecot's new master config implementation.
-* Implement notify extension with sole support for mailto mechanism.
-* Implement editheader extension
-* Implement mimeloop extension
-* Give the byte code format some more thought, it is currently quite rough and
- to the point.
-* Try to implement proposed notify mechanisms other than mailto. Currently: xmpp
- and sip
-* Implement namespace support for variables extension
- (possibly needed by test suite; in that case priority is much higher)
-
+Refer to TODO file.
diff --git a/TODO b/TODO
new file mode 100644
index 0000000000000000000000000000000000000000..5f07dd63cad03803e3cd27c3c89b1664e9d1be30
--- /dev/null
+++ b/TODO
@@ -0,0 +1,59 @@
+Current:
+* Implement variables extension
+ Implemented:
+ -> Core variable substitution works
+ -> Accept namespaces (parse, no real support; proper error messages)
+ -> Support match variables
+ -> Implement variables support for include extension
+ Current:
+ -> Provide support for extensions that (partly) depend on variables support.
+ -> Implement variables support for imapflags extension
+
+Next (in order of descending priority/precedence):
+* Vacation extension accepts duplicate tags (not allowed)
+* Finish implementing all extensions supported by cmusieve, except notify.
+* Limit the maximum number of errors.
+* Verify outgoing mail addresses
+* Make this implementation conform section 2.7.2 of RFC3028 (Comparisons Across
+ Character Sets).
+* Get rid of all <stdio.h> printf()s in the library; use trace macro instead
+* Use lib/llist.h for the AST implementation.
+* Make the sieve plugins true plugins and add a SIEVE_PLUGINS config item to the
+ lda-sieve plugin.
+* Revise extension support for comparators, match-types, address-parts and
+ side-effects.
+
+* Full security review. Enforce limits on number of created objects, script
+ size, execution time, etc...
+* Full standards compliance review for the engine and all fully implemented
+ sieve extensions.
+* Code cleanup
+* Make sure cmusieve can be replaced seamlessly with the new plugin.
+* Make simple test suite for the base functionality
+
+* ## MAKE A FIRST RELEASE ##
+
+* Implement dropping errors in the user's mailbox as a mail message.
+* Add normalize() method to comparators to normalize the string before mathing
+ (for efficiency).
+* Implement comparator-i;unicode-casemap
+* Automate script tests; i.e. build a test suite.
+* Use lib/str-find.h for :contains and :matches match types
+* Resolve code duplication introduced for handling address-parts and match-types
+ in different command implementations.
+* Resolve code duplication amongst comparator, address-part and match-type
+ support as much as possible.
+* Add development documentation, i.e. comment on library functions and document
+ the binary and byte-code format.
+* Make the engine and its extensions much more configurable. Possibly this can
+ be merged with Dovecot's new master config implementation.
+* Implement notify extension with sole support for mailto mechanism.
+* Implement editheader extension
+* Implement mimeloop extension
+* Give the byte code format some more thought, it is currently quite rough and
+ to the point.
+* Try to implement proposed notify mechanisms other than mailto. Currently: xmpp
+ and sip
+* Implement namespace support for variables extension
+ (possibly needed by test suite; in that case priority is much higher)
+
diff --git a/src/lib-sieve/Makefile.am b/src/lib-sieve/Makefile.am
index 2dcf2ccb0abd8819aaaf44c2688074fab43dcec6..7b0ac4f36a43bed2dd614e4c10de3f799465e871 100644
--- a/src/lib-sieve/Makefile.am
+++ b/src/lib-sieve/Makefile.am
@@ -80,7 +80,7 @@ libsieve_la_SOURCES = \
noinst_HEADERS = \
sieve-lexer.h \
sieve-script.h \
- sieve-script-private.h
+ sieve-script-private.h \
sieve-ast.h \
sieve-binary.h \
sieve-parser.h \
@@ -89,6 +89,7 @@ noinst_HEADERS = \
sieve-interpreter.h \
sieve-code-dumper.h \
sieve-binary-dumper.h \
+ sieve-dump.h \
sieve-result.h \
sieve-error.h \
sieve-error-private.h \
@@ -96,6 +97,10 @@ noinst_HEADERS = \
sieve-match-types.h \
sieve-address-parts.h \
sieve-commands.h \
+ sieve-commands-private.h \
sieve-code.h \
+ sieve-actions.h \
sieve-extensions.h \
+ sieve-extensions-private.h \
+ sieve-common.h \
sieve.h
diff --git a/src/lib-sieve/plugins/body/Makefile.am b/src/lib-sieve/plugins/body/Makefile.am
index 641c32e945bbeed4b5b0e2d22e5a3a0e3430fd35..9f8d8d80489908549b06b53e1a6c47565b4f90eb 100644
--- a/src/lib-sieve/plugins/body/Makefile.am
+++ b/src/lib-sieve/plugins/body/Makefile.am
@@ -15,3 +15,7 @@ libsieve_ext_body_la_SOURCES = \
$(tsts) \
ext-body.c
+noinst_HEADERS = \
+ ext-body-common.h
+
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/comparator-i-ascii-numeric/Makefile.am b/src/lib-sieve/plugins/comparator-i-ascii-numeric/Makefile.am
index ca0d8abf3a9019c6a9d12787a15e8d739602decc..89c2e4ed97dd717745e01e7754105361a704e880 100644
--- a/src/lib-sieve/plugins/comparator-i-ascii-numeric/Makefile.am
+++ b/src/lib-sieve/plugins/comparator-i-ascii-numeric/Makefile.am
@@ -10,3 +10,4 @@ AM_CPPFLAGS = \
libsieve_ext_comparator_i_ascii_numeric_la_SOURCES = \
ext-cmp-i-ascii-numeric.c
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/copy/Makefile.am b/src/lib-sieve/plugins/copy/Makefile.am
index e6df8563a8947f8cb33b9489726f25b12b526ee9..1d71d3ac5c6b55bd8eb9d538fc519b891bfa0feb 100644
--- a/src/lib-sieve/plugins/copy/Makefile.am
+++ b/src/lib-sieve/plugins/copy/Makefile.am
@@ -10,3 +10,4 @@ AM_CPPFLAGS = \
libsieve_ext_copy_la_SOURCES = \
ext-copy.c
+EXTRA_DIST = *.sieve *.txt
diff --git a/src/lib-sieve/plugins/imapflags/Makefile.am b/src/lib-sieve/plugins/imapflags/Makefile.am
index 8d551234e48944b06588925bfba6bb077fff4428..944c32625430f6c9739bb45c4a6e69fbd5c26cc3 100644
--- a/src/lib-sieve/plugins/imapflags/Makefile.am
+++ b/src/lib-sieve/plugins/imapflags/Makefile.am
@@ -27,3 +27,5 @@ libsieve_ext_imapflags_la_SOURCES = \
noinst_HEADERS = \
ext-imapflags-common.h
+
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/imapflags/ext-imapflags-common.c b/src/lib-sieve/plugins/imapflags/ext-imapflags-common.c
index 4c203b9247d14320278469a192ddfbce0dd8b0fc..a18d5eed2f07f150876b417a7828db366265f22a 100644
--- a/src/lib-sieve/plugins/imapflags/ext-imapflags-common.c
+++ b/src/lib-sieve/plugins/imapflags/ext-imapflags-common.c
@@ -1,4 +1,5 @@
#include "lib.h"
+#include "str.h"
#include "sieve-commands-private.h"
#include "sieve-code.h"
diff --git a/src/lib-sieve/plugins/include/Makefile.am b/src/lib-sieve/plugins/include/Makefile.am
index 44236db117c341585000bad48a80c2e93dac72c9..75fc10c049039f9cc8b135ae49fcbd101b626429 100644
--- a/src/lib-sieve/plugins/include/Makefile.am
+++ b/src/lib-sieve/plugins/include/Makefile.am
@@ -24,3 +24,5 @@ noinst_HEADERS = \
ext-include-common.h \
ext-include-binary.h \
ext-include-variables.h
+
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/regex/Makefile.am b/src/lib-sieve/plugins/regex/Makefile.am
index d80bbedef641490c36f7f602d902a3319dadbb51..1e02e47ae1ed9526558ec6571bfe233950047246 100644
--- a/src/lib-sieve/plugins/regex/Makefile.am
+++ b/src/lib-sieve/plugins/regex/Makefile.am
@@ -13,3 +13,5 @@ libsieve_ext_regex_la_SOURCES = \
noinst_HEADERS = \
ext-regex-common.h
+
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/relational/Makefile.am b/src/lib-sieve/plugins/relational/Makefile.am
index d5ae35babcdee3d5c1680d95fdf4cfa6ef0197e7..2b5fbd8a235b47d1d81a6d9419820ece97745633 100644
--- a/src/lib-sieve/plugins/relational/Makefile.am
+++ b/src/lib-sieve/plugins/relational/Makefile.am
@@ -13,5 +13,7 @@ libsieve_ext_relational_la_SOURCES = \
mcht-count.c \
ext-relational.c
-noinst_HEADER = \
+noinst_HEADERS = \
ext-relational-common.h
+
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/subaddress/Makefile.am b/src/lib-sieve/plugins/subaddress/Makefile.am
index 02777f61e168aeb87a47ce29e8f9104409f26fd9..119effbca3837f74725959f0beac52da5e9a5fa5 100644
--- a/src/lib-sieve/plugins/subaddress/Makefile.am
+++ b/src/lib-sieve/plugins/subaddress/Makefile.am
@@ -10,3 +10,4 @@ AM_CPPFLAGS = \
libsieve_ext_subaddress_la_SOURCES = \
ext-subaddress.c
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/vacation/Makefile.am b/src/lib-sieve/plugins/vacation/Makefile.am
index 3bd21db0e1eb5b4acc863871d537ecf96ff7e4ba..cc10d1d4385f92342363e3f47691f3e89c5e4eeb 100644
--- a/src/lib-sieve/plugins/vacation/Makefile.am
+++ b/src/lib-sieve/plugins/vacation/Makefile.am
@@ -10,3 +10,4 @@ AM_CPPFLAGS = \
libsieve_ext_vacation_la_SOURCES = \
ext-vacation.c
+EXTRA_DIST = *.txt *.sieve
diff --git a/src/lib-sieve/plugins/variables/Makefile.am b/src/lib-sieve/plugins/variables/Makefile.am
index de6badd9b37b8995ec5e04b2fec911c661b37076..067d1b2c417e5498801b14d91c38edb9de6ec300 100644
--- a/src/lib-sieve/plugins/variables/Makefile.am
+++ b/src/lib-sieve/plugins/variables/Makefile.am
@@ -29,3 +29,4 @@ noinst_HEADERS = \
ext-variables-operands.h \
sieve-ext-variables.h
+EXTRA_DIST = *.txt *.sieve