Switch the keep mechanism to rewriting datasets

While this is strictly less efficient than just hard-linking the old datasets, it also automatically migrates datasets to our schema version, includes the kept ones in the metrics and automatically upgrades them when the generic write_dataset function gains any new features.