Use much simpler and more efficient in-memory deduplication

While the scaling of our current approach is only limited by the file system and storage capacity, the keys of our index will fit into memory for the foreseeable future and hence we can utilize and much simpler and more efficient in-memory approach.