Support encodings other than UTF-8 for fetch_text
Our fetch_text
helper and its wrapper for external harvesters currently assumes UTF-8 encoding which fails for example when we download a ISO-8859-encoded CSV file. We need to extend our infrastructure to support specifying the encoding e.g. via text_with_charset
to properly handle resources with insufficient HTTP metadata (e.g. no Content-Type
header).