Skip to content
Snippets Groups Projects
Verified Commit f8a3b736 authored by Adam Reichold's avatar Adam Reichold
Browse files

Create CustomTokenizer based on Tantivy's SimpleTokenizer 

This allows us to add special cases like internal hyphens
which we want to handle differently than upstream.
parent d3becbb5
No related branches found
No related tags found
Checking pipeline status
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

Consent

On this website, we use the web analytics service Matomo to analyze and review the use of our website. Through the collected statistics, we can improve our offerings and make them more appealing for you. Here, you can decide whether to allow us to process your data and set corresponding cookies for these purposes, in addition to technically necessary cookies. Further information on data protection—especially regarding "cookies" and "Matomo"—can be found in our privacy policy. You can withdraw your consent at any time.