The stem algorithms available for German in Elasticsearch perform not good. But there are alternatives. Namely Apache Lucene/Solr offers already the GermanStemmer based on the Caumann stemming algorithm. There is the newer Cistem stemmer, too. This stemmer is already available for NLTK . The authors…

The GermanStemmer is available via the german_stem token filter, as that one uses a GermanStemFilter, which in turn uses the GermanStemmer. For the Cistem stemmer you would need to write your own plugin...

Thanks for the reply. I couldn't find any information about the GermanStemmer/GermanStemFilter in the elasticsearch documenation. (Is there any reference to this stem filter in the official documentation?) I tried the Cistem stemmer via a custum plugin, too. For example the words Wunsch, Wünsche…

stemmers for german languages are tricky in general. If those results are not good enough you could always resort to a dictionary based stemmer like the hunspell one. We should update the documentation to update the german_stem. I'm happy to do that, unless you would be willing to submit a pull req…

[image] htsr: mix stemming with synonyms for words known not to be stemmed While this is a possible workaround, please note that there might be caveats in ranking synonyms slightly differently than stemmed tokens. For example, we currently explore ways to slightly deboost matches on synonym t…

[image] htsr: I tried the Cistem stemmer via a custom plugin... btw. this sounds very interesting, which plugin did you use? Is it publicly available? Looks like the library is MIT licensed at least, which is great.

See https://github.com/LeonieWeissweiler/CISTEM/blob/master/Cistem.java (not the plugin, just the java class), I hope that's what you meant?

Ah, okay. Sounded like there's a ready-to-install plugin already, but apparently not.

I wrote my own custom plugin using the stempel and ICU plugins as templates.

That sounds very interesting. I would be interested to try it if you want and are able to share, even if its in a WIP state, as long as I can build it from source. No problem if thats not possible though.

If you are interested in testing the plugin, I will try my best. But it will be next week that I will notify you.

German stemmer - Cistem und Caumann stemmer

Elastic Stack Elasticsearch

htsr December 5, 2019, 5:09pm 5

Hi, Alex,
once again thanks for the information.

Another option could be mix stemming with synonyms for words known not to be stemmed appropriately. But that would be a somewhat static way to deal with this problem.

Could you please update the documentation.

Topic		Replies	Views
German stemmer - looking for snowball alternative Elasticsearch	5	2075	July 6, 2017
Custom Stemmer Elasticsearch	1	702	July 5, 2017
Better French and German stemming? Elasticsearch	4	888	July 16, 2020
Stemming Elasticsearch	2	657	July 6, 2017
Search with stemming and stopwords (german) Elasticsearch	9	3526	July 6, 2017

German stemmer - Cistem und Caumann stemmer

Related topics