# Ann: ElasticSearch Hunspell Analysis plugin **URL:** \ **Category:** Elasticsearch\ **Created:** [December 29, 2011, 8:45pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253 "2011-12-29T20:45:12Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [December 29, 2011, 8:45pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/1 "2011-12-29T20:45:12Z")
Hi, because all of you are eager to keep up with Lucene 3.5 features, I just wrote an ElasticSearch Hunspell Analysis plugin. Project URL: > **[jprante/elasticsearch-analysis-hunspell](https://github.com/jprante/elasticsearch-analysis-hunspell)** > > elasticsearch-analysis-hunspell - Hunspell analysis for ElasticSearch For discussion, see > Please note: included are hunspell dict/aff files from Chromium for convenience. The license for the third-party files is a tri-license MPL 1.1/GPL 2.0/LGPL. Example usage: index: analysis: filter: hunspell\_de: type: hunspell locale: de\_DE ignoreCase: true Cheers, Jörg ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 29, 2011, 9:17pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/2 "2011-12-29T21:17:20Z")
Hi Jörg, This feature looks great. Correct me if I misunderstood: your plugin corrects spelling errors before indexing ? And when a user runs a search, terms are modified (spell check) before running the search ? Is that it ? Thanks David. Le 29 décembre 2011 à 21:45, "Jörg Prante" [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit : > Hi, > > because all of you are eager to keep up with Lucene 3.5 features, I just > wrote an Elasticsearch Hunspell Analysis plugin. > > Project URL: > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > For discussion, see > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/646) > > Please note: included are hunspell dict/aff files from Chromium for > convenience. The license for the third-party files is a tri-license MPL > 1.1/GPL 2.0/LGPL. > > Example usage: > > index: > analysis: > filter: > hunspell\_de: > type: hunspell > locale: de\_DE > ignoreCase: true > > Cheers, > > Jörg -- David Pilato [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) Twitter : @dadoonet ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [December 30, 2011, 7:23am UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/3 "2011-12-30T07:23:19Z")
Hi David, I forgot to mention, right now, hunspell is used in Lucene as a token filter for stemming. "TokenFilter that uses hunspell affix rules and words to stem tokens. Since hunspell supports a word having multiple stems, this filter can emit multiple tokens for each consumed token." [http://lucene.apache.org/java/3\_5\_0/api/contrib-analyzers/org/apache/lucene/analysis/hunspell/HunspellStemFilter.html](http://lucene.apache.org/java/3_5_0/api/contrib-analyzers/org/apache/lucene/analysis/hunspell/HunspellStemFilter.html) From a linguistic point of view, hunspell stemming is an improvement over snowball stemming. "Hunspell provides stemming for all languages that have OpenOffice spellcheck dictionaries. Being dictionary based, it requires high quality and well maintained dictionaries to work well for stemming - in which case it may give more precise stemming than the Snowball algorithms." [http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters](http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters) Jörg On Dec 29, 10:17 pm, "[da...@pilato.fr](mailto:da...@pilato.fr)" [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > Hi Jörg, > > This feature looks great. > Correct me if I misunderstood: your plugin corrects spelling errors before > indexing ? > And when a user runs a search, terms are modified (spell check) before running > the search ? > > Is that it ? > > Thanks > David. > > Le 29 décembre 2011 à 21:45, "Jörg Prante" [joergpra...@gmail.com](mailto:joergpra...@gmail.com) a écrit : > > > Hi, > > > because all of you are eager to keep up with Lucene 3.5 features, I just > > wrote an Elasticsearch Hunspell Analysis plugin. > > > Project URL: > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > For discussion, see > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/646) > > > Please note: included are hunspell dict/aff files from Chromium for > > convenience. The license for the third-party files is a tri-license MPL > > 1.1/GPL 2.0/LGPL. > > > Example usage: > > > index: > > analysis: > > filter: > > hunspell\_de: > > type: hunspell > > locale: de\_DE > > ignoreCase: true > > > Cheers, > > > Jörg > > -- > David Pilatohttp://dev.david.pilato.fr/ > Twitter : @dadoonet ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 9:55pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/4 "2011-12-30T21:55:51Z")
This looks great!, especially with all the built in dicts. Did you download those from open office? On Fri, Dec 30, 2011 at 9:23 AM, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Hi David, > > I forgot to mention, right now, hunspell is used in Lucene as a token > filter for stemming. > > "TokenFilter that uses hunspell affix rules and words to stem tokens. > Since hunspell supports a word having multiple stems, this filter can > emit multiple tokens for each consumed token." > > [HunspellStemFilter (Lucene 3.5.0 API)](http://lucene.apache.org/java/3_5_0/api/contrib-analyzers/org/apache/lucene/analysis/hunspell/HunspellStemFilter.html) > > From a linguistic point of view, hunspell stemming is an improvement > over snowball stemming. > > "Hunspell provides stemming for all languages that have OpenOffice > spellcheck dictionaries. Being dictionary based, it requires high > quality and well maintained dictionaries to work well for stemming - > in which case it may give more precise stemming than the Snowball > algorithms." > > [AnalyzersTokenizersTokenFilters - Solr - Apache Software Foundation](http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters) > > Jörg > > On Dec 29, 10:17 pm, "[da...@pilato.fr](mailto:da...@pilato.fr)" [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > > > Hi Jörg, > > > > This feature looks great. > > Correct me if I misunderstood: your plugin corrects spelling errors > > before > > indexing ? > > And when a user runs a search, terms are modified (spell check) before > > running > > the search ? > > > > Is that it ? > > > > Thanks > > David. > > > > Le 29 décembre 2011 à 21:45, "Jörg Prante" [joergpra...@gmail.com](mailto:joergpra...@gmail.com) a > > écrit : > > > > > Hi, > > > > > because all of you are eager to keep up with Lucene 3.5 features, I > > > just > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > Project URL: > > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > For discussion, see > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/646) > > > > > Please note: included are hunspell dict/aff files from Chromium for > > > convenience. The license for the third-party files is a tri-license MPL > > > 1.1/GPL 2.0/LGPL. > > > > > Example usage: > > > > > index: > > > analysis: > > > filter: > > > hunspell\_de: > > > type: hunspell > > > locale: de\_DE > > > ignoreCase: true > > > > > Cheers, > > > > > Jörg > > > > -- > > David Pilatohttp://dev.david.pilato.fr/ > > Twitter : @dadoonet ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [December 30, 2011, 11:22pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/5 "2011-12-30T23:22:35Z")
I downloaded them from Chromium git. git clone [chromium Git repositories - Git at Google](http://git.chromium.org/chromium/deps/hunspell_dictionaries.git) A Big Thanks to the Chromium folks at Google. They selected hunspell dicts from Open Office and from all over the place. Unfortunately I had to skip the dic\_delta / bdic effort which Chromium uses for dict enhancement, for details see README.chromium. Two aff files couldn't get parsed by Lucene's HunspellDictionary because of missing SET tags. But I was able to fix that. At least I hope so, the parse exception went away. Did not much testing though. The idea for future development is to create morpheme lists (finite state transducer) on the fly while indexing words to ES and prepare custom dict/aff files for spell check and autosuggestion. Jörg On Dec 30, 10:55 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > This looks great!, especially with all the built in dicts. Did you download > those from open office? > > On Fri, Dec 30, 2011 at 9:23 AM, jprante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > Hi David, > > > I forgot to mention, right now, hunspell is used in Lucene as a token > > filter for stemming. > > > "TokenFilter that uses hunspell affix rules and words to stem tokens. > > Since hunspell supports a word having multiple stems, this filter can > > emit multiple tokens for each consumed token." > > > [Index of /\_\_root/docs.lucene.apache.org/core/3\_5\_0/api/contrib-analyzers/org/apache](http://lucene.apache.org/java/3_5_0/api/contrib-analyzers/org/apache/)... > > > From a linguistic point of view, hunspell stemming is an improvement > > over snowball stemming. > > > "Hunspell provides stemming for all languages that have OpenOffice > > spellcheck dictionaries. Being dictionary based, it requires high > > quality and well maintained dictionaries to work well for stemming - > > in which case it may give more precise stemming than the Snowball > > algorithms." > > > [AnalyzersTokenizersTokenFilters - Solr - Apache Software Foundation](http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters) > > > Jörg > > > On Dec 29, 10:17 pm, "[da...@pilato.fr](mailto:da...@pilato.fr)" [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > > > > > Hi Jörg, > > > > This feature looks great. > > > Correct me if I misunderstood: your plugin corrects spelling errors > > > before > > > indexing ? > > > And when a user runs a search, terms are modified (spell check) before > > > running > > > the search ? > > > > Is that it ? > > > > Thanks > > > David. > > > > Le 29 décembre 2011 à 21:45, "Jörg Prante" [joergpra...@gmail.com](mailto:joergpra...@gmail.com) a > > > écrit : > > > > > Hi, > > > > > because all of you are eager to keep up with Lucene 3.5 features, I > > > > just > > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > Project URL: > > > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > For discussion, see > > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/646) > > > > > Please note: included are hunspell dict/aff files from Chromium for > > > > convenience. The license for the third-party files is a tri-license MPL > > > > 1.1/GPL 2.0/LGPL. > > > > > Example usage: > > > > > index: > > > > analysis: > > > > filter: > > > > hunspell\_de: > > > > type: hunspell > > > > locale: de\_DE > > > > ignoreCase: true > > > > > Cheers, > > > > > Jörg > > > > -- > > > David Pilatohttp://dev.david.pilato.fr/ > > > Twitter : @dadoonet ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 31, 2011, 12:03am UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/6 "2011-12-31T00:03:44Z")
On Sat, Dec 31, 2011 at 1:22 AM, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > I downloaded them from Chromium git. > > git clone [chromium Git repositories - Git at Google](http://git.chromium.org/chromium/deps/hunspell_dictionaries.git) > > A Big Thanks to the Chromium folks at Google. They selected hunspell > dicts from Open Office and from all over the place. Unfortunately I > had to skip the dic\_delta / bdic effort which Chromium uses for dict > enhancement, for details see README.chromium. Two aff files couldn't > get parsed by Lucene's HunspellDictionary because of missing SET tags. > But I was able to fix that. At least I hope so, the parse exception > went away. Did not much testing though. Great!. > The idea for future development is to create morpheme lists (finite > state transducer) on the fly while indexing words to ES and prepare > custom dict/aff files for spell check and autosuggestion. That would be great to have. Especially one that is realtime! > Jörg > > On Dec 30, 10:55 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > This looks great!, especially with all the built in dicts. Did you > > download > > those from open office? > > > > On Fri, Dec 30, 2011 at 9:23 AM, jprante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > > Hi David, > > > > > I forgot to mention, right now, hunspell is used in Lucene as a token > > > filter for stemming. > > > > > "TokenFilter that uses hunspell affix rules and words to stem tokens. > > > Since hunspell supports a word having multiple stems, this filter can > > > emit multiple tokens for each consumed token." > > > > > [Index of /\_\_root/docs.lucene.apache.org/core/3\_5\_0/api/contrib-analyzers/org/apache](http://lucene.apache.org/java/3_5_0/api/contrib-analyzers/org/apache/). > > > .. > > > > > From a linguistic point of view, hunspell stemming is an improvement > > > over snowball stemming. > > > > > "Hunspell provides stemming for all languages that have OpenOffice > > > spellcheck dictionaries. Being dictionary based, it requires high > > > quality and well maintained dictionaries to work well for stemming - > > > in which case it may give more precise stemming than the Snowball > > > algorithms." > > > > > [AnalyzersTokenizersTokenFilters - Solr - Apache Software Foundation](http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters) > > > > > Jörg > > > > > On Dec 29, 10:17 pm, "[da...@pilato.fr](mailto:da...@pilato.fr)" [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > > > > > > > Hi Jörg, > > > > > > This feature looks great. > > > > Correct me if I misunderstood: your plugin corrects spelling errors > > > > before > > > > indexing ? > > > > And when a user runs a search, terms are modified (spell check) > > > > before > > > > running > > > > the search ? > > > > > > Is that it ? > > > > > > Thanks > > > > David. > > > > > > Le 29 décembre 2011 à 21:45, "Jörg Prante" [joergpra...@gmail.com](mailto:joergpra...@gmail.com) a > > > > écrit : > > > > > > > Hi, > > > > > > > because all of you are eager to keep up with Lucene 3.5 features, I > > > > > just > > > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > > > Project URL: > > > > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > > > For discussion, see > > > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/646) > > > > > > > Please note: included are hunspell dict/aff files from Chromium for > > > > > convenience. The license for the third-party files is a > > > > > tri-license MPL > > > > > 1.1/GPL 2.0/LGPL. > > > > > > > Example usage: > > > > > > > index: > > > > > analysis: > > > > > filter: > > > > > hunspell\_de: > > > > > type: hunspell > > > > > locale: de\_DE > > > > > ignoreCase: true > > > > > > > Cheers, > > > > > > > Jörg > > > > > > -- > > > > David Pilatohttp://dev.david.pilato.fr/ > > > > Twitter : @dadoonet ---
**Author:** ![Damien\_Hardy](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@Damien\_Hardy](https://discuss.elastic.co/u/Damien_Hardy)\ **Post date:** [January 2, 2012, 12:49pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/7 "2012-01-02T12:49:58Z")
On 29 déc 2011, 21:45, Jörg Prante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > Hi, > > because all of you are eager to keep up with Lucene 3.5 features, I just > wrote an Elasticsearch Hunspell Analysis plugin. > > Project URL:[GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > For discussion, seehttps://github.com/elasticsearch/elasticsearch/issues/646 > > Please note: included are hunspell dict/aff files from Chromium for > convenience. The license for the third-party files is a tri-license MPL > 1.1/GPL 2.0/LGPL. > > Example usage: > > index: > analysis: > filter: > hunspell\_de: > type: hunspell > locale: de\_DE > ignoreCase: true > > Cheers, > > Jörg Hello, Great job. But installation proccess is not working.. We miss the compiled jar available for downloading form github to install it on elasticsearch via the plugin utillity. Best regards, -- Damien ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 3, 2012, 8:38am UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/8 "2012-01-03T08:38:55Z")
Thank you for pointing this out. I uploaded a zip file elasticsearch- analysis-hunspell-1.0.0.zip to the github download area. Best regards, Jörg On Jan 2, 1:49 pm, Damien Hardy [damienhardy....@gmail.com](mailto:damienhardy....@gmail.com) wrote: > On 29 déc 2011, 21:45, Jörg Prante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > Hi, > > > because all of you are eager to keep up with Lucene 3.5 features, I just > > wrote an Elasticsearch Hunspell Analysis plugin. > > > Project URL:[GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > For discussion, seehttps://github.com/elasticsearch/elasticsearch/issues/646 > > > Please note: included are hunspell dict/aff files from Chromium for > > convenience. The license for the third-party files is a tri-license MPL > > 1.1/GPL 2.0/LGPL. > > > Example usage: > > > index: > > analysis: > > filter: > > hunspell\_de: > > type: hunspell > > locale: de\_DE > > ignoreCase: true > > > Cheers, > > > Jörg > > Hello, > > Great job. > But installation proccess is not working.. > We miss the compiled jar available for downloading form github to > install it on elasticsearch via the plugin utillity. > > Best regards, > > -- > Damien ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [January 26, 2012, 2:39pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/9 "2012-01-26T14:39:04Z")
Hi Jörg, I gave a hunspell plugin a try and have some doubts whether it can really qualify as a stemmer. The problem I see with it is that it can emit way too many different options for some terms (especially short one) that this can IMO seriously harm the relevancy. I was testing it for the Czech language but I guess the same situation is for some other languages as well (based on my short test English seems to work a lot better). I can clearly see benefit of hunspell as a spelling tool but stemmer? I am not familiar with hunspell API but are there any options that can influence the stemming process that might be useful to expose tinES plugin API as well? Regards, Lukas On Tue, Jan 3, 2012 at 9:38 AM, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Thank you for pointing this out. I uploaded a zip file elasticsearch- > analysis-hunspell-1.0.0.zip to the github download area. > > Best regards, > > Jörg > > On Jan 2, 1:49 pm, Damien Hardy [damienhardy....@gmail.com](mailto:damienhardy....@gmail.com) wrote: > > > On 29 déc 2011, 21:45, Jörg Prante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > > Hi, > > > > > because all of you are eager to keep up with Lucene 3.5 features, I > > > just > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > Project URL:[GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > For discussion, seehttps:// > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/646) > > > > > Please note: included are hunspell dict/aff files from Chromium for > > > convenience. The license for the third-party files is a tri-license MPL > > > 1.1/GPL 2.0/LGPL. > > > > > Example usage: > > > > > index: > > > analysis: > > > filter: > > > hunspell\_de: > > > type: hunspell > > > locale: de\_DE > > > ignoreCase: true > > > > > Cheers, > > > > > Jörg > > > > Hello, > > > > Great job. > > But installation proccess is not working.. > > We miss the compiled jar available for downloading form github to > > install it on elasticsearch via the plugin utillity. > > > > Best regards, > > > > -- > > Damien ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [January 26, 2012, 2:47pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/10 "2012-01-26T14:47:11Z")
Just to give an illustration, there is a czech word "rada" which in given context means "board" (but it can also mean "advice"). Hunspell with cs\_CZ locale yields the following terms: rada (board) rada (the same term but I guess it is meant that this time it means advice) raď (give advice - a verb) radon (radon - a noun) This really can not qualify as a stemmer. Regards, Lukas On Thu, Jan 26, 2012 at 3:39 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi Jörg, > > I gave a hunspell plugin a try and have some doubts whether it can really > qualify as a stemmer. The problem I see with it is that it can emit way too > many different options for some terms (especially short one) that this can > IMO seriously harm the relevancy. I was testing it for the Czech language > but I guess the same situation is for some other languages as well (based > on my short test English seems to work a lot better). > > I can clearly see benefit of hunspell as a spelling tool but stemmer? I am > not familiar with hunspell API but are there any options that can influence > the stemming process that might be useful to expose tinES plugin API as > well? > > Regards, > Lukas > > On Tue, Jan 3, 2012 at 9:38 AM, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > > > Thank you for pointing this out. I uploaded a zip file elasticsearch- > > analysis-hunspell-1.0.0.zip to the github download area. > > > > Best regards, > > > > Jörg > > > > On Jan 2, 1:49 pm, Damien Hardy [damienhardy....@gmail.com](mailto:damienhardy....@gmail.com) wrote: > > > > > On 29 déc 2011, 21:45, Jörg Prante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > > > > Hi, > > > > > > > because all of you are eager to keep up with Lucene 3.5 features, I > > > > just > > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > > > Project URL: > > > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > > > For discussion, seehttps:// > > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/646) > > > > > > > Please note: included are hunspell dict/aff files from Chromium for > > > > convenience. The license for the third-party files is a tri-license > > > > MPL > > > > 1.1/GPL 2.0/LGPL. > > > > > > > Example usage: > > > > > > > index: > > > > analysis: > > > > filter: > > > > hunspell\_de: > > > > type: hunspell > > > > locale: de\_DE > > > > ignoreCase: true > > > > > > > Cheers, > > > > > > > Jörg > > > > > > Hello, > > > > > > Great job. > > > But installation proccess is not working.. > > > We miss the compiled jar available for downloading form github to > > > install it on elasticsearch via the plugin utillity. > > > > > > Best regards, > > > > > > -- > > > Damien ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 26, 2012, 4:32pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/11 "2012-01-26T16:32:23Z")
Hi Lukáš, thanks for pointing this out. Yes, it's true, the use of hunspell for stemming must be carefully evaluated for each dictionary. See also the warnings in [http://wiki.apache.org/solr/HunspellStemFilterFactory](http://wiki.apache.org/solr/HunspellStemFilterFactory) Robert Muir gave caution about this in [[SOLR-2769] HunspellStemFilterFactory - ASF JIRA](https://issues.apache.org/jira/browse/SOLR-2769) I assume the czeck dictionary I found in Chromium is not the best choice. To be honest, I am just in the process of learning to write Elasticsearch plugins, and I started with a very tiny project. Most attractive was a feature that appeared in Lucene 3.5, the hunspell stem filter. In a more advanced dictionary plugin I am busy with, I will use hunspell dictionaries in the more appropriate way, that is, for spell suggestions. Best regard, Jörg On Jan 26, 3:47 pm, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > Just to give an illustration, there is a czech word "rada" which in given > context means "board" (but it can also mean "advice"). > Hunspell with cs\_CZ locale yields the following terms: > > rada (board) > rada (the same term but I guess it is meant that this time it means advice) > raď (give advice - a verb) > radon (radon - a noun) > > This really can not qualify as a stemmer. > > Regards, > Lukas > > On Thu, Jan 26, 2012 at 3:39 PM, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > > > Hi Jörg, > > > I gave a hunspell plugin a try and have some doubts whether it can really > > qualify as a stemmer. The problem I see with it is that it can emit way too > > many different options for some terms (especially short one) that this can > > IMO seriously harm the relevancy. I was testing it for the Czech language > > but I guess the same situation is for some other languages as well (based > > on my short test English seems to work a lot better). > > > I can clearly see benefit of hunspell as a spelling tool but stemmer? I am > > not familiar with hunspell API but are there any options that can influence > > the stemming process that might be useful to expose tinES plugin API as > > well? > > > Regards, > > Lukas > > > On Tue, Jan 3, 2012 at 9:38 AM, jprante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > Thank you for pointing this out. I uploaded a zip file elasticsearch- > > > analysis-hunspell-1.0.0.zip to the github download area. > > > > Best regards, > > > > Jörg > > > > On Jan 2, 1:49 pm, Damien Hardy [damienhardy....@gmail.com](mailto:damienhardy....@gmail.com) wrote: > > > > > > > On 29 déc 2011, 21:45, Jörg Prante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > > > Hi, > > > > > > because all of you are eager to keep up with Lucene 3.5 features, I > > > > > just > > > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > > Project URL: > > > > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > > For discussion, seehttps:// > > > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/646) > > > > > > Please note: included are hunspell dict/aff files from Chromium for > > > > > convenience. The license for the third-party files is a tri-license > > > > > MPL > > > > > 1.1/GPL 2.0/LGPL. > > > > > > Example usage: > > > > > > index: > > > > > analysis: > > > > > filter: > > > > > hunspell\_de: > > > > > type: hunspell > > > > > locale: de\_DE > > > > > ignoreCase: true > > > > > > Cheers, > > > > > > Jörg > > > > > Hello, > > > > > Great job. > > > > But installation proccess is not working.. > > > > We miss the compiled jar available for downloading form github to > > > > install it on elasticsearch via the plugin utillity. > > > > > Best regards, > > > > > -- > > > > Damien ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [January 26, 2012, 4:56pm UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/12 "2012-01-26T16:56:56Z")
Sounds cool, and thanks for the effort! Lukas On Thu, Jan 26, 2012 at 5:32 PM, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Hi Lukáš, > > thanks for pointing this out. Yes, it's true, the use of hunspell for > stemming must be carefully evaluated for each dictionary. See also the > warnings in > > [HunspellStemFilterFactory - Solr - Apache Software Foundation](http://wiki.apache.org/solr/HunspellStemFilterFactory) > > Robert Muir gave caution about this in > [[SOLR-2769] HunspellStemFilterFactory - ASF JIRA](https://issues.apache.org/jira/browse/SOLR-2769) > > I assume the czeck dictionary I found in Chromium is not the best > choice. > > To be honest, I am just in the process of learning to write > Elasticsearch plugins, and I started with a very tiny project. Most > attractive was a feature that appeared in Lucene 3.5, the hunspell > stem filter. > > In a more advanced dictionary plugin I am busy with, I will use > hunspell dictionaries in the more appropriate way, that is, for spell > suggestions. > > Best regard, > > Jörg > > On Jan 26, 3:47 pm, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > > > Just to give an illustration, there is a czech word "rada" which in given > > context means "board" (but it can also mean "advice"). > > Hunspell with cs\_CZ locale yields the following terms: > > > > rada (board) > > rada (the same term but I guess it is meant that this time it means > > advice) > > raď (give advice - a verb) > > radon (radon - a noun) > > > > This really can not qualify as a stemmer. > > > > Regards, > > Lukas > > > > On Thu, Jan 26, 2012 at 3:39 PM, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) > > wrote: > > > > > Hi Jörg, > > > > > I gave a hunspell plugin a try and have some doubts whether it can > > > really > > > qualify as a stemmer. The problem I see with it is that it can emit > > > way too > > > many different options for some terms (especially short one) that this > > > can > > > IMO seriously harm the relevancy. I was testing it for the Czech > > > language > > > but I guess the same situation is for some other languages as well > > > (based > > > on my short test English seems to work a lot better). > > > > > I can clearly see benefit of hunspell as a spelling tool but stemmer? > > > I am > > > not familiar with hunspell API but are there any options that can > > > influence > > > the stemming process that might be useful to expose tinES plugin API as > > > well? > > > > > Regards, > > > Lukas > > > > > On Tue, Jan 3, 2012 at 9:38 AM, jprante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > > > Thank you for pointing this out. I uploaded a zip file elasticsearch- > > > > analysis-hunspell-1.0.0.zip to the github download area. > > > > > > Best regards, > > > > > > Jörg > > > > > > On Jan 2, 1:49 pm, Damien Hardy [damienhardy....@gmail.com](mailto:damienhardy....@gmail.com) wrote: > > > > > > > > > On 29 déc 2011, 21:45, Jörg Prante [joergpra...@gmail.com](mailto:joergpra...@gmail.com) wrote: > > > > > > > > Hi, > > > > > > > > because all of you are eager to keep up with Lucene 3.5 features, > > > > > > I > > > > > > just > > > > > > wrote an Elasticsearch Hunspell Analysis plugin. > > > > > > > > Project URL: > > > > > > [GitHub - jprante/elasticsearch-analysis-hunspell: Hunspell analysis for ElasticSearch](https://github.com/jprante/elasticsearch-analysis-hunspell) > > > > > > > > For discussion, seehttps:// > > > > > > [Analysis: Integration with Hunspell · Issue #646 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/646) > > > > > > > > Please note: included are hunspell dict/aff files from Chromium > > > > > > for > > > > > > convenience. The license for the third-party files is a > > > > > > tri-license > > > > > > MPL > > > > > > 1.1/GPL 2.0/LGPL. > > > > > > > > Example usage: > > > > > > > > index: > > > > > > analysis: > > > > > > filter: > > > > > > hunspell\_de: > > > > > > type: hunspell > > > > > > locale: de\_DE > > > > > > ignoreCase: true > > > > > > > > Cheers, > > > > > > > > Jörg > > > > > > > Hello, > > > > > > > Great job. > > > > > But installation proccess is not working.. > > > > > We miss the compiled jar available for downloading form github to > > > > > install it on elasticsearch via the plugin utillity. > > > > > > > Best regards, > > > > > > > -- > > > > > Damien ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:41am UTC](https://discuss.elastic.co/t/ann-elasticsearch-hunspell-analysis-plugin/6253/13 "2017-07-06T03:41:21Z")