# Synonyms not working on 0.17.8? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 12, 2011, 12:32am UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569 "2011-10-12T00:32:03Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [October 12, 2011, 12:32am UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/1 "2011-10-12T00:32:03Z")
Hello, I am running 0.17.8 in our dev environment and 0.16.5 in QA and production. I am doing some work around synonyms and noticed that they don't seem to be working on 0.17.8 or 0.17.7 (I had updated to .8 to check if this had been fixed). Here is a gist that shows what I am observing: > I was testing around punctuation in the synonym list, but even the simple case of work \<=\> business doesn't work on 0.17.8. Please let me know if there are any other details I can provide. Hopefully, I'm not missing something obvious. Thanks, Paul ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [October 12, 2011, 7:47am UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/2 "2011-10-12T07:47:09Z")
FYI, this was caused by updates in lucene3.4 to handle synonym files more efficiently. I made some tweaks and sent a pull request: > > > Here's a quick fix for synonym handling. Not too pretty, feel free to hack up to… your standards :-) As a side note, it'd be very nice to define a list of token filters to apply to the synonym list to ensure the behavior is in sync with the analysis that has occurred up to that point on the data. Thanks, Paul On Oct 11, 6:32 pm, ppearcy [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > Hello, > I am running 0.17.8 in our dev environment and 0.16.5 in QA and > production. I am doing some work around synonyms and noticed that they > don't seem to be working on 0.17.8 or 0.17.7 (I had updated to .8 to > check if this had been fixed). > > Here is a gist that shows what I am observing:[ES synonym failure on 0.17.8 · GitHub](https://gist.github.com/1279879) > > I was testing around punctuation in the synonym list, but even the > simple case of work \<=\> business doesn't work on 0.17.8. > > Please let me know if there are any other details I can provide. > Hopefully, I'm not missing something obvious. > > Thanks, > Paul ---
**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\ **Post date:** [October 12, 2011, 12:38pm UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/3 "2011-10-12T12:38:34Z")
On Wed, Oct 12, 2011 at 3:47 AM, ppearcy [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > FYI, this was caused by updates in lucene3.4 to handle synonym files This is not true. lucene did not have a multiword synonym capability before 3.4 -- > **[Home](https://lucidworks.com/)** > > Lucidworks' Fusion platform uses industry-leading search technology to power search & discovery for the largest & most successful companies. Request a demo today. ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [October 12, 2011, 4:21pm UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/4 "2011-10-12T16:21:19Z")
Yes, should have really said this was caused by elasticsearch updates to pull in lucene 3.4. Robert, do you have any further details on multiword synonym support? Is the link below the best reference or are there others? [http://lucene.apache.org/java/3\_4\_0/api/all/org/apache/lucene/analysis/synonym/SynonymFilter.html](http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/analysis/synonym/SynonymFilter.html) Was that added with this feature or is there another ticket tracking? [https://issues.apache.org/jira/browse/LUCENE-3233](https://issues.apache.org/jira/browse/LUCENE-3233) Thanks, Paul On Oct 12, 6:38 am, Robert Muir [rcm...@gmail.com](mailto:rcm...@gmail.com) wrote: > On Wed, Oct 12, 2011 at 3:47 AM, ppearcy [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > FYI, this was caused by updates in lucene3.4 to handle synonym files > > This is not true. lucene did not have a multiword synonym capability before 3.4 > > -- > [lucidimagination.com](http://lucidimagination.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 12, 2011, 10:21pm UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/5 "2011-10-12T22:21:40Z")
Grr, ugly bug. The fix is actually simpler, commented on the pull request. On Wed, Oct 12, 2011 at 9:47 AM, ppearcy [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > FYI, this was caused by updates in lucene3.4 to handle synonym files > more efficiently. I made some tweaks and sent a pull request: > [Fix synonym handling by ppearcy · Pull Request #1386 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/pull/1386) > > As a side note, it'd be very nice to define a list of token filters to > apply to the synonym list to ensure the behavior is in sync with the > analysis that has occurred up to that point on the data. > > Thanks, > Paul > > On Oct 11, 6:32 pm, ppearcy [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > Hello, > > I am running 0.17.8 in our dev environment and 0.16.5 in QA and > > production. I am doing some work around synonyms and noticed that they > > don't seem to be working on 0.17.8 or 0.17.7 (I had updated to .8 to > > check if this had been fixed). > > > > Here is a gist that shows what I am observing: > > [ES synonym failure on 0.17.8 · GitHub](https://gist.github.com/1279879) > > > > I was testing around punctuation in the synonym list, but even the > > simple case of work \<=\> business doesn't work on 0.17.8. > > > > Please let me know if there are any other details I can provide. > > Hopefully, I'm not missing something obvious. > > > > Thanks, > > Paul ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [October 13, 2011, 10:42am UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/6 "2011-10-13T10:42:26Z")
Hi, not an expert here but I tried to look quickly into Lucene 3.4/Solr code and it seems to me that synonym filter code has been ported from Solr to Lucene. This means that probably also Solr wiki applies as a documentation: [http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory](http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory) I haven't tried it myself yet. HTH, Regards, Lukas On Wed, Oct 12, 2011 at 6:21 PM, ppearcy [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > Yes, should have really said this was caused by elasticsearch updates > to pull in lucene 3.4. > > Robert, do you have any further details on multiword synonym support? > Is the link below the best reference or are there others? > > [SynonymFilter (Lucene 3.4.0 API)](http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/analysis/synonym/SynonymFilter.html) > > Was that added with this feature or is there another ticket tracking? > [[LUCENE-3233] HuperDuperSynonymsFilter™ - ASF JIRA](https://issues.apache.org/jira/browse/LUCENE-3233) > > Thanks, > Paul > > On Oct 12, 6:38 am, Robert Muir [rcm...@gmail.com](mailto:rcm...@gmail.com) wrote: > > > On Wed, Oct 12, 2011 at 3:47 AM, ppearcy [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > FYI, this was caused by updates in lucene3.4 to handle synonym files > > > > This is not true. lucene did not have a multiword synonym capability > > before 3.4 > > > > -- > > [lucidimagination.com](http://lucidimagination.com) ---
**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\ **Post date:** [October 13, 2011, 1:07pm UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/7 "2011-10-13T13:07:47Z")
On Thu, Oct 13, 2011 at 6:42 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi, > not an expert here but I tried to look quickly into Lucene 3.4/Solr code and > it seems to me that synonym filter code has been ported from Solr to Lucene. This is not true. This was rewritten from scratch. -- > **[Home](https://lucidworks.com/)** > > Lucidworks' Fusion platform uses industry-leading search technology to power search & discovery for the largest & most successful companies. Request a demo today. ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [October 13, 2011, 1:48pm UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/8 "2011-10-13T13:48:46Z")
Hi Robert, thanks for clarification! So if I understand it correctly, the Solr's SynonymFilterFactory delegates either to SlowSynonymFilter (if using Lucene \< 3.4) or to SynonymFilter (via FSTSynonymFilterFactory) which is located in Lucene module. And that Lucene module was written from scratch, correct? Which means that SynonymFilter JavaDoc mentioned by @ppearcy is probably the best source of up-to-date documentation about synonym filter for now (speaking from the side of Elasticsearch user). Lukas On Thu, Oct 13, 2011 at 3:07 PM, Robert Muir [rcmuir@gmail.com](mailto:rcmuir@gmail.com) wrote: > On Thu, Oct 13, 2011 at 6:42 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) > wrote: > > > Hi, > > not an expert here but I tried to look quickly into Lucene 3.4/Solr code > > and > > it seems to me that synonym filter code has been ported from Solr to > > Lucene. > > This is not true. This was rewritten from scratch. > > -- > [lucidimagination.com](http://lucidimagination.com) ---
**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\ **Post date:** [October 13, 2011, 1:58pm UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/9 "2011-10-13T13:58:20Z")
On Thu, Oct 13, 2011 at 9:48 AM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi Robert, > thanks for clarification! > So if I understand it correctly, the Solr's SynonymFilterFactory delegates > either to SlowSynonymFilter (if using Lucene \< 3.4) or to SynonymFilter (via > FSTSynonymFilterFactory) which is located in Lucene module. And that Lucene > module was written from scratch, correct? Which means that SynonymFilter > JavaDoc mentioned by @ppearcy is probably the best source of up-to-date > documentation about synonym filter for now (speaking from the side of > Elasticsearch user). exactly. Lucene didnt have a synonymfilter before (Except a very limited single-word one in the wordnet package, replaced by a wordnet PARSER for this filter) But, Solr had a synonymfilter before. there are a couple of corner cases where it didnt make sense to try to emulate the old solr functionality exactly for backwards compatibility, so instead we did this delegation trick so that solr users can continue to use the exact version they had before in those cases. for Lucene users its all new functionality... and javadocs are really the documentation for lucene, since its a library. -- > **[Home](https://lucidworks.com/)** > > Lucidworks' Fusion platform uses industry-leading search technology to power search & discovery for the largest & most successful companies. Request a demo today. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:52am UTC](https://discuss.elastic.co/t/synonyms-not-working-on-0-17-8/5569/10 "2017-07-06T03:52:06Z")