# Synonym search filter
**URL:** https://discuss.elastic.co/t/synonym-search-filter/156400
**Category:** Elasticsearch
**Created:** [November 13, 2018, 7:33am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400 "2018-11-13T07:33:31Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@ronyarmon](https://discuss.elastic.co/u/ronyarmon)
#### Post date: [November 13, 2018, 7:33am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/1 "2018-11-13T07:33:31Z")
Hello, I'm testing a search engine that should retrieve documents with synonyms to the search term.
Version: 6.4.2
Example:
Search term='good'; Synonyms='right, in\_effect, proficient, in\_force, unspoiled'
Index: syn\_search\_test
Code:
```auto
PUT /syn_search_test
{
"settings": {
"index": {
"analysis": {
"filter": {
"search_synonym_filter": {
"type": "synonym",
"lenient": true,
"synonyms": ["right, in_effect, proficient, in_force, unspoiled"]
},
"analyzer": {
"search_synonyms": {
"type": "custom",
"tokenizer": "keyword",
"filter": ["lowercase", "search_synonym_filter"]
}
}
}
}
}
}
}
```
I'm getting the following error even after deleting and rebuilding the index with the documents:
> {
> "error": {
> "root\_cause": [
> {
> "type": "resource\_already\_exists\_exception",
> "reason": "index [syn\_search\_test/1Q38vCelTAuxVagoiKqRrg] already exists",
> "index\_uuid": "1Q38vCelTAuxVagoiKqRrg",
> "index": "syn\_search\_test"
> }
> ],
> "type": "resource\_already\_exists\_exception",
> "reason": "index [syn\_search\_test/1Q38vCelTAuxVagoiKqRrg] already exists",
> "index\_uuid": "1Q38vCelTAuxVagoiKqRrg",
> "index": "syn\_search\_test"
> },
> "status": 400
> }
Can you tell me what am I doing wrong?
Cheers,
Rony
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [November 13, 2018, 9:28am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/2 "2018-11-13T09:28:53Z")
> [@ronyarmon](#):
>
> "reason": "index [syn\_search\_test/1Q38vCelTAuxVagoiKqRrg] already exists",
This looks like the deletion of the index didn't work as expected. Which steps did you take to delete and reindex? What is the request where you are getting this error?
---
### Author:  [@ronyarmon](https://discuss.elastic.co/u/ronyarmon)
#### Post date: [November 13, 2018, 10:05am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/3 "2018-11-13T10:05:26Z")
I'm using requests (Python) as follows:
#delete the old version
response = requests.delete('[http://localhost:9200/syn\_search\_test?pretty](http://localhost:9200/syn_search_test?pretty)')
```
#create the new version
response = requests.put('http://localhost:9200/syn_search_test?pretty')
print (json.loads(json.dumps (response.text)))
#check indices list
response = requests.get('http://localhost:9200/_cat/indices?v')
print (json.loads(json.dumps (response.text)))
```
I'm getting the following response indicating that a new index was created. I'm using the same statements to delete and re-create the index which is searchable and seems to work fine:
> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
> yellow open syn\_search\_test 1Q38vCelTAuxVagoiKqRrg 5 1 0 0 1.1kb 1.1kb
> yellow open customer wol9nQkzQ9igbEqLlsA9HA 5 1 0 0 1.2kb 1.2kb
> yellow open emails fLI6MBkTSEG8qjMcaVnHpQ 5 1 0 0 1.2kb 1.2kb
> green open .kibana au3mvglaRF-nEuVIQeaRuw 1 0 3 0 14.8kb 14.8kb
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [November 13, 2018, 10:27am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/4 "2018-11-13T10:27:46Z")
> [@ronyarmon](#):
>
> PUT /syn\_search\_test { "settings": { "index": { "analysis": { "filter": { "search\_synonym\_filter": { "type": "synonym", "lenient": true, "synonyms": ["right, in\_effect, proficient, in\_force, unspoiled"] }, "analyzer": { "search\_synonyms": { "type": "custom", "tokenizer": "keyword", "filter": ["lowercase", "search\_synonym\_filter"] } } } } } }
So you issue the above PUT statement _after_ you have programatically created the index? That won't work because the index already exists (like the exception says). You either need to create the index programmatically with all the analysis settings already (don't know how this works with the python client to be honest) or you can [update the index analysis](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html) later, but you will need to close and later reopen the index and use the "\_settings" endpoint like so:
```auto
POST /syn_search_test/_close
PUT /syn_search_test/_settings
{
"analysis": {
"filter": {
"search_synonym_filter": {
"type": "synonym",
"lenient": true,
"synonyms": [
"right, in_effect, proficient, in_force, unspoiled"
]
}
},
"analyzer": {
"search_synonyms": {
"type": "custom",
"tokenizer": "keyword",
"filter": [
"lowercase",
"search_synonym_filter"
]
}
}
}
}
POST /syn_search_test/_open
```
---
### Author:  [@ronyarmon](https://discuss.elastic.co/u/ronyarmon)
#### Post date: [November 14, 2018, 7:03am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/5 "2018-11-14T07:03:25Z")
Thanks Christoph, I've missed that and updating did the trick. But I cannot use this statement to update the search criteria. My idea was that when searching for one of the words (say 'right') I'll get the sentences having with the other synonyms as a results.
But executing:
GET /syn\_search\_test/\_search
{
"query": {
"match": {
"text": {
"query": "in\_effect",
"analyzer": "search\_synonyms"
}
}
}
}
I'm getting:
> {
> "took": 1,
> "timed\_out": false,
> "\_shards": {
> "total": 5,
> "successful": 5,
> "skipped": 0,
> "failed": 0
> },
> "hits": {
> "total": 1,
> "max\_score": 0.9808292,
> "hits": [
> {
> "\_index": "syn\_search\_test",
> "\_type": "\_doc",
> "\_id": "2",
> "\_score": 0.9808292,
> "\_source": {
> "text": "This dog is the in\_effect one"
> }
> }
> ]
> }
> }
What am I doing wrong?
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [November 14, 2018, 7:54am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/6 "2018-11-14T07:54:21Z")
> [@ronyarmon](#):
>
> What am I doing wrong?
What the mapping for the "text" field and what is an example of a document you expect to find with the above query but don't?
---
### Author:  [@ronyarmon](https://discuss.elastic.co/u/ronyarmon)
#### Post date: [November 14, 2018, 11:15am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/7 "2018-11-14T11:15:28Z")
To test synonym search I produced and loaded the following sentences to the field text in each document: \>
> '\n{\n "text":"This dog is the well one" \n}',
> '\n{\n "text":"This dog is the in\_force one" \n}',
> '\n{\n "text":"This dog is the serious one" \n}',
> '\n{\n "text":"This dog is the undecomposed one" \n}',
> '\n{\n "text":"This dog is the commodity one" \n}',
> '\n{\n "text":"This dog is the honorable one" \n}',
> '\n{\n "text":"This dog is the skilful one" \n}',
> '\n{\n "text":"This dog is the dependable one" \n}',
> '\n{\n "text":"This dog is the expert one" \n}',
> '\n{\n "text":"This dog is the honest one" \n}'
The synonyms (updated in the filter): "well, in\_force, serious, undecomposed, commodity"
Executing:
GET /syn\_search\_test/\_search
{
"query": {
"match" : {
"text" : "well"
}
}
}
Gets:
> {
> "took": 3,
> "timed\_out": false,
> "\_shards": {
> "total": 5,
> "successful": 5,
> "skipped": 0,
> "failed": 0
> },
> "hits": {
> "total": 1,
> "max\_score": 0.6931472,
> "hits": [
> {
> "\_index": "syn\_search\_test",
> "\_type": "\_doc",
> "\_id": "1",
> "\_score": 0.6931472,
> "\_source": {
> "text": "This dog is the well one"
> }
> }
> ]
> }
> }
I want to get the documents where text= in\_force/serious,/ undecomposed/ commodity as well
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [November 14, 2018, 1:38pm UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/8 "2018-11-14T13:38:43Z")
> [@ronyarmon](#):
>
> The synonyms (updated in the filter): "well, in\_force, serious, undecomposed, commodity"
I might have missed it in your last response, but what is the mapping for the "text" field? Or the whole index for that matter (e.g. output of `GET /syn_search_test/_mapping`)
---
### Author:  [@ronyarmon](https://discuss.elastic.co/u/ronyarmon)
#### Post date: [November 14, 2018, 2:17pm UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/9 "2018-11-14T14:17:31Z")
```auto
{
"syn_search_test": {
"mappings": {
"_doc": {
"properties": {
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
```
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [November 14, 2018, 5:46pm UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/10 "2018-11-14T17:46:11Z")
> [@ronyarmon](#):
>
> I want to get the documents where text= in\_force/serious,/ undecomposed/ commodity as well
I tried to re-create your whole example now and all seems to work well for me, at least on 6.4.3.
See my the reproduction below to check where the differences might be? I haven't asked yet but which version of ES are you using?
```auto
DELETE /syn_search_test
PUT /syn_search_test
{
"settings": {
"index": {
"analysis": {
"filter": {
"search_synonym_filter": {
"type": "synonym",
"lenient": true,
"synonyms": [
"well, in_force, serious, undecomposed, commodity"
]
}
},
"analyzer": {
"search_synonyms": {
"type": "custom",
"tokenizer": "keyword",
"filter": [
"lowercase",
"search_synonym_filter"
]
}
}
}
}
}
}
POST /syn_search_test/_doc/_bulk
{ "index" : { "_id" : "1" } }
{ "text":"This dog is the well one"}
{ "index" : { "_id" : "2" } }
{ "text":"This dog is the in_force one"}
{ "index" : { "_id" : "3" } }
{ "text":"This dog is the serious one" }
{ "index" : { "_id" : "4" } }
{ "text":"This dog is the undecomposed one" }
{ "index" : { "_id" : "5" } }
{ "text":"This dog is the commodity one" }
{ "index" : { "_id" : "6" } }
{ "text":"This dog is the honorable one" }
{ "index" : { "_id" : "7" } }
{ "text":"This dog is the skilful one" }
{ "index" : { "_id" : "8" } }
{ "text":"This dog is the dependable one" }
{ "index" : { "_id" : "9" } }
{ "text":"This dog is the expert one" }
{ "index" : { "_id" : "10" } }
{ "text":"This dog is the honest one" }
GET /syn_search_test/_search
{
"query": {
"match": {
"text": {
"query": "well",
"analyzer": "search_synonyms"
}
}
}
}
```
Gives:
```auto
{
"took": 11,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 5,
"max_score": 1.2039728,
"hits": [
{
"_index": "syn_search_test",
"_type": "_doc",
"_id": "5",
"_score": 1.2039728,
"_source": {
"text": "This dog is the commodity one"
}
},
{
"_index": "syn_search_test",
"_type": "_doc",
"_id": "2",
"_score": 0.9808292,
"_source": {
"text": "This dog is the in_force one"
}
},
{
"_index": "syn_search_test",
"_type": "_doc",
"_id": "4",
"_score": 0.9808292,
"_source": {
"text": "This dog is the undecomposed one"
}
},
{
"_index": "syn_search_test",
"_type": "_doc",
"_id": "1",
"_score": 0.6931472,
"_source": {
"text": "This dog is the well one"
}
},
{
"_index": "syn_search_test",
"_type": "_doc",
"_id": "3",
"_score": 0.2876821,
"_source": {
"text": "This dog is the serious one"
}
}
]
}
}
```
---
### Author:  [@ronyarmon](https://discuss.elastic.co/u/ronyarmon)
#### Post date: [November 15, 2018, 6:53am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/11 "2018-11-15T06:53:57Z")
Brilliant, problem solved though we did use the same statements to (re)produce the index. I was using 6.4.2 and upgraded to 6.5 where I ran your script. Could it be a version issue? In any case, many thanks for your help.
---
### Author:  [@cbuescher](https://discuss.elastic.co/u/cbuescher)
#### Post date: [November 15, 2018, 9:38am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/12 "2018-11-15T09:38:40Z")
I used 6.4.3 when trying and I don't think it differs at all from 6.4.2 in those regards. So I think something might have been off somewhere else, but great it works now.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 13, 2018, 9:38am UTC](https://discuss.elastic.co/t/synonym-search-filter/156400/13 "2018-12-13T09:38:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.