# Elastic Search Similarity Not working

**URL:** <https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179>\
**Category:** Elasticsearch\
**Created:** [July 11, 2016, 10:46am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179 "2016-07-11T10:46:19Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 11, 2016, 10:46am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/1 "2016-07-11T10:46:19Z")

</div>

Hi

Elastic Search version : 1.7.2  
I tried giving the custom similarity in both setting and mappings.First I copied the plugin in the ES\_HOME/plugin diredtory (the jar file). Then I unzipped it in the same folder.  
Then I posted this in the myIndex/\_settings. But even after this the score was not at all [changing.It](http://changing.It) was the same as the previous. I had ovveriden tf, idf, coord methids and gave my values.But the score didnt change.  
{  
"myIndex": {  
"settings": {  
"index": {  
"creation\_date": "1467945023081",  
"number\_of\_shards": "5",  
"uuid": "Ja\_vqHohQjSZVnSKxgLBSw",  
"version": {  
"created": "1070299"  
},  
"number\_of\_replicas": "1",  
"similarity": {  
"index": {  
"type": "stefansavev.esplugins.OverlapSimilarityProvider"  
},  
"search": {  
"type": "stefansavev.esplugins.OverlapSimilarityProvider"  
}  
}  
}  
}  
}  
}

Thanks  
Nisar

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 14, 2016, 9:30am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/2 "2016-07-14T09:30:55Z")

</div>

Any body has an idea on the above question.

---

<div class="post-metadata">

**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\
**Post date:** [July 14, 2016, 9:35am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/3 "2016-07-14T09:35:36Z")

</div>

Can you post the code for your similarity please and all the exact steps you did to configure it?

Thanks

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 15, 2016, 5:08am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/4 "2016-07-15T05:08:35Z")

</div>

I directly ran it from postman..  
Below are the setting i have added.  
GET [http://localhost:9200/ppp6](http://localhost:9200/ppp6) ---\>\>This gives the below output

{  
"ppp6": {  
"aliases": {},  
"mappings": {  
"doc": {  
"properties": {  
"body": {  
"type": "string",  
"similarity": "overlapsimilarity"  
},  
"index": {  
"type": "string",  
"similarity": "overlapsimilarity"  
}  
}  
},  
"er": {  
"properties": {  
"index": {  
"type": "string"  
},  
"search": {  
"type": "string"  
}  
}  
}  
},  
"settings": {  
"index": {  
"creation\_date": "1468298653590",  
"number\_of\_shards": "5",  
"uuid": "XorHvK2RRrKft\_rnaJYMrg",  
"version": {  
"created": "1070299"  
},  
"number\_of\_replicas": "1",  
"similarity": {  
"overlapsimilarity": {  
"type": "stefansavev.esplugins.OverlapSimilarityProvider",  
"b": "0"  
}  
}  
}  
},  
"warmers": {}  
}  
}

I have placed the jar from the below given git hub project in the ES\_HOME/plugins directory [https://github.com/stefansavev/elasticsearch-custom-similarity-example](https://github.com/stefansavev/elasticsearch-custom-similarity-example)

Below are the data i have indexed.  
GET [http://localhost:9200/ppp6/\_search](http://localhost:9200/ppp6/_search)

{  
"took": 33,  
"timed\_out": false,  
"\_shards": {  
"total": 5,  
"successful": 5,  
"failed": 0  
},  
"hits": {  
"total": 9,  
"max\_score": 1,  
"hits": [  
{  
"\_index": "ppp6",  
"\_type": "er",  
"\_id": "AVXdbGIQH55KQhunBScf",  
"\_score": 1,  
"\_source": {  
"index": "abcgdfggas99",  
"search": "12145444"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "kk",  
"\_id": "4",  
"\_score": 1,  
"\_source": {  
"index": "zzz abc zzz",  
"body": "2323"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "er",  
"\_id": "AVXdbIGCH55KQhunBScg",  
"\_score": 1,  
"\_source": {  
"index": "xcvcx abc ccgdfggas99",  
"search": "12145444"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "kk",  
"\_id": "1",  
"\_score": 1,  
"\_source": {  
"index": "ABC D",  
"body": "77"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "er",  
"\_id": "AVXdbNADH55KQhunBSci",  
"\_score": 1,  
"\_source": {  
"index": "xcvcxabcgdfggas99",  
"search": "56566"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "er",  
"\_id": "AVXdbP4oH55KQhunBScj",  
"\_score": 1,  
"\_source": {  
"index": "abc"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "kk",  
"\_id": "2",  
"\_score": 1,  
"\_source": {  
"index": "RRFRF ABC DDFDF",  
"body": "11111"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "er",  
"\_id": "AVXdbJ3BH55KQhunBSch",  
"\_score": 1,  
"\_source": {  
"index": "abc asadfd dfdf dfsdfsfv sesfss",  
"search": "12145444"  
}  
},  
{  
"\_index": "ppp6",  
"\_type": "kk",  
"\_id": "3",  
"\_score": 1,  
"\_source": {  
"index": "index abc pdfodp sfsd",  
"body": "6546"  
}  
}  
]  
}  
}

Then I searched for this  
{"query":  
{ "match":  
{ "index":"abc"  
}  
}  
}

The result was the same with this similarity settings and without this settings.The score was the same in both the cases.

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 15, 2016, 5:10am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/5 "2016-07-15T05:10:08Z")

</div>

Would you please check. Is this expected. I was expecting different scores in both the cases.  
Or else can you give a valid example for us to test.

---

<div class="post-metadata">

**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\
**Post date:** [July 15, 2016, 6:59am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/6 "2016-07-15T06:59:35Z")

</div>

What you posted is the situation after you did all the steps. I am missing how you setup your mapping etc. but I can see that you get back some documents from type `er` and its fields are not picking up the custom similarity. Also, can you post your query? Which fields are you querying?

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 15, 2016, 7:05am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/7 "2016-07-15T07:05:10Z")

</div>

The document i am indexing has two fields : field1: index and field2:body  
Sample Doc: {  
"index": "ABC D",  
"body": "77"  
}

I am quertying on the field 'index'.. 'index' is my fieldname..dont get confused with elastic search index.

POST [http://localhost:9200/ppp6/\_search](http://localhost:9200/ppp6/_search)  
{"query":  
{ "match":  
{ "index":"abc"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\
**Post date:** [July 15, 2016, 7:20am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/8 "2016-07-15T07:20:22Z")

</div>

Ok, as I said, I see two types. `docs` and `er`. The former has the similarity properly setup on its `index` field, while `er` doesn't it seems. I would double check how you set up your mappings.

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 15, 2016, 7:26am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/9 "2016-07-15T07:26:54Z")

</div>

Indexed all the docs for index:ppp6 & type:doc

did the query  
POST [http://localhost:9200/ppp6/doc/\_search](http://localhost:9200/ppp6/doc/_search)  
{"query":{"match":{"index":"abc" } } }

below error came:  
{  
"took": 11,  
"timed\_out": false,  
"\_shards": {  
"total": 5,  
"successful": 4,  
"failed": 1,  
"failures": [  
{  
"index": "ppp6",  
"shard": 1,  
"status": 500,  
"reason": "QueryPhaseExecutionException[[ppp6][1]: query[filtered(index:abc)-\>cache(\_type:doc)],from[0],size[10]: Query Failed [Failed to execute main query]]; nested: AbstractMethodError; "  
}  
]  
},  
"hits": {  
"total": 0,  
"max\_score": null,  
"hits": []  
}  
}

---

<div class="post-metadata">

**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\
**Post date:** [July 15, 2016, 8:45am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/10 "2016-07-15T08:45:42Z")

</div>

Not sure about this error, it may just mean that you similarity kicked in but causes problems.

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 15, 2016, 9:51am UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/11 "2016-07-15T09:51:42Z")

</div>

There are some default similarities which we can try right. like BM25, DFR, IB etc..  
But all these gave the same score. Is it expected.

---

<div class="post-metadata">

**Author:** ![Nisar\_Adappadathil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nisar_adappadathil/32/13224_2.png) [@Nisar\_Adappadathil](https://discuss.elastic.co/u/Nisar_Adappadathil)\
**Post date:** [July 20, 2016, 1:55pm UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/12 "2016-07-20T13:55:06Z")

</div>

I need a custom similarity where in which only coord is taken into consideration. The TF-IDF should be ignored.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:34pm UTC](https://discuss.elastic.co/t/elastic-search-similarity-not-working/55179/13 "2017-07-05T22:34:02Z")

</div>