Percolate API issues

Elastic Stack Elasticsearch
1

we have about 3000 percolation API's in a single index with nothing else in it. I am facing a very strange situation, where in once i did a rolling restart of the servers in the cluster, the cluster is not matching the documents against the percolation api.

I have taken sample production percolation queries and put them in test cluster with the same document against the test cluster the match happens but not in production.

I have checked index settings, user mappings and same docs and queries, There are no other specific settings in elastic search.yaml. there are no errors in logs in production.

running on ES 1.7.5 -- using geo_point type mapping heavily in percolation Queries. Can someone give me any direction as what next could i do to debug this ? Thanks in advance !

2

Strange, that sounds like a problem.

It sounds like you might have, but did you take a query from the percolator and manually run it to see if it matches documents

Also, when you did the rolling restart, was anything changed such as upgrading ES? Or was it just a regular rolling restart?

3

I thought I had replied to it, doesn't look like i pressed "Reply" :(, but the problem was as below

we had 1 index with 1 mapping type named "user_profiles", we had about 3000 percolator queries under there index. I don't know how but by someone's actions there was now another mapping under the same index by name "user_profile" and it looked like was dynamically created such that all geo_type mapping was converted as type "long". The main problem seemed to be that all the percolator queries now suddenly got associated with the user_proflie mapping rather than user_profiles and hence no matter what document we threw at it nothing was matched. as we hit {host}:{port}/index1/user_profiles/_percolate

To Fix i deleted the user_profile mapping and did rolling restart of nodes and everything went back to normal.

To answer your question @dakrone as to why was original rolling restart done:

we had 5 ES data nodes and 2 of the 5 had version 1.7.4 and other 3 had 1.7.5. I upgraded the 2 nodes to 1.7.5 and did a rolling restart of all nodes starting from the upgrade nodes.

4

Okay, glad to hear it's fixed now!