Search returns fewers results than count

[jakehschwartz]

Using Elasticsearch 1.7.5

We were attempting to do a count query followed by a search query if a certain threshold was returned by the count but we saw the sometimes that could would say there are X documents, but when we did the search, the total was X-1.

Note: We saw the same behavior for search_type=count and using size=0 to return no results. Did not use _count API because I thought I read that it had different behavior.

Here are some logs:

Count:
request:
POST /2017-03-11/_search?search_type=count { "filter": { "and": [ { "query": { "query_string": { "query": "..." } } }, { "range": { "created_at": { "gte": "2017-03-11T04:38:13Z", "lt": "2017-03-11T04:40:12Z" } } } ] }, "sort": [ { "created_at": { "order": "desc" } } ], "size": 25, "_source": [ "field_names" ] }

response:
{"took":58,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":69,"max_score":0.0,"hits":[]}}

Search:
request:
POST /2017-03-11/_search { "filter": { "and": [ { "query": { "query_string": { "query": "..." } } }, { "range": { "created_at": { "gte": "2017-03-11T04:38:13Z", "lt": "2017-03-11T04:40:12Z" } } } ] }, "sort": [ { "created_at": { "order": "desc" } } ], "size": 25, "_source": [ "field_names" ] }

response:
{"took":35,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":68,"max_score":null,"hits":[ ... ]}}

Let me know if there would be any more useful information

Thanks

[spinscale (Alexander Reelsen)]

Hey,

can you try this with a preference and see if it yields the same results?

Maybe (note: wild assumption here) data in your shards is diverting.

P.S. Upgrading might be a wise idea (but I guess you knew that already

--Alex

[jakehschwartz]

Upgrading is unfortunately off the table for right now, but I'll try using a preference. This doesn't happen every time so I'll keep it running and see if it happens in the next 24 hours. Thanks.