q={"aggs"=>{"ids"=>{"terms"=>{"field"=>"id", "size"=>10000}}}}
I am using ES 1.7.4
a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search' -d '#{q.to_json}')['aggregations']['ids']['buckets'].count
s_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search?q=_exists_:id')["hits"]
^^ Both are returning different result. s_count is correct and a_count is lower than s_count. Aggregation is returning all buckets but ids are empty.
After doing some research, I found that aggregation is skipping some shards.
But If I reindex or migrate the index to different cluster then count is same for search and aggregation.
I have no idea what may be causing this. Any help would be appreciated.
EDIT: I did this query in row
a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search' -d '#{q.to_json}')['aggregations']['ids']['buckets'].count
--> 4
a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search' -d '#{q.to_json}')['aggregations']['ids']['buckets'].count
--> 3
a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search' -d '#{q.to_json}')['aggregations']['ids']['buckets'].count
--> 4
a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search' -d '#{q.to_json}')['aggregations']['ids']['buckets'].count
-> 3
How same query is returning two results when executed again and again?
did some more research
irb(main):184:0* a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search?routing=5' -d '#{q.to_json}')['aggregations']['ids']['buckets']
=> [{"key"=>3435973840252, "doc_count"=>1}]
irb(main):185:0> a_count = JSON.parse(curl -s 'http://localhost:9200}/index/_search?routing=5' -d '#{q.to_json}')['aggregations']['ids']['buckets']
=> []