Aggregation query

Elastic Stack Elasticsearch
1

When I run the following query on a 5 shard ES db I don't get accurate
results. I have had to reduce the amount of shards on my ES server to 1 to
get the accuracy I need? Has anyone had a similar issue?

GET /incidents/_search?search_type=count
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must": {
"range" : {
"Date" : {
"from" : "2014-08-05T00:00:00.000Z",
"to" : "2014-08-06T00:00:00.000Z"
}
}
},
"must": {
"exists" : { "field" : "AttackTypes" }
}
}
}
}
},
"aggs": {
"by_attackType" : {
"terms": {
"field": "AttackTypes",
"order": {
"_count": "desc"
},
"shard_size": 0,
"size": 10
},
"aggs": {
"by_perpertrator" : {
"terms": {
"field": "Perpetrators",
"order": {
"_term": "asc"
},
"min_doc_count": 0,
"shard_size": 0,
"size": 0
}
}
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cf1f5980-717d-4da9-b55b-3262a284e144%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hello Ivan ,

This is expected.
Only the top N(size mentioned in aggregation) results are taken from each
shard before reducing the result.
Due this , the accuracy is not guaranteed but the order is guaranteed.
As a fix , you can use this to improve accuracy at the cost of memory -

Thanks
Vineeth

On Fri, Aug 15, 2014 at 9:16 PM, Ivan Stone reachout2me@gmail.com wrote:

When I run the following query on a 5 shard ES db I don't get accurate
results. I have had to reduce the amount of shards on my ES server to 1 to
get the accuracy I need? Has anyone had a similar issue?

GET /incidents/_search?search_type=count
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must": {
"range" : {
"Date" : {
"from" : "2014-08-05T00:00:00.000Z",
"to" : "2014-08-06T00:00:00.000Z"
}
}
},
"must": {
"exists" : { "field" : "AttackTypes" }
}
}
}
}
},
"aggs": {
"by_attackType" : {
"terms": {
"field": "AttackTypes",
"order": {
"_count": "desc"
},
"shard_size": 0,
"size": 10
},
"aggs": {
"by_perpertrator" : {
"terms": {
"field": "Perpetrators",
"order": {
"_term": "asc"
},
"min_doc_count": 0,
"shard_size": 0,
"size": 0
}
}
}
}
}
}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cf1f5980-717d-4da9-b55b-3262a284e144%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cf1f5980-717d-4da9-b55b-3262a284e144%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m%3DTcrAtwCO7ovfjRgnO%2BbmxP8DEK%2BRmUvEsyE-1taYcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.