Miscalculation on aggregations?


(Shin Tanimoto) #1

Hello,
I'm analyzing access logs with Elasticsearch 2.2 and Kibana 4.4.1. And I have trouble with calculation results of the filter aggregations.

For example, I created the "Data table" with following condition.

metrics

  • Aggregation : Unique count
  • Field : clientip

buckets

  • Split Table
    • Filters
      • useragent.device: "iPhone"
  • Split Row
    • Filters

(Actually I want to create filter of user agents and url paths)

I got "41,780" as the result count with my log collections.
Then when I remove the second filter, Filters of Split Row, I got "56,386".

I have understood that "" filter does nothing then the results should be same, but as a fact, they are different.
It seems doubled filter aggregations cause some calculation error. Of course not only "
" filter, but also other condition seems to cause miscalculation.

Do I misunderstand something? Or is there any difficulties with doubled filter aggregation?

Here I copy my queries with/without second filter.

Without second filter.

{
"query": {
"filtered": {
"query": {
"query_string": {
"analyze_wildcard": true,
"lowercase_expanded_terms": false,
"query": "*"
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": 1456106274723,
"lte": 1456711074723,
"format": "epoch_millis"
}
}
}
],
"must_not": []
}
}
}
},
"size": 0,
"aggs": {
"2": {
"filters": {
"filters": {
"iPhone": {
"query": {
"query_string": {
"query": "useragent.device: "iPhone"",
"analyze_wildcard": true,
"lowercase_expanded_terms": false
}
}
}
}
},
"aggs": {
"4": {
"cardinality": {
"field": "clientip"
}
}
}
}
}
}

With second filter.

{
"query": {
"filtered": {
"query": {
"query_string": {
"analyze_wildcard": true,
"lowercase_expanded_terms": false,
"query": ""
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": 1456106274723,
"lte": 1456711074723,
"format": "epoch_millis"
}
}
}
],
"must_not": []
}
}
}
},
"size": 0,
"aggs": {
"2": {
"filters": {
"filters": {
"iPhone": {
"query": {
"query_string": {
"query": "useragent.device: "iPhone"",
"analyze_wildcard": true,
"lowercase_expanded_terms": false
}
}
}
}
},
"aggs": {
"5": {
"filters": {
"filters": {
"
": {
"query": {
"query_string": {
"query": "*",
"analyze_wildcard": true,
"lowercase_expanded_terms": false
}
}
}
}
},
"aggs": {
"4": {
"cardinality": {
"field": "clientip"
}
}
}
}
}
}
}
}

Thank you!


(system) #2