Help with the percentiles aggregation

Elastic Stack Elasticsearch
1

Hi,

Am trying to run a single command which calculates percentiles for multiple
search queries.
The data for this is an Apache log file, and I want to get the percentile
response times for the gets, posts, heads (etc) in one go

If I run this:
curl -XPOST 'http://localhost:9200/_search?search_type=count&pretty=true'
-d '{
"facets": {
"0": {"query" : {"term" : { "verb" : "get" }}},
"1": {"query" : {"term" : { "verb" : "post" }}}
},
"aggs" : {"load_time_outlier" : {"percentiles" : {"field" :
"responsetime"}}}
}'

The response I get back has the counts for each subquery but only does the
aggregations for the overall dataset
"facets" : {
"0" : {
"_type" : "query",
"count" : 5678
},
"1" : {
"_type" : "query",
"count" : 1234
}
},
"aggregations" : {
"load_time_outlier" : {
"values" : {
"1.0" : 0.0,
...
"99.0" : 1234
}
}
}

I cant figure out how to structure the request so that I get the
percentiles separately for each of the queries

Could someone point me in in the right direction please

Many thanks
John

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9d9696cb-adfa-4812-bd81-5efee0d29032%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi John,

You should be able to do something like:

{
"aggs": {
"verb": {
"terms": {
"field": "verb"
},
"aggs": {
"load_time_outliers": {
"percentiles": {
"field": "responsetime"
}
}
}
}
}
}

This will first break down your documents according to the http verb that
is being used and then compute percentiles separately for each unique verb.

On Fri, Aug 15, 2014 at 11:23 AM, John Ogden johnog65536@gmail.com wrote:

Hi,

Am trying to run a single command which calculates percentiles for
multiple search queries.
The data for this is an Apache log file, and I want to get the percentile
response times for the gets, posts, heads (etc) in one go

If I run this:
curl -XPOST 'http://localhost:9200/_search?search_type=count&pretty=true'
-d '{
"facets": {
"0": {"query" : {"term" : { "verb" : "get" }}},
"1": {"query" : {"term" : { "verb" : "post" }}}
},
"aggs" : {"load_time_outlier" : {"percentiles" : {"field" :
"responsetime"}}}
}'

The response I get back has the counts for each subquery but only does the
aggregations for the overall dataset
"facets" : {
"0" : {
"_type" : "query",
"count" : 5678
},
"1" : {
"_type" : "query",
"count" : 1234
}
},
"aggregations" : {
"load_time_outlier" : {
"values" : {
"1.0" : 0.0,
...
"99.0" : 1234
}
}
}

I cant figure out how to structure the request so that I get the
percentiles separately for each of the queries

Could someone point me in in the right direction please

Many thanks
John

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9d9696cb-adfa-4812-bd81-5efee0d29032%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9d9696cb-adfa-4812-bd81-5efee0d29032%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5JwTLK2q10fEKX6bVBzYH69dSRgA2njoEvhhronqfh1A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

That's spot on. Thanks!
On 18 Aug 2014 09:08, "Adrien Grand" adrien.grand@elasticsearch.com wrote:

Hi John,

You should be able to do something like:

{
"aggs": {
"verb": {
"terms": {
"field": "verb"
},
"aggs": {
"load_time_outliers": {
"percentiles": {
"field": "responsetime"
}
}
}
}
}
}

This will first break down your documents according to the http verb that
is being used and then compute percentiles separately for each unique verb.

On Fri, Aug 15, 2014 at 11:23 AM, John Ogden johnog65536@gmail.com
wrote:

Hi,

Am trying to run a single command which calculates percentiles for
multiple search queries.
The data for this is an Apache log file, and I want to get the percentile
response times for the gets, posts, heads (etc) in one go

If I run this:
curl -XPOST 'http://localhost:9200/_search?search_type=count&pretty=true'
-d '{
"facets": {
"0": {"query" : {"term" : { "verb" : "get" }}},
"1": {"query" : {"term" : { "verb" : "post" }}}
},
"aggs" : {"load_time_outlier" : {"percentiles" : {"field" :
"responsetime"}}}
}'

The response I get back has the counts for each subquery but only does
the aggregations for the overall dataset
"facets" : {
"0" : {
"_type" : "query",
"count" : 5678
},
"1" : {
"_type" : "query",
"count" : 1234
}
},
"aggregations" : {
"load_time_outlier" : {
"values" : {
"1.0" : 0.0,
...
"99.0" : 1234
}
}
}

I cant figure out how to structure the request so that I get the
percentiles separately for each of the queries

Could someone point me in in the right direction please

Many thanks
John

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9d9696cb-adfa-4812-bd81-5efee0d29032%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9d9696cb-adfa-4812-bd81-5efee0d29032%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/6tHMOeWYtoo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5JwTLK2q10fEKX6bVBzYH69dSRgA2njoEvhhronqfh1A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5JwTLK2q10fEKX6bVBzYH69dSRgA2njoEvhhronqfh1A%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGfq%3DRjVu58Jetkgf%3DGvJ4BkLjhWYPvm789UGPrr0U%2BOiA_Wxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Slight follow on - do you know if returning this sort of stuff via Kibana
is on the cards?
Just looking for an easy way to graph the results.

Thanks.

On Friday, 15 August 2014 10:23:16 UTC+1, John Ogden wrote:

Hi,

Am trying to run a single command which calculates percentiles for
multiple search queries.
The data for this is an Apache log file, and I want to get the percentile
response times for the gets, posts, heads (etc) in one go

If I run this:
curl -XPOST 'http://localhost:9200/_search?search_type=count&pretty=true'
-d '{
"facets": {
"0": {"query" : {"term" : { "verb" : "get" }}},
"1": {"query" : {"term" : { "verb" : "post" }}}
},
"aggs" : {"load_time_outlier" : {"percentiles" : {"field" :
"responsetime"}}}
}'

The response I get back has the counts for each subquery but only does the
aggregations for the overall dataset
"facets" : {
"0" : {
"_type" : "query",
"count" : 5678
},
"1" : {
"_type" : "query",
"count" : 1234
}
},
"aggregations" : {
"load_time_outlier" : {
"values" : {
"1.0" : 0.0,
...
"99.0" : 1234
}
}
}

I cant figure out how to structure the request so that I get the
percentiles separately for each of the queries

Could someone point me in in the right direction please

Many thanks
John

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/579dad15-4470-4f0d-a787-9b51fd7b447a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Support for aggregations is indeed something that is on the roadmap for the
next version of Kibana (Kibana 4), see this message from Rashid:
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/elasticsearch/I7um1mX4GSk/aUsT2EmyxysJ

On Mon, Aug 18, 2014 at 4:33 PM, John Ogden johnog65536@gmail.com wrote:

Slight follow on - do you know if returning this sort of stuff via Kibana
is on the cards?
Just looking for an easy way to graph the results.

Thanks.

On Friday, 15 August 2014 10:23:16 UTC+1, John Ogden wrote:

Hi,

Am trying to run a single command which calculates percentiles for
multiple search queries.
The data for this is an Apache log file, and I want to get the percentile
response times for the gets, posts, heads (etc) in one go

If I run this:
curl -XPOST 'http://localhost:9200/_search?search_type=count&pretty=true'
-d '{
"facets": {
"0": {"query" : {"term" : { "verb" : "get" }}},
"1": {"query" : {"term" : { "verb" : "post" }}}
},
"aggs" : {"load_time_outlier" : {"percentiles" : {"field" :
"responsetime"}}}
}'

The response I get back has the counts for each subquery but only does
the aggregations for the overall dataset
"facets" : {
"0" : {
"_type" : "query",
"count" : 5678
},
"1" : {
"_type" : "query",
"count" : 1234
}
},
"aggregations" : {
"load_time_outlier" : {
"values" : {
"1.0" : 0.0,
...
"99.0" : 1234
}
}
}

I cant figure out how to structure the request so that I get the
percentiles separately for each of the queries

Could someone point me in in the right direction please

Many thanks
John

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/579dad15-4470-4f0d-a787-9b51fd7b447a%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/579dad15-4470-4f0d-a787-9b51fd7b447a%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5pu8of4T06R8nVv1%3DvBy3wrX5Oqqowwhiiiqv5jhyK0w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.