Pipeline Aggregation

Elastic Stack Elasticsearch
1

How do I get average of transcations(count) per minute over a large set of data.

I am currently using this:

curl -XGET 'localhost:9200/splunkindex/_search?scroll=10m&pretty' -d '{ "query": { "filtered": { "query": { "match_all": {} } } }, {
"aggs" : {
"requests_per_minute" : {
"date_histogram" : {
"field" : "timestamp",
"interval" : "minute"
},
"aggs": {
"requests": {
"sum": {
"field": "txn"
}
}
}
},
"avg_minutely_requests": {
"avg_bucket": {
"buckets_path": "requests_per_minute>requests"
}
}
}
} } ' | jq '.hits.hits[]._source' | jq 'keys'

But it fails.
txn is a string - transaction name
I have to find number of occurance of transcation per minute

2

Not sure if I fully understand what you are wanting to do but if you are wanting to average the count of transactions per minute then you could use the value_count aggregation instead of the sum aggregation inside your date_histogram aggregation

3

Hello Colin,

Thanks for the correct response.
My use case is now that I got the number of transactions per second using value_count. I need to find the percentage of requests/second falling under say 0-2 , 2-4 , 4-6 , 6-8, 8-10.
That is, what is the percentage of requests /second falling under those ranges.
How do I compute this?

curl -XGET 'localhost:9200/splunkjob/_search?size=1&pretty=true' -d '{ "query": { "filtered": { "query": { "bool": { "must": [ { "match": { "txn": "/api/secure/matcher" } } ] } } } },
"aggs" : {
"requests_per_second" : {
"date_histogram" : {
"field" : "@timestamp",
"interval" : "second"
},
"aggs": {
"requests": {
"value_count": {
"field": "txn"
}
}
}
},
"stats_secondly_requests": {
"stats_bucket": {
"buckets_path": "requests_per_second>requests"
}
}
}
} } '

Here requests -> gives me the request per second.
Using the stats_bucket, I am able to find count,min,max,sum,average.
Now i need to find the percentages of requests/second under 0-2 , 2-4 , 4-6 , 6-8, 8-10(dynamic ranges) etc requests/second. Is this possible using Elasticsearch.

4

Unfortunately we don't currently have a percentile_ranks_bucket pipeline aggregation which is the aggregation you would need to do this. We do have a percentiles_bucket aggregation which answers the opposite question: How much requests/second does 20%, 50%, 90% etc. of the values fall under.