# 95th percentile aggregation for time series like documents

**URL:** <https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331>\
**Category:** Elasticsearch\
**Created:** [May 6, 2020, 11:36am UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331 "2020-05-06T11:36:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nroccolsw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nroccolsw/32/67693_2.png) [@nroccolsw](https://discuss.elastic.co/u/nroccolsw)\
**Post date:** [May 6, 2020, 11:36am UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331/1 "2020-05-06T11:36:50Z")

</div>

Hi there,

I am struggling with elastic search (and finally Grafana) to display the 95th percentile value for a given time period.

Consider the following setup:

I have an an index called traffic:

```auto
POST /traffic/_search

```

This index stores regular time series with exactly 5 minute intervals.

The document contains the following fields (left out others for brevity:

```auto
    {
        "@timestamp": "2020-04-02T00:00:00Z"
        ....
        "bytesInPerSecond": 1237832,
        "bytesOutPerSecond" 1232922,
        "interface": "eth0",
        "server": "my-db-server",
        ....
    },
    ....
    {
        "@timestamp": "2020-04-02T00:05:00Z"
        ....
        "bytesInPerSecond": 898239,
        "bytesOutPerSecond" 892,
        "interface": "eth1",
        ....
        "server": "my-db-server",
    }

```

I would like to have elastic search give me the 95th percentile for a given month (in my example for April).

```auto
    {
      "size": 0,
      "query": {
        "bool": {
          "filter": [
            {
              "range": {
                "@timestamp": {
                  "gte": 1585699200000,
                  "lte": 1588291120000,
                  "format": "epoch_millis"
                }
              }
            },
            {
              "query_string": {
                "analyze_wildcard": true,
                "query": "server:my-db-server"
              }
            }
          ]
        }
      },
      "aggs": {
        "prepare_the_data_aggregation": {
          "date_histogram": {
            "interval": "5m",
            "field": "@timestamp",
            "min_doc_count": 0,
            "extended_bounds": {
              "min": 1585699200000,
              "max": 1588291120000
            },
            "format": "epoch_millis"
          },
          "aggs": {
            "in": {
              "sum": {
                "field": "bytesInPerSecond"
              }
            },
            "out": {
              "sum": {
                "field": "bytesOutPerSecond"
              }
            }
          }
        },
        "95th_in": {
          "percentiles_bucket" : {
            "buckets_path": "prepare_the_data_aggregation>in",
            "percents": [95]
          }
        },
        "95th_out": {
          "percentiles_bucket" : {
            "buckets_path": "prepare_the_data_aggregation>out",
            "percents": [95]
          }
        }
      }
    }

```

The above query works but returns all the data for the 3 aggreggations: prepare\_the\_data\_aggregation, 95th\_in and 95th\_out.

Especially the data for the first aggregation _prepare\_the\_data\_aggregation_ is very large as it contains all the 5 minute data points for the entire month .

The only information I need is the result of _95th\_in_ and _95th\_out_. Is there a way for me tell elastic search that I only want those, and not the results of _prepare\_the\_data\_aggregation_?

Since this relies on _Percentiles Bucket Aggregation_ which is a form of _Pipeline Aggregations_, do you know if this kind of querying is also support via Grafana?

Thanks a lot for this amazing product.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [May 6, 2020, 2:23pm UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331/2 "2020-05-06T14:23:54Z")

</div>

Hi @nroccolsw,

do you just want to visualize this in just any tool or do you need to be able to query the 95th percentile from Elasticsearch to use somewhere?

If you just need to see this then a Timelion visualization in Kibana could probably show this from the raw data.

I know noting about _Pipeline Aggregations_ so can't comment on that 😬

---

<div class="post-metadata">

**Author:** ![nroccolsw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nroccolsw/32/67693_2.png) [@nroccolsw](https://discuss.elastic.co/u/nroccolsw)\
**Post date:** [May 6, 2020, 4:39pm UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331/3 "2020-05-06T16:39:05Z")

</div>

Hi A\_B,

I primarily need this via the http api to use it from our application code. So it is not ‘just visualizing’.

​

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [May 6, 2020, 4:51pm UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331/4 "2020-05-06T16:51:02Z")

</div>

Ok, then I know nothing that might help 🙂

---

<div class="post-metadata">

**Author:** ![nroccolsw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nroccolsw/32/67693_2.png) [@nroccolsw](https://discuss.elastic.co/u/nroccolsw)\
**Post date:** [May 13, 2020, 2:22pm UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331/5 "2020-05-13T14:22:52Z")

</div>

Is there maybe someone else here that can help me with my question?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 10, 2020, 2:22pm UTC](https://discuss.elastic.co/t/95th-percentile-aggregation-for-time-series-like-documents/231331/6 "2020-06-10T14:22:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.