# How to get ES aggregation data as Logstash input?

**URL:** <https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776>\
**Category:** Logstash\
**Created:** [November 2, 2016, 10:18pm UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776 "2016-11-02T22:18:00Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![KIVagant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kivagant/32/7182_2.png) [@KIVagant](https://discuss.elastic.co/u/KIVagant)\
**Post date:** [November 2, 2016, 10:18pm UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776/1 "2016-11-02T22:18:01Z")

</div>

I have a Logstash configuration with the 'elasticsearch' input.

```
input {
  elasticsearch {
    hosts => "localhost"
    index => "logstash-*"
    type => 'aggregation_metric'
    size => 0
    query => '
      {
        "size": 0,
        "aggs": {
           ... here is multi-level aggregation without buckets
        }
      }'

```

I'm trying to send [aggregation result](https://discuss.elastic.co/t/how-to-get-dau-wau-charts-in-elasticsearch/64774) as a new document in new index of ES. I don't need any documents from the root level of searching, but maybe some of documents from [top hits aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-top-hits-aggregation.html)

How can I get the response without buckets as Logstash output?

> **Response example with expected value**
>
> ```
> {
> "took": 5,
> "timed_out": false,
> "_shards": {
> "total": 12,
> "successful": 12,
> "failed": 0
> },
> "hits": {
> "total": 198973,
> "max_score": 0,
> "hits": []
> },
> "aggregations": {
> "first": {
> "doc_count": 34924,
> "another_one": {
> "value": 18 <--- I need values like this one as a new documents 
> (that I want to create with mutate filter)
> }
> }
> }
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![KIVagant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kivagant/32/7182_2.png) [@KIVagant](https://discuss.elastic.co/u/KIVagant)\
**Post date:** [November 4, 2016, 5:24pm UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776/2 "2016-11-04T17:24:21Z")

</div>

My trick:

1. I wrote the php-utility that made multiple queries to ES with different console arguments tuning.
2. In logstash I used the ['exec' input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-exec.html)

If someone interesting:

> **Utility response example (new line separated)**
>
> ```
> {"id":1478231999,"created_at":"2016-11-03T23:59:59.000Z","one-chart-complex-value":17,"another-chart-value":3,"something-else-calculated":4}
> {"id":1478145599,"created_at":"2016-11-02T23:59:59.000Z","one-chart-complex-value":14,"another-chart-value":2,"something-else-calculated":5}
> 
> ```

> **Logstash exec plugin config**
>
> input {  
> exec {  
> command =\> '/etc/logstash/php/daily\_chart 5'  
> interval =\> 14400  
> type =\> 'daily\_chart'  
> codec =\> multiline {  
> pattern =\> "^\n"  
> what =\> "previous"  
> }  
> }  
> }  
> filter {  
> if [type] == "daily\_chart" {  
> json {  
> source =\> "message"  
> }  
> mutate  
> {  
> convert =\> ["created\_at", "string"]  
> remove\_field =\> ["command", "message", "host"]  
> }  
> date {  
> match =\> ["created\_at", "ISO8601"]  
> target =\> ["@timestamp"]  
> }  
> }  
> }  
> output {  
> if [type] == "daily\_chart" {  
> elasticsearch {  
> # common ES output  
> }  
> }  
> }

---

<div class="post-metadata">

**Author:** ![sharon.c](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharon.c/32/16076_2.png) [@sharon.c](https://discuss.elastic.co/u/sharon.c)\
**Post date:** [March 13, 2017, 8:47pm UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776/3 "2017-03-13T20:47:59Z")

</div>

I am also building a php utility so that logstash can invoke and get the aggregated result, would you please tell me where the php utility writes the aggregated response?

---

<div class="post-metadata">

**Author:** ![KIVagant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kivagant/32/7182_2.png) [@KIVagant](https://discuss.elastic.co/u/KIVagant)\
**Post date:** [March 14, 2017, 4:45pm UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776/4 "2017-03-14T16:45:24Z")

</div>

@sharon.c, the php tool writing results in stdout and ['exec' input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-exec.html) is reading this output.

---

<div class="post-metadata">

**Author:** ![sharon.c](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharon.c/32/16076_2.png) [@sharon.c](https://discuss.elastic.co/u/sharon.c)\
**Post date:** [March 14, 2017, 4:46pm UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776/5 "2017-03-14T16:46:16Z")

</div>

Got it, thank you very much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:27am UTC](https://discuss.elastic.co/t/how-to-get-es-aggregation-data-as-logstash-input/64776/6 "2017-07-06T04:27:52Z")

</div>