# Problem with multiple aggregate function

**URL:** <https://discuss.elastic.co/t/problem-with-multiple-aggregate-function/177583>\
**Category:** Logstash\
**Created:** [April 19, 2019, 8:50am UTC](https://discuss.elastic.co/t/problem-with-multiple-aggregate-function/177583 "2019-04-19T08:50:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alankrit\_Aman\_Mishra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alankrit_aman_mishra/32/43504_2.png) [@Alankrit\_Aman\_Mishra](https://discuss.elastic.co/u/Alankrit_Aman_Mishra)\
**Post date:** [April 19, 2019, 8:50am UTC](https://discuss.elastic.co/t/problem-with-multiple-aggregate-function/177583/1 "2019-04-19T08:50:00Z")

</div>

Hello,  
I have logs that have a similar format.  
INFO - 98765 - http\_Method - POST  
WARN - 12345 - latency - 55  
ERROR - 12345 - status\_code - 200  
WARN - 98765 - latency - 99  
XYZ - 12345 - country - XYZ  
ERROR - 98765 - status\_code - 404  
INFO - 12345 - http\_Method - GET  
XYZ - 98765 - country - ABC

in short, logs may come in any sequence. we only have four fields to care about.  
We need to copy and paste data from the different fields to the logs file which contains the country field.

Expected Output:  
12345-http\_Method-GET-country-XYZ-status\_code-200-latency-55  
98765-http\_Method-POST-country-ABC-status\_code-404-latency-99

We are using 3 aggregate functions.

First aggregate function.  
if [level] == "INFO" {  
aggregate {  
task\_id =\> "%{api\_id}"  
code =\> "map['status\_code'] ||= 0 ; map['status\_code'] += event.get('status\_code')"  
}

if [level] == "XYZ" {  
aggregate {  
task\_id =\> "%{api\_id}"  
code =\> "event.set('status\_code', map['status\_code'])"  
}

Second aggregate function

if [level] == "WARN " {  
aggregate {  
task\_id =\> "%{api\_id}"  
code =\> "map['latency'] ||= 0 ; map['latency'] += event.get('latency')"  
}

if [level] == "XYZ" {  
aggregate {  
task\_id =\> "%{api\_id}"  
code =\> "event.set('latency', map['latency'])"  
}

Third aggregate function

if [level] == "ERROR " {  
aggregate {  
task\_id =\> "%{api\_id}"  
code =\> "map['status\_code'] ||= 0 ; map['status\_code'] += event.get('status\_code')"  
}

if [level] == "XYZ" {  
aggregate {  
task\_id =\> "%{api\_id}"  
code =\> "event.set('status\_code', map['status\_code'])"  
}

The issue is that in the output some times, all values are present, but many times all three values are not present.

Anyone has any idea or any kind of approach please let us know.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 19, 2019, 3:22pm UTC](https://discuss.elastic.co/t/problem-with-multiple-aggregate-function/177583/2 "2019-04-19T15:22:20Z")

</div>

I would suggest something like

```
    dissect { mapping => { "message" => "%{level} - %{api_id} - %{key} - %{value}" } }
    aggregate {
        task_id => "%{api_id}"
        code => '
            case event.get("key")
            when "http_Method"
                map["method"] = event.get("value")
            when "latency"
                map["latency"] = event.get("value").to_i
            when "status_code"
                map["status"] = event.get("value").to_i
            when "country"
                map["country"] = event.get("value")
            end
            event.cancel
        '
        push_map_as_event_on_timeout => true
        timeout_task_id_field => "api_id"
        timeout => 6
    }

```

If you require a specific output format you could rearrange the fields on the event in the timeout\_code option.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 17, 2019, 3:22pm UTC](https://discuss.elastic.co/t/problem-with-multiple-aggregate-function/177583/3 "2019-05-17T15:22:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.