# Logstach Aggregation filter not working. Data is not displayed in nested columns

**URL:** <https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525>\
**Category:** Logstash\
**Created:** [April 18, 2019, 6:26pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525 "2019-04-18T18:26:55Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![aldol](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@aldol](https://discuss.elastic.co/u/aldol)\
**Post date:** [April 18, 2019, 6:26pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/1 "2019-04-18T18:26:55Z")

</div>

I have created the following aggregation in logstach. The data is not aggregated and is displayed next to the parent columns whereas they are supposed to be displayed in the nested columns:  
Here is the mapping before running the logstach:  
PUT s\_c\_nss  
{  
"mappings": {  
"doc": {  
"properties": {  
"assigns": {  
"type": "nested"  
},  
"others": {  
"type": "nested"  
}  
}  
}  
}  
}

* * *

Here is the filter in logstach:  
filter {  
aggregate {  
task\_id =\> "%{MyID}"  
code =\> "  
map['abc'] = event.get('abc')  
map['cate'] = event.get('cate')  
map['operator'] = event.get('operator')

```
  map['assigns_list'] ||= []
  map['assigns'] ||= []
  if (event.get('assigns_ID') != nil)
    if !( map['assigns_list'].include? event.get('assigns_ID') ) 
      map['assigns_list'] << event.get('assigns_ID')

      map['assigns'] << {
        'assigns.id' => event.get('assigns_ID'), 
        'assigns.abr' => event.get('abr'),
      }
    end
  end

  event.cancel()
"
push_previous_map_as_event => true
timeout => 5

```

}  
mutate {  
remove\_field =\> ["assigns\_list"]  
}  
}  
--- The result shows the 'assigns.id' and 'assigns.abr' next to 'abc' and 'cate' columns and not nested under "assigns'.

---

<div class="post-metadata">

**Author:** ![aldol](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@aldol](https://discuss.elastic.co/u/aldol)\
**Post date:** [April 18, 2019, 7:51pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/2 "2019-04-18T19:51:11Z")

</div>

I just did further trouble shooting and found out my filter is not functioning.  
Any help is appreciated.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 18, 2019, 8:10pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/3 "2019-04-18T20:10:32Z")

</div>

With that filter I get this. What don't you like about it?

```
{
      "cate" => "B",
   "assigns" => [
    [0] {
        "assigns.abr" => "F",
         "assigns.id" => "D"
    }
],
      "tags" => [
    [0] "_aggregatefinalflush"
],
  "operator" => "C",
       "abc" => "A"
}
```

---

<div class="post-metadata">

**Author:** ![aldol](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@aldol](https://discuss.elastic.co/u/aldol)\
**Post date:** [April 19, 2019, 5:13pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/4 "2019-04-19T17:13:39Z")

</div>

I don't get such a result. Can you share the complete conf file? It should be some stupid extra comma or something like this.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 19, 2019, 5:26pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/5 "2019-04-19T17:26:03Z")

</div>

```
input { generator { count => 1 message => '' } }

filter {
    mutate { add_field => { "abc" => "A" "cate" => "B" operator => "C" assigns_ID => "D" abr => "E" "MyID" => "F" } }
    aggregate {
    task_id => "%{MyID}"
    code => "
        map['abc'] = event.get('abc')
        map['cate'] = event.get('cate')
        map['operator'] = event.get('operator')

        map['assigns_list'] ||= []
        map['assigns'] ||= []
        if (event.get('assigns_ID') != nil)
            if !( map['assigns_list'].include? event.get('assigns_ID') )
                map['assigns_list'] << event.get('assigns_ID')

                map['assigns'] << {
                    'assigns.id' => event.get('assigns_ID'),
                    'assigns.abr' => event.get('abr'),
                }
            end
        end

        event.cancel()
    "
    push_previous_map_as_event => true
    timeout => 5
    }
}

output { stdout { codec => rubydebug { metadata => false } } }

```

gets me

```
{
         "abc" => "A",
    "operator" => "C",
"assigns_list" => [
    [0] "D"
],
     "assigns" => [
    [0] {
        "assigns.abr" => "E",
         "assigns.id" => "D"
    }
],
        "tags" => [
    [0] "_aggregatefinalflush"
],
        "cate" => "B",
[...]
}
```

---

<div class="post-metadata">

**Author:** ![aldol](https://avatars.discourse-cdn.com/v4/letter/a/9f8e36/32.png) [@aldol](https://discuss.elastic.co/u/aldol)\
**Post date:** [April 21, 2019, 11:01pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/6 "2019-04-21T23:01:46Z")

</div>

This helped me to troubleshoot my code and fix the issue.  
I owe you a loud "Thank you!!!"

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2019, 11:16pm UTC](https://discuss.elastic.co/t/logstach-aggregation-filter-not-working-data-is-not-displayed-in-nested-columns/177525/7 "2019-05-19T23:16:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.