# Filter with select in logstash

**URL:** <https://discuss.elastic.co/t/filter-with-select-in-logstash/253732>\
**Category:** Logstash\
**Created:** [October 29, 2020, 4:42pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732 "2020-10-29T16:42:20Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![juuuhuuu](https://avatars.discourse-cdn.com/v4/letter/j/958977/32.png) [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\
**Post date:** [October 29, 2020, 4:42pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/1 "2020-10-29T16:42:20Z")

</div>

Hi Community,  
I would like to have a filter to dissect my logs and aggregate them like this

> SELECT COUNT(CustomerID), Country  
> FROM Customers  
> GROUP BY Country;

Could you please help me?

=\> in the logname are countries for example like: user 123 in Germany logelevel: Info /path...

```
       filter {
dissect {mapping => {"message" => "[%{logdate}] %{application}.%{loglevel}: %{logname} /%{path}"}}
       
 if "info" in [loglevel] {
        
          aggregate {
               task_id => "%{logname}"
               code => "
                 map['logname'] ||= 0; map['logname'] += 1;
                 map['logname'] ||= event.get('logname')
                 map['loglevel'] ||= []
                 map['loglevel'] << {'loglevel' => event.get('loglevel')}
                 event.cancel()
               "
               push_previous_map_as_event => true
               timeout => 3
             }
         }else{
         dissect { mapping => { "message" => "[%{logdate}] %{exception}.%{errorlogname}.%{errorCode}.%{loglevel}: %{errorMessage} /%{path}" } }
        }

```

---

<div class="post-metadata">

**Author:** ![wangqinghuan](https://avatars.discourse-cdn.com/v4/letter/w/d26b3c/32.png) [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)\
**Post date:** [October 30, 2020, 6:33am UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/2 "2020-10-30T06:33:19Z")

</div>

Why don't do it in Elasticsearch?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 30, 2020, 5:08pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/3 "2020-10-30T17:08:52Z")

</div>

What does the data you are trying to aggregate look like?

---

<div class="post-metadata">

**Author:** ![juuuhuuu](https://avatars.discourse-cdn.com/v4/letter/j/958977/32.png) [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\
**Post date:** [November 2, 2020, 12:01pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/4 "2020-11-02T12:01:22Z")

</div>

[2020-11-02 12:44:04] application.INFO: Today is beautiful weather /path/....  
[2020-11-02 12:44:05] application.INFO: Today is beautiful weather /path/....  
[2020-11-02 12:44:06] application.INFO: today rainbow /path/....  
[2020-11-02 12:44:06] Chttpexception.ERROR: 34 rainy day /path/....

---

<div class="post-metadata">

**Author:** ![juuuhuuu](https://avatars.discourse-cdn.com/v4/letter/j/958977/32.png) [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\
**Post date:** [November 2, 2020, 12:11pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/5 "2020-11-02T12:11:22Z")

</div>

How 🙂

---

<div class="post-metadata">

**Author:** ![juuuhuuu](https://avatars.discourse-cdn.com/v4/letter/j/958977/32.png) [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\
**Post date:** [November 3, 2020, 10:11pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/6 "2020-11-03T22:11:42Z")

</div>

please help 😬

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 3, 2020, 10:47pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/7 "2020-11-03T22:47:27Z")

</div>

You could try

```
    dissect {mapping => {"message" => "[%{logdate}] %{application}.%{loglevel}: %{}"}}
    if "INFO" in [loglevel] {
        dissect {mapping => {"message" => "[%{logdate}] %{application}.%{loglevel}: %{logname} /%{path}"}}
        aggregate {
           task_id => "%{logname}"
           code => "
             map['logname'] ||= 0; map['logname'] += 1;
             map['logname'] ||= event.get('logname')
             map['loglevel'] ||= []
             map['loglevel'] << {'loglevel' => event.get('loglevel')}
             event.cancel()
           "
           push_previous_map_as_event => true
           timeout => 3
         }
    }else{
        dissect { mapping => { "message" => "[%{logdate}] %{application}.%{loglevel}: %{errorCode} %{errorMessage} /%{path}" } }
    }

```

Note that the

```
map['logname'] ||= event.get('logname')

```

will never perform the assignment, because the previous line has ensured that map['logname'] is never nil.

---

<div class="post-metadata">

**Author:** ![juuuhuuu](https://avatars.discourse-cdn.com/v4/letter/j/958977/32.png) [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\
**Post date:** [November 3, 2020, 11:47pm UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/8 "2020-11-03T23:47:10Z")

</div>

thank you very much, Ill try it and let you know !! 🤩  
I see so .. 🧐

---

<div class="post-metadata">

**Author:** ![juuuhuuu](https://avatars.discourse-cdn.com/v4/letter/j/958977/32.png) [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\
**Post date:** [November 5, 2020, 8:15am UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/9 "2020-11-05T08:15:02Z")

</div>

Hello,

i tried but the problem is, that the logstash is not going in to the if condition.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 3, 2020, 8:15am UTC](https://discuss.elastic.co/t/filter-with-select-in-logstash/253732/10 "2020-12-03T08:15:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.