# Filtering multiple logs

**URL:** <https://discuss.elastic.co/t/filtering-multiple-logs/68685>\
**Category:** Logstash\
**Created:** [December 12, 2016, 9:38am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685 "2016-12-12T09:38:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ilalorel](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@ilalorel](https://discuss.elastic.co/u/ilalorel)\
**Post date:** [December 12, 2016, 9:38am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685/1 "2016-12-12T09:38:55Z")

</div>

Hello i'm fairly new to logstash, so far i've managed to set up 2 servers, one is running the elastic stack and the other one is sending logs with beats to my elk stack. Now I have 2 different logs that i'm filtering both containing IPs, I managed to get one working with the geomap. Now my question is the following, Is it possible to setup a second geomap only containing the ips from my second log?

My logstash config files if those help:

input/output:

```
input {
  beats {
    port => 5044
    ssl => false
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    template => "/etc/logstash/templates/filebeat-index-template.json"
    template_name => "filebeat"
  }
}

```

geoip filter:

```
   filter{
           grok {
            match => { "message" => " %{IP:client}"}
        }
        geoip{
                source => "client"
                target => "geoip"
                database => "/etc/logstash/GeoLite2-City.mmdb"
                add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ]
                add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ]
        }
                 mutate {
         convert => ["[geoip][coordinates]", "float" ]

                }

```

cowrie filter:

```
filter {
if [fields][log_type] == "cowrie" {
   grok {
            match => { "message" => "%{GREEDYDATA:request}"}
        }

    json{
        source => "request"
        target => "parsedJson"
        remove_field=>["request"]
    }
    mutate {
        add_field => {
           "src_ip" => "%{[parsedJson][src_ip]}"
           "input" => "%{[parsedJson][input]}"
           "eventid" => "%{[parsedJson][eventid]}"
           "message" => "%{[parsedJson][message]}"
           "system" => "%{[parsedJson][system]}"
        }
    }
}
}

```

second log filter:

```
filter {
  if [fields][log_type] == "messages" {
    grok{
       match=>{"message"=>"%{SYSLOGTIMESTAMP:nf_timestamp}\s*%{HOSTNAME:nf_host}\s*kernel\S+\s*%{WORD:nf_action}?.*IN=%{USERNAME:nf_in_interface}?.*OUT=%{USERNAME:nf_out_interface}?.*MAC=%{COMMONMAC:nf_dst_mac}:%{COMMONMAC:nf_src_mac}?.*SRC=%{IPV4:nf_src_ip}.*DST=%{IPV4:nf_dst_ip}.*PROTO=%{WORD:nf_protocol}.?*SPT=%{INT:nf_src_port}?.*DPT=%{INT:nf_dst_port}?.*"}
       add_field=>{"eventName"=>"groke"}
    }
  }
}

```

Thanks in advance 🙂

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 12, 2016, 9:50am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685/2 "2016-12-12T09:50:47Z")

</div>

> Now my question is the following, Is it possible to setup a second geomap only containing the ips from my second log?

Yes, of course. Just add a second geoip filter and have it process another field.

---

<div class="post-metadata">

**Author:** ![ilalorel](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@ilalorel](https://discuss.elastic.co/u/ilalorel)\
**Post date:** [December 12, 2016, 10:03am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685/3 "2016-12-12T10:03:41Z")

</div>

Hello magnus

Thanks for the fast reply, I've tried what you said but I'm wondering doesn't the ips from both logs get stored in the same location (geoip)? How is it possible to visualise those 2 different maps in kibana? This is how my geo filters looks like at the moment:

```
   filter{
           grok {
            match => { "message" => " %{IP:client}"}
        }
        geoip{
                source => "client"
                target => "geoip"
                database => "/etc/logstash/GeoLite2-City.mmdb"
                add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ]
                add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ]
        }
                 mutate {
         convert => ["[geoip][coordinates]", "float" ]

                }

        geoip{
                source => "nf_src_ip"
                target => "geoip"
                database => "/etc/logstash/GeoLite2-City.mmdb"
                add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ]
                add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ]
        }
                 mutate {
         convert => ["[geoip][coordinates]", "float" ]

                }
}

```

Thanks in advance 🙂

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 12, 2016, 10:28am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685/4 "2016-12-12T10:28:44Z")

</div>

> How is it possible to visualise those 2 different maps in kibana?

Just use a query in Kibana; `fields.log_type:cowrie` to only show those events and `fields.log_type:messages` to show the rest.

You can also tell the geoip filter to store the coordinates in another field.

---

<div class="post-metadata">

**Author:** ![ilalorel](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@ilalorel](https://discuss.elastic.co/u/ilalorel)\
**Post date:** [December 12, 2016, 11:01am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685/5 "2016-12-12T11:01:21Z")

</div>

Thanks alot magnus, got it working 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 9, 2017, 11:01am UTC](https://discuss.elastic.co/t/filtering-multiple-logs/68685/6 "2017-01-09T11:01:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.