# Filter Logs from Firewall

**URL:** <https://discuss.elastic.co/t/filter-logs-from-firewall/172494>\
**Category:** Logstash\
**Created:** [March 15, 2019, 9:15am UTC](https://discuss.elastic.co/t/filter-logs-from-firewall/172494 "2019-03-15T09:15:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![david-vazquez](https://avatars.discourse-cdn.com/v4/letter/d/ac8455/32.png) [@david-vazquez](https://discuss.elastic.co/u/david-vazquez)\
**Post date:** [March 15, 2019, 9:15am UTC](https://discuss.elastic.co/t/filter-logs-from-firewall/172494/1 "2019-03-15T09:15:03Z")

</div>

Hello everybody!

Im new using ELK, so I have some doubts in some stuff I´m trying to do.  
I´m sending logs from my Firewall to a Rsyslog server. I was sending logs throught Filebeat,but when I´m visualizing data in Kibana, I only see one field called "system.syslog.message with all the i,formation, due to its unstructured.  
The thing is, I need to get fields from that message to make better visualizations in Kibana. Logs are different each one, so I need to extract the fields that Im interested in each case.  
For example, log could be as follow:  
Mar 14 18:01:38 192.168.1.6 date=2019-03-14 time=18:01:38 devname=ELK devid=1234 logid=001 type=event subtype=vpn level=information vd=root logdesc="SSL VPN tunnel down" action="tunnel-down" tunneltype="ssl-web" tunnelid=1234 remip=10.10.10.11 user="david-beckam" group="Football" dst\_host="N/A" reason="N/A" duration=218 sentbyte=0 rcvdbyte=0 msg="SSL tunnel shutdown"

So, in that message I need to extract some fields to prepare a Dashboard to monitoring...  
Fields like, action,logdesc,remip,user,group,duration, time and date. With that fields, I could prepare Dashboard and check if a user was connected, how much time it was, and so so.

I think the way to do it is using Logstash as filter, but I don´t know exactly what to do and how.  
Should I use grok or dissect to do it? Someone could give me like an example to do it right?  
Could I send to Elasticsearch like different kinds of logs? I mean, from the same log file, I need to extract different fields depending of the log line struct because each line of log file can be different.

Thank you very much to all of you!

Best Regards 🙂

---

<div class="post-metadata">

**Author:** ![david-vazquez](https://avatars.discourse-cdn.com/v4/letter/d/ac8455/32.png) [@david-vazquez](https://discuss.elastic.co/u/david-vazquez)\
**Post date:** [March 15, 2019, 10:52am UTC](https://discuss.elastic.co/t/filter-logs-from-firewall/172494/2 "2019-03-15T10:52:38Z")

</div>

Im trying the following code for the same example I said before, but it doesn´t work...

input {  
file {  
path =\>"/var/log/messages"  
start\_position =\> "beginning"  
}  
}

filter {

dissect {  
mapping =\> {  
"message" =\> "%{month} %{+day} %{+time} %{iprouter} date=%{+fecha} time=%{hora} devname=%{devname} devid=%{devid} logid=%{logid} type=%{type} subtype=%{event} subtype=%{subtype} level=%{level} vd=%{vd} logdesc=%{logdesc} action=%{action} tunneltype=%{tunneltype} tunnelid=%{tunnelid} remip=%{remiip} user=%{user} group=%{group} dst\_host=%{dst\_host} reason=%{reason} duration=%{duration} sentbyte=%{sentbyte} rcvdbyte=%{receivedbyte} msg=%{mensaje}"}

}  
}  
output {  
elasticsearch { hosts =\> ["ip:9200"]}  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 15, 2019, 12:45pm UTC](https://discuss.elastic.co/t/filter-logs-from-firewall/172494/3 "2019-03-15T12:45:38Z")

</div>

You can use dissect to parse the first two elements, and a kv filter to parse the rest

```
dissect { mapping => { "message" => "%{[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} %{ip} %{[@metadata][restOfLine]}" } }
kv { source => "[@metadata][restOfLine]" }
date { match => ["[@metadata][ts]", "MMM dd HH:mm:ss" ] }
```

---

<div class="post-metadata">

**Author:** ![david-vazquez](https://avatars.discourse-cdn.com/v4/letter/d/ac8455/32.png) [@david-vazquez](https://discuss.elastic.co/u/david-vazquez)\
**Post date:** [March 25, 2019, 3:50pm UTC](https://discuss.elastic.co/t/filter-logs-from-firewall/172494/4 "2019-03-25T15:50:16Z")

</div>

Sorry for delay.

Im gonna try 🙂 Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 22, 2019, 3:50pm UTC](https://discuss.elastic.co/t/filter-logs-from-firewall/172494/5 "2019-04-22T15:50:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.