# Completely lost

**URL:** <https://discuss.elastic.co/t/completely-lost/37632>\
**Category:** Logstash\
**Created:** [December 19, 2015, 9:56pm UTC](https://discuss.elastic.co/t/completely-lost/37632 "2015-12-19T21:56:50Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![new2logstash](https://avatars.discourse-cdn.com/v4/letter/n/d78d45/32.png) [@new2logstash](https://discuss.elastic.co/u/new2logstash)\
**Post date:** [December 19, 2015, 9:56pm UTC](https://discuss.elastic.co/t/completely-lost/37632/1 "2015-12-19T21:56:50Z")

</div>

Hello all. I have followed the excellent Digital Ocean filebeat/logstash/ES/Kibana ([How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS 7 | DigitalOcean](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7)). I am getting messages from one of my files into elasticsearch/kibana.

However, my input lines look like the following:

> Dec 19 17:54:49 [myserver.example.com](http://myserver.example.com) kernel: [12954239.200000] DROP IN=eth1 OUT= MAC=30:46:9a:15:66:b0:e8:b7:48:0c:8a:da:08:00 SRC=178.175.38.24 DST=10.1.1.1 LEN=60 TOS=0x00 PREC=0x00 TTL=47 ID=12841 DF PROTO=TCP SPT=50806 DPT=20012 SEQ=1057317385 ACK=0 WINDOW=5808 R

My current filter is

```
 filter {
   if [type] == "syslog" {
     grok {
       match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
       add_field => ["received_at", "%{@timestamp}"]
       add_field => ["received_from", "%{host}"]
     }
     syslog_pri { }
     date {
       match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
     }
   }
 }

```

I'm trying to parse out the "SRC" ip, "DST" ip into separate fields. I would also like to use the geoip plugin.

I read through a bunch of documentation and I am completely lost at how I might accomplish this task.

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [December 20, 2015, 1:03am UTC](https://discuss.elastic.co/t/completely-lost/37632/2 "2015-12-20T01:03:15Z")

</div>

My grok lines:  
`match =\> ["message", "%{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{WORD:kernel}: [%{DATA:unixtime}] IN=%{DATA:in\_int} OUT=%{DATA:out\_int} MAC=%{DATA:junk} SRC=%{IP:src\_ip} DST=%{IP:dst\_ip} LEN=%{NUMBER:len} TOS=%{DATA:junk} PREC=%{DATA:junk} TTL=%{NUMBER:junk} ID=%{NUMBER:junk} DF PROTO=%{WORD:proto} SPT=%{NUMBER:src\_port} DPT=%{NUMBER:dst\_port} WINDOW=%{NUMBER:junk} RES=%{DATA:junk} %{DATA:flags} URGP=%{NUMBER:junk}" ]

match =\> ["message", "%{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{WORD:kernel}: [%{DATA:unixtime}] IN=%{DATA:in\_int} OUT=%{DATA:out\_int} MAC=%{DATA:junk} SRC=%{IP:src\_ip} DST=%{IP:dst\_ip} LEN=%{NUMBER:len} TOS=%{DATA:junk} PREC=%{DATA:junk} TTL=%{NUMBER:junk} ID=%{NUMBER:junk} PROTO=%{WORD:proto} SPT=%{NUMBER:src\_port} DPT=%{NUMBER:dst\_port} WINDOW=%{NUMBER:junk} RES=%{DATA:junk} %{DATA:flags} URGP=%{NUMBER:junk}" ]

match =\> ["message", "%{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{WORD:kernel}: [%{DATA:unixtime}] IN=%{DATA:in\_int} OUT=%{DATA:out\_int} MAC=%{DATA:junk} SRC=%{IP:src\_ip} DST=%{IP:dst\_ip} LEN=%{POSINT:fullen} TOS=%{DATA:junk} PREC=%{DATA:junk} TTL=%{POSINT:junk} ID=%{NUMBER:junk} DF PROTO=%{WORD:proto} SPT=%{POSINT:src\_port} DPT=%{POSINT:dst\_port} LEN=%{POSINT:len}" ]

match =\> ["message", "%{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{WORD:kernel}: [%{DATA:unixtime}] IN=%{DATA:in\_int} OUT=%{DATA:out\_int} MAC=%{DATA:junk} SRC=%{IP:src\_ip} DST=%{IP:dst\_ip} LEN=%{POSINT:fullen} TOS=%{DATA:junk} PREC=%{DATA:junk} TTL=%{POSINT:junk} ID=%{NUMBER:junk} PROTO=%{WORD:proto} SPT=%{POSINT:src\_port} DPT=%{POSINT:dst\_port} LEN=%{POSINT:len} MARK=%{DATA:junk}" ]

match =\> ["message", "%{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{WORD:kernel}: [%{DATA:unixtime}] IN=%{DATA:in\_int} OUT=%{DATA:out\_int} MAC=%{DATA:junk} SRC=%{IP:src\_ip} DST=%{IP:dst\_ip} LEN=%{POSINT:fullen} TOS=%{DATA:junk} PREC=%{DATA:junk} TTL=%{POSINT:junk} ID=%{NUMBER:junk} PROTO=%{WORD:proto} SPT=%{POSINT:src\_port} DPT=%{POSINT:dst\_port} LEN=%{POSINT:len}" ]`

Season to taste.

---

<div class="post-metadata">

**Author:** ![new2logstash](https://avatars.discourse-cdn.com/v4/letter/n/d78d45/32.png) [@new2logstash](https://discuss.elastic.co/u/new2logstash)\
**Post date:** [December 20, 2015, 4:56pm UTC](https://discuss.elastic.co/t/completely-lost/37632/3 "2015-12-20T16:56:32Z")

</div>

Thank you - that helped tremendously!

One followup question - mixed in with these events, I occasionally get ssh login failures. For example:

> Dec 20 13:22:21 [router01.example.com](http://router01.example.com) dropbear[31382]: bad password attempt for 'root' from 182.75.33.126:61332

I am able to parse these with:

> %{SYSLOGTIMESTAMP:date} %{IPORHOST:device} %{WORD:kernel}[%{DATA:unix\_pid}]: bad password attempt for 'root' from %{IP:src\_ip}:%{NUMBER:src\_port}

I would love to graph these separately from the other type of log entries. Is there a way to "mutate" these to another "type" so I can process these independently?

---

<div class="post-metadata">

**Author:** ![new2logstash](https://avatars.discourse-cdn.com/v4/letter/n/d78d45/32.png) [@new2logstash](https://discuss.elastic.co/u/new2logstash)\
**Post date:** [December 20, 2015, 7:01pm UTC](https://discuss.elastic.co/t/completely-lost/37632/4 "2015-12-20T19:01:43Z")

</div>

Nevermind - I figured it out with some googling and trial and error. Thanks again!

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [December 21, 2015, 1:15am UTC](https://discuss.elastic.co/t/completely-lost/37632/5 "2015-12-21T01:15:02Z")

</div>

Ah good deal...busy day here..sorry about no response.

---

<div class="post-metadata">

**Author:** ![James\_Tang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_tang/32/8112_2.png) [@James\_Tang](https://discuss.elastic.co/u/James_Tang)\
**Post date:** [December 21, 2015, 2:38am UTC](https://discuss.elastic.co/t/completely-lost/37632/6 "2015-12-21T02:38:02Z")

</div>

If the issue is with grokking, I recommend you book marked this tool [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/)  
Otherwise the plugin documentation is your friend in this case.

[https://www.elastic.co/guide/en/logstash/current/input-plugins.html](https://www.elastic.co/guide/en/logstash/current/input-plugins.html)  
[https://www.elastic.co/guide/en/logstash/current/output-plugins.html](https://www.elastic.co/guide/en/logstash/current/output-plugins.html)  
[https://www.elastic.co/guide/en/logstash/current/filter-plugins.html](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html)

BTW, event type value once set in logstash cannot be changed within logstash. You have to do so externally.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/completely-lost/37632/7 "2017-07-06T05:17:44Z")

</div>


