# Grok for this log

**URL:** <https://discuss.elastic.co/t/grok-for-this-log/45862>\
**Category:** Logstash\
**Created:** [March 31, 2016, 2:12am UTC](https://discuss.elastic.co/t/grok-for-this-log/45862 "2016-03-31T02:12:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Farouk\_Samu](https://avatars.discourse-cdn.com/v4/letter/f/9e8a1a/32.png) [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\
**Post date:** [March 31, 2016, 2:12am UTC](https://discuss.elastic.co/t/grok-for-this-log/45862/1 "2016-03-31T02:12:06Z")

</div>

I want to create a grok for the log below.

"2016-03-30 19:14:06-0500 [HonsshServerTransport,13657,195.154.43.29] [PLUG IN][EXAMPLE] - {'honey\_port': '22', 'sensor\_name': 'node3', 'session': {'auths': [{'username': 'admin', 'date\_time': '20160330\_191406\_202662', 'spoofed': False, 'password': 'default', 'succe ss': False}], 'country': 'France', 'start\_time': '20160330\_191402\_828898', 'log\_location': 'ses sions/node3/195.154.43.29/', 'session\_id': 'defe2632f71c405f827e693c8da6577b', 'peer\_port': '56 043', 'channels': [], 'version': 'SSH-2.0-JSCH-0.1.51', 'end\_time': '20160330\_191406\_388121', ' peer\_ip': '195.154.43.29'}, 'honey\_ip': '192.168.1.8'}"

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 31, 2016, 4:50am UTC](https://discuss.elastic.co/t/grok-for-this-log/45862/2 "2016-03-31T04:50:42Z")

</div>

> [@Farouk\_Samu](#):
>
> 2016-03-30 19:14:06-0500 [HonsshServerTransport,13657,195.154.43.29] [PLUG IN][EXAMPLE] -

The easiest way is probably to parse the part above using grok and capture the rest of the message in a single field, as the rest look similar to JSON. It does however not seem to be strictly valid JSON, so you may need to run a [mutate gsub](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-gsub) filter on it and potentially replace `'` with `"` and `False` with `false`. You should then be able to parse that field using the [json filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html).

---

<div class="post-metadata">

**Author:** ![Farouk\_Samu](https://avatars.discourse-cdn.com/v4/letter/f/9e8a1a/32.png) [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\
**Post date:** [March 31, 2016, 1:06pm UTC](https://discuss.elastic.co/t/grok-for-this-log/45862/3 "2016-03-31T13:06:56Z")

</div>

Thanks Christian,

I added this lines in my filter and the logstash configuration was ok but yet I got this error -\> Invalid argument - Invalid file

mutate{  
gsub =\> ["message", "'", '"',  
"message", "False", "false"]  
}

json {  
source =\> "message"  
}

---

<div class="post-metadata">

**Author:** ![Farouk\_Samu](https://avatars.discourse-cdn.com/v4/letter/f/9e8a1a/32.png) [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\
**Post date:** [March 31, 2016, 1:22pm UTC](https://discuss.elastic.co/t/grok-for-this-log/45862/4 "2016-03-31T13:22:15Z")

</div>

Below is my logstash.conf, the configuration runs ok but yet I got this error -\> Invalid argument - Invalid file

input {  
file {  
path =\> "/opt/honssh/logs/\*"  
type =\> "honsshdaily"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
}

}  
filter {  
if [type] == "honsshdaily" {  
grok {  
match =\> { "message" =\> "%{NUMBER}_%{NUMBER}_%{NUMBER},%{IPV4:attack\_src},%{USERNAME:username},%{WORD:password},%{NONNEGINT:success\_code}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]

```
}
geoip {
  source => "attack_src"
  target => "geoip"
  database => "/opt/logstash/GeoLiteCity.dat"
  add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ]
  add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
  convert => ["[geoip][coordinates]", "float"]
   gsub => ["message", "'", '"',
            "message", "False", "false"]
}

```

json {  
source =\> "message"  
}  
}  
}

output{

```
 elasticsearch {hosts => "127.0.0.1:9200"}
 stdout {codec => rubydebug }

```

}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 31, 2016, 2:09pm UTC](https://discuss.elastic.co/t/grok-for-this-log/45862/5 "2016-03-31T14:09:23Z")

</div>

Try separating out the fields before the start of the JSON like structure, the mutate the last field into JSON and apply the son filter, possibly something like this example:

```
filter {
    grok {
        match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} \[%{WORD:text},%{NUMBER:num},%{IPV4:attack_src}\]\s*\[%{GREEDYDATA:username}\]\[%{WORD:password}\] -\s*%{GREEDYDATA:rest}" }
    }

    mutate {
        gsub => ["rest", "'", '"']
        gsub => ["rest", "False", "false"]
    }

    json {
        source => "rest"
    }

    mutate {
        remove_field => ["rest", "message"]
    }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:04am UTC](https://discuss.elastic.co/t/grok-for-this-log/45862/6 "2017-07-06T05:04:29Z")

</div>

Closed on July 6, 2017, 5:04am UTC