# Logstash Grok Filter Issue

**URL:** <https://discuss.elastic.co/t/logstash-grok-filter-issue/79051>\
**Category:** Logstash\
**Created:** [March 17, 2017, 3:42pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-issue/79051 "2017-03-17T15:42:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![peter.alfafara](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@peter.alfafara](https://discuss.elastic.co/u/peter.alfafara)\
**Post date:** [March 17, 2017, 3:42pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-issue/79051/1 "2017-03-17T15:42:52Z")

</div>

Hi,

Can anyone help us in creating the proper filter for our logs. See sample below as we see in Kibana:

\_@timestamp:March 17th 2017, 23:22:49.774 port:52,718 @version:1 host:127.0.0.1 message:{"client":{"name":"Client","address":"127.0.0.1","subscriptions":["ALL","client:Client"],"version":"0.28.4","timestamp":1489764153},"check":{"command":"/opt/sensu/embedded/bin/check-memory.rb","interval":60,"standalone":true,"handler":"event\_stream","subscribers":["ALL"],"name":"check\_memory","issued":1489764169,"executed":1489764169,"duration":0.109,"output":"MEM OK - free system memory: 7541 MB\n","status":0,"type":"standard","history":["0","0","0","0","0","0","0","0","0","0","0","0","0","0","0","0","0","0","0","0","0"],"total\_state\_change":0},"occurrences":1,"occurrences\_watermark":1,"action":"create","timestamp":1489764169,"id":"044e51bf-8a90-4df8-9e73-15a7a2e5c0c1","last\_ok":1489764169,"silenced":false,"silenced\_by":[]} tags:\_grokparsefailure \_id:AVrc3ThKoYxWJ5cxJt6c _type:log_

What we want to achieve is to remove all fields except for " message" and "timestamp" and modify the message field to look something like  
**_clientName: "Client" clientAddress: "127.0.0.1" checkCommand: "/opt/sensu/embedded/bin/check-memory.rb" checkOutput: "MEM OK - free system memory: 7541 MB\n"_**

Below is our current conf file

input {  
tcp {  
port =\> 5514  
}  
}

filter {  
grok {  
match =\> { "message" =\> "clientName %{WORD:[client][name]} clientAddress %{IP:[client][address]}  
checkThresholds %{DATA:[check][thresholds]} checkOutput %{DATA:[check][output]}}  
}

if "Sensu" in [message] {  
drop { }  
}

mutate {  
remove\_field =\> ["port", "@version", "host", [check][history], "tags", "\_id", "\_type", "\_index"]  
}  
}

output {  
elasticsearch {  
hosts =\> "[http://localhost:9200](http://localhost:9200)"  
}  
}

Hope you can help. Thank you

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [March 17, 2017, 11:57pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-issue/79051/2 "2017-03-17T23:57:02Z")

</div>

@peter.alfafara

Since the message looks like a json format message , if I were you , I would parse the field with [json filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html) first and then do some extra working with [mutate filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html) .

Like,

```
filter {
  json { source => "message" }

  mutate {
    add_filed => {"clientName" => "%{[client][name] ...}"}
    remove_filed =>["client"...]
  }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2017, 11:57pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-issue/79051/3 "2017-04-14T23:57:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.