# Grok filter Mappings ignored by logstash

**URL:** <https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295>\
**Category:** Logstash\
**Created:** [July 31, 2018, 7:21am UTC](https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295 "2018-07-31T07:21:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Md.Khan](https://avatars.discourse-cdn.com/v4/letter/m/77aa72/32.png) [@Md.Khan](https://discuss.elastic.co/u/Md.Khan)\
**Post date:** [July 31, 2018, 7:21am UTC](https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295/1 "2018-07-31T07:21:00Z")

</div>

we have created below grok filter for mapping the log fields, but still logstash is using default mapping when pipeline initiated.  
//  
filter {  
grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} +%{LOGLEVEL:severity} +%{JAVACLASS:class}: +allowed=%{WORD:allowed} +ugi=%{USERNAME:ugi} +(auth:%{WORD:auth}) +via +%{USERNAME:ugi2} +(auth:%{WORD:auth2}) +ip=/%{IP:ip} +cmd=%{WORD:cmd} +src=%{PATH:src} +dst=%{USERNAME:dst} +perm=%{USERNAME:perm} +proto=%{USERNAME:proto} +callerContext=%{GREEDYDATA:callerContext}" }  
add\_field =\> {  
"type" =\> "pattern1"  
}  
}  
grok {  
match =\> { "message" =\>"%{TIMESTAMP\_ISO8601:timestamp} +%{LOGLEVEL:severity} +%{JAVACLASS:class}: +allowed=%{WORD:allowed} +ugi=%{USERNAME:ugi} +(auth:%{WORD:auth}) +ip=/%{IP:ip} +cmd=%{WORD:cmd} +src=%{PATH:src} +dst=%{USERNAME:dst} +perm=%{USERNAME:perm} +proto=%{USERNAME:proto}"}  
add\_field =\> {  
"type" =\> "pattern2"  
}  
}  
}  
//

mapping in results are generated like below (but it should follow the mapping provided in filter ?) :=\>  
"@version" =\> "1",  
"tags" =\> [  
[0] "\_grokparsefailure"  
],  
"path" =\> "/var/log/hadoop/hdfs/hdfs-audit.log.2018-07-26",  
"message" =\> "2018-07-26 23:59:57,497 INFO FSNamesystem.audit: allowed=true\tugi=yarn (auth:SIMPLE)\tip=/10.155.140.194\tcmd=listStatus\tsrc=/ats/active\tdst=null\tperm=null\tproto=rpc\tcallerContext=yarn\_ats\_server\_v1\_5",  
"@timestamp" =\> 2018-07-31T07:01:56.905Z,  
"host" =\> "CTSC00557182401"

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 31, 2018, 11:38am UTC](https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295/2 "2018-07-31T11:38:56Z")

</div>

Your grok patterns appear to match spaces using ' +', but your event contains tabs '\t'.

---

<div class="post-metadata">

**Author:** ![Md.Khan](https://avatars.discourse-cdn.com/v4/letter/m/77aa72/32.png) [@Md.Khan](https://discuss.elastic.co/u/Md.Khan)\
**Post date:** [July 31, 2018, 12:48pm UTC](https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295/3 "2018-07-31T12:48:24Z")

</div>

Thanks for the reply badger , that's added because "message" field was interpreting multi space with "\t" to mitigate I used multiple grok filters (one more with "+") but still it's prompting default.

like :  
filter {  
grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} +%{LOGLEVEL:severity} +%{JAVACLASS:class}: +allowed=%{WORD:allowed} +ugi=%{USERNAME:ugi} +(auth:%{WORD:auth}) +via +%{USERNAME:ugi2} +(auth:%{WORD:auth2}) +ip=/%{IP:ip} +cmd=%{WORD:cmd} +src=%{PATH:src} +dst=%{USERNAME:dst} +perm=%{USERNAME:perm} +proto=%{USERNAME:proto} +callerContext=%{GREEDYDATA:callerContext}"}  
}  
grok {  
# tab with callerContext with via  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} +%{LOGLEVEL:severity} +%{JAVACLASS:class}: +allowed=%{WORD:allowed}\tugi=%{USERNAME:ugi} +(auth:%{WORD:auth}) +via +%{USERNAME:ugi2} +(auth:%{WORD:auth2})\tip=/%{IP:ip}\tcmd=%{WORD:cmd}\tsrc=%{PATH:src}\tdst=%{USERNAME:dst}\tperm=%{USERNAME:perm}\tproto=%{USERNAME:proto}\tcallerContext=%{GREEDYDATA:callerContext}"}  
}

}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 31, 2018, 1:31pm UTC](https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295/4 "2018-07-31T13:31:29Z")

</div>

Please show both the message and the grok pattern indented by four spaces, so that escapes are shown.

Like this:

```
"message" => "2018-07-26 23:59:57,497 INFO FSNamesystem.audit: allowed=true\tugi=yarn (auth:SIMPLE)\tip=/10.155.140.194\tcmd=listStatus\tsrc=/ats/active\tdst=null\tperm=null\tproto=rpc\tcallerContext=yarn_ats_server_v1_5",

```

Not like this

"message" =\> "2018-07-26 23:59:57,497 INFO FSNamesystem.audit: allowed=true\tugi=yarn (auth:SIMPLE)\tip=/10.155.140.194\tcmd=listStatus\tsrc=/ats/active\tdst=null\tperm=null\tproto=rpc\tcallerContext=yarn\_ats\_server\_v1\_5",

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 28, 2018, 1:31pm UTC](https://discuss.elastic.co/t/grok-filter-mappings-ignored-by-logstash/142295/5 "2018-08-28T13:31:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.