# Logstash KV filter for multiline message

**URL:** <https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950>\
**Category:** Logstash\
**Created:** [September 10, 2018, 11:51am UTC](https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950 "2018-09-10T11:51:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![eswarloges](https://avatars.discourse-cdn.com/v4/letter/e/8e7dd6/32.png) [@eswarloges](https://discuss.elastic.co/u/eswarloges)\
**Post date:** [September 10, 2018, 11:51am UTC](https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950/1 "2018-09-10T11:51:51Z")

</div>

Hi Team,

I am creating a KV filter for the following message

> 1535125123989 error targetResponse m=GET, u=/favicon.ico, [h=test.com](http://h=test.com), r=192.168.1.5:2383, s=404, name=Error, message=no match found for /favicon.ico, code=undefined, stack=Error: no match found for /favicon.ico  
> at Error (native)  
> at /apps/apigee/lib/node\_modules/edgemicro/node\_modules/microgateway-core/lib/config-proxy-middleware.js:114:23  
> at /apps/apigee/lib/node\_modules/edgemicro/node\_modules/microgateway-core/lib/gateway.js:40:11  
> at /apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:718:13  
> at iterate (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:262:13)  
> at async.forEachOfSeries.async.eachOfSeries (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:281:9)  
> at \_parallel (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:717:9)  
> at Object.async.series (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:739:9)  
> at Server.serverMiddleware (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/microgateway-core/lib/gateway.js:38:13)  
> at emitTwo (events.js:106:13)  
> at Server.emit (events.js:191:7)  
> at HTTPParser.parserOnIncoming [as onIncoming] (\_http\_server.js:546:12)  
> at HTTPParser.parserOnHeadersComplete (\_http\_common.js:99:23)

And the logstash conf is as follows

> input {  
> file {
> 
> ```
> path => "test.log"
> type => "logs"
> codec => multiline {
> pattern => "^%{NUMBER}"
> negate => true
> what => "previous"
> }
> start_position => "beginning"
> 
> ```
> 
> }  
> }  
> filter {  
> grok {  
> match =\> {"message" =\> "%{NUMBER:unixTime} %{LOGLEVEL:severity} %{DATA:eventType} %{GREEDYDATA:logMessage}"}  
> }
> 
> ```
> date {
> match => ["unixTime", "UNIX_MS"]
> target => "logTime"
> }
> 
> kv {
> source => "logMessage"
> trim_key => "<>\[\], "
> field_split => ","
> value_split => "="
> 
> ```
> 
> }
> 
> if "\_grokparsefailure" in [tags] {  
> mutate {  
> remove\_field =\> ["@version","path","type","host"]  
> remove\_tag =\> ["\_grokparsefailure"]  
> }  
> } else {
> 
> ```
> if [name] == "Error" {
> mutate {
> rename => { "message" => "errorMessage" }
> }
> }
> 
> mutate {
> rename => { "m" => "httpMethodName" }
> rename => { "u" => "urlPath" }
> rename => { "h" => "gatewaybackendHostPort" }
> rename => { "r" => "clientHostPort" }
> rename => { "i" => "requestId" }
> rename => { "d" => "duration" }
> rename => { "s" => "httpVerb" }
> gsub => ["requestId", "\r", ""]
> gsub => ["stack", "\r", ""]
> gsub => ["stack", "\n", ""]
> remove_field => ["@version","path","type","host"]
> }
> }
> 
> ```
> 
> }
> 
> output {  
> stdout {}  
> }

And I m getting the following as the output

> {  
> "logMessage" =\> "m=GET, u=/favicon.ico, [h=api.kbmg.com](http://h=api.kbmg.com), r=::ffff:192.168.94.5:2383, s=404, name=Error, message=no match found for /favicon.ico, code=undefined, stack=Error: no match found for /favicon.ico\r\n at Error (native)\r\n at /apps/apigee/lib/node\_modules/edgemicro/node\_modules/microgateway-core/lib/config-proxy-middleware.js:114:23\r\n at /apps/apigee/lib/node\_modules/edgemicro/node\_modules/microgateway-core/lib/gateway.js:40:11\r\n at /apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:718:13\r\n at iterate (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:262:13)\r\n at async.forEachOfSeries.async.eachOfSeries (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:281:9)\r\n at \_parallel (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:717:9)\r\n at Object.async.series (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/async/lib/async.js:739:9)\r\n at Server.serverMiddleware (/apps/apigee/lib/node\_modules/edgemicro/node\_modules/microgateway-core/lib/gateway.js:38:13)\r\n at emitTwo (events.js:106:13)\r\n at Server.emit (events.js:191:7)\r\n at HTTPParser.parserOnIncoming [as onIncoming] (\_http\_server.js:546:12)\r\n at HTTPParser.parserOnHeadersComplete (\_http\_common.js:99:23)\r",  
> "name" =\> "Error",  
> "stack" =\> "Error: no match found for /favicon.ico",  
> "@timestamp" =\> 2018-09-10T11:49:07.147Z,  
> "errorMessage" =\> "no match found for /favicon.ico",  
> "httpMethodName" =\> "GET",  
> "gatewaybackendHostPort" =\> "[test.com](http://test.com)",  
> "urlPath" =\> "/favicon.ico",  
> "logTime" =\> 2018-08-24T15:38:43.989Z,  
> "httpVerb" =\> "404",  
> "unixTime" =\> "1535125123989",  
> "severity" =\> "error",  
> "eventType" =\> "targetResponse",  
> "tags" =\> [  
> [0] "multiline"  
> ],  
> "code" =\> "undefined",  
> "clientHostPort" =\> "192.168.1.5:2383"  
> }

I am expecting the stack to have the complete stack trace. Anything wrong with the KV filter or do i have to add something in the configuration file. Kindly help. Thanks

---

<div class="post-metadata">

**Author:** ![eswarloges](https://avatars.discourse-cdn.com/v4/letter/e/8e7dd6/32.png) [@eswarloges](https://discuss.elastic.co/u/eswarloges)\
**Post date:** [September 11, 2018, 10:52am UTC](https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950/2 "2018-09-11T10:52:32Z")

</div>

The issue has been fixed by adding a mutate filter to replace \r and \n in the logmessage.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2018, 10:52am UTC](https://discuss.elastic.co/t/logstash-kv-filter-for-multiline-message/147950/3 "2018-10-09T10:52:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.