Remove extra lines from output message

Elastic Stack Logstash
1

Hi , I am using multiline pattern to filter my output message from Exec plugin , Now I am able to get the lines separately according to my required pattern ^D122, But i am getting some extra characters in the message like below.

Output :
'***************************************************************
<D1220001022A SvrTblCleanup Shutdown Manual 0 1 Server Tables Cleanup

223 rows returned.

srvrmgr> list server show SBLSRVR_NAME,SBLSRVR_STATE

SBLSRVR_NAME SBLSRVR_STATE
------------ ------------- />

My Logstash config file looks like this :

'# Sample Logstash configuration for creating a simple

Beats -> Logstash -> Elasticsearch pipeline.

input {

beats {
port => 5044
tags => ["srvr_logs"]
}

exec {
command => "E:\ELK\logstash\scripts\DEV_Srvrmgr.bat"
interval => 120
#type => "string"
tags => ["srvrmgr"]
codec => multiline {
pattern => "^D122"
negate => true
what => "previous"

    }


  }

}

filter {

mutate {
remove_field => [ "host" ]
gsub => ["message", "\n", ""]
}
if "srvr_logs" in [tags]

{
grok {
match => {"message" => "%{WORD:EventType}%{SPACE}%{WORD:EventSubType}%{SPACE}%{INT:Severity}%{SPACE}%{WORD:SARMID}%{NOTSPACE}%{SPACE}%{PROG:EventDate}%{SPACE}%{TIME:EventTime}%{SPACE}%{GREEDYDATA:LogMessage}"}
}

}

else {
grok {
match => {
"message" => [
#Most specific grok:
"%{WORD:ServerName}%{SPACE}%{WORD:Comp_Alias}%{SPACE}%{WORD:CompStatus}%{SPACE}%{WORD:CompStartMode}%{SPACE}%{WORD:RunningTasks}%{SPACE}%{WORD:MaxTasks}%{SPACE}%{GREEDYDATA:CompName}",
#Less specific:
"%{WORD:SBLSRVR_NAME}%{SPACE}%{WORD:SBLSRVR_STATE}"
]
}
}
}
}

output {

if "srvr_logs" in [tags] {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "srvrlog-%{+YYYY.MM.dd}"
}

}

else
{
elasticsearch {
hosts => ["http://localhost:9200"]
index => "srvrmgr-%{+YYYY.MM.dd}"
}
}

}
'

Here GREEDY DATA Gives me extra lines as well which is not at all required , Checked forums,stackoverflow and all . Not able to get proper solution .Please let me know how can we omit extra lines ? In other words , how can say logstash to process message till new line \n ?

2

Hi ,

This got fixed with below ruby code.

     ruby 	{
        code => '
            event.set("message", event.get("message").split("\n"))
        '
			}
    split 	{
        field => "message"
			}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.