Sometimes messages are split

Hi

I'm new with ELK stack. And I'm trying to get logs from ESXi hosts.
Rsyslog (version 7.4.4-1) gets the messages on port 1514,
it uses such a template:

//template(name="bladetemplate" type="list") {
property(name="pri")
constant(value=" ")property(name="timereported" dateFormat="rfc3339")
constant(value=" ")property(name="hostname")
constant(value=" ")property(name="syslogfacility-text")
constant(value=" ")property(name="syslogseverity-text")
property(name="msg")
}//

Then it forwards them to local port 10515 using the settings:

//if $fromhost contains "blade" then {
. action (type="omfwd"
queue.type="LinkedList"
action.resumeRetryCount="-1"
queue.size="10000"
queue.saveonshutdown="on"
Target="localhost" Port="10515" Protocol="tcp" Template="bladetemplate")
}//

Logstash (version 1:5.6.16-1) catches them on port 10515:
//input {
tcp {
port => 10515
codec => multiline
{
pattern => "-->"
what => "previous"
auto_flush_interval => 3
}
type => "nsk.esxi"
}
}//

filter:
//filter
{
if [type] == "nsk.esxi"
{
grok
{
match => {
"message" => "%{POSINT:PRI} %{TIMESTAMP_ISO8601:@timestamp} %{SYSLOGHOST:sysloghost} %{DATA:facility} %{DATA:severity} %{DATA:syslog_level} %{DATA:programname}[%{DATA:procid}] (?(?<message_system_info>([%{DATA:message_service} %{DATA:sub_id} %{DATA:message_opID}])) [%{DATA:message_service_info}]\ (?(%{GREEDYDATA})))"
}
overwrite => [ "message" ]
}
}
}//

and then forwards to local port of elasticsearch (version 5.6.16).

In Kibana I can see that some messages are split:
correct message - 167 2020-02-03T09:43:26.497Z blade202 local4 debug verbose vpxa[34262B70] [Originator@6876 sub=vpxaMoService opID=2fcfa7f9-df] Adding querySpec. Had=9, has=9
split message - 167 2020-02-03T09:43:24.750Z blade202 local4 debug verbose vpxa[34262B70] [Origin

Can somebody help me in looking for the proper way of troubleshooting?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.