In ELK i have found that during implementation using syslog as a shipper for logs to logstash server cause couple of issues like
- During regression mulltiple logs shipped from logs file to logstash server at same time.(in JAVA application) cauing cascading issue when multiline plugin defined in configuration of logstash server.
- If you remove multiline plugin, log message broker down to small chunk and hence a single instance on log block(JSON) didnt get captured.
As syslog shipped logs in logstash server...is not persistance with sequence in case of high volumn.
As per solution : We have get rid of syslog and used LogstashTcpSocketAppender to improve performace.
refer link for more info on implementation.
As this give substential performace...as logs are persitant to event order that happen.
also its reaches to kibana page in real time when compare to syslog.
as it cut 1 layer down. as show below:
Application --> application log file --> using syslog --> logstash server --> elastic search --> kibana
Application --> using tcp appender --> logstash server --> elastic search --> kibana
try this at your end and share input on same.