Data Dropped by Logstash

[kunal.1709 (Kunal)]

Hello Everyone,

While debugging the issue with TCP input with logstash, I have introduced Rsyslog as an intermediate between SIEM and Logstash as a temporary workaround.

Please refer to below link -

Unable to split message received on TCP port into individual records

A tcp input creates one event for each line of text. If your messages do not contain a newline it will not flush them until it is stopped.

SIEM -> Rsyslog (TCP) -> Logstash (TCP)

Even though this approach is working, there is significant amount of data being dropped between Rsyslog and Logstash.

For instance, If SIEM is sending 1,000,000 records, Rsyslog is capturing 997,000 on average but Logstash is capturing only 500,000 records

I have tried persistent queues as well
Please refer to below logstash.yml

path.data: /var/lib/logstash
queue.type: persisted
path.queue: /home/logstash
queue.max_bytes: 128mb
dead_letter_queue.enable: true
path.dead_letter_queue: /home/logstash/dead_queue
path.logs: /var/log/logstash
config.reload.automatic: true
config.reload.interval: 1s

Kindly let me know if you have any suggestions to try out

Thank You !!!