Cloud foundry wihh Logstash

Elastic Stack Logstash
1

in logstash for cloud foundry i used tcp and udp input plugin as below
input {
tcp {
port => 5000
type => syslog
}
udp {
port => 5000
type => syslog
}
}
but i'm looking data resilient(persistent queue) feature of lostash.

as per logstash document tcp and upd etc don't not have a mechanism to acknowledge receipt to the sender so can not be protected from data loss(not able to use persistent queue feature). and in cloud foundry i used tcp and udp for input
what is solution if i want to use persistent queue or any way to protect from data loss in above case ?

2

Back-pressure is an integral part of the Beats (formerly "lumberjack") protocol, so if it's possible to install Filebeat or another appropriate Beat on your edge nodes, that's probably going to be your best bet.

The http input also supports back-pressure simply because it is customary for an http client sending data to wait for a response.

However, it is all about trade-offs. In the above scenarios, back-pressure means that when Logstash is overburdened, data will spool up on your edge nodes; if those are volatile or ephemeral, the data that has not yet shipped from them when they disappear would be lost.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.