Filebeat forwarding to Logstash running in Mesos

Elastic Stack Beats
1

In my current organization, we are planning to run our ELK stack on Mesos. For log-forwarding, we plan on having filebeat instance running on each of the hosts that run a backend service.

Is there a recommended way to configure file-beat to forward messages to logstash hosted as a mesos task? Since mesos master can move tasks around depending on resource availability logstash host/port may not be static afterall.

2

Sounds more like a Mesos question. Filebeat has no support for dynamic hosts and ports apart from what DNS provides.

3

filebeat does a DNS lookup every time it connects/reconnects. If host name remains, but IP:PORT changes, filebeat should still be able to reconnect to new IP, given filebeat can fetch updated DNS entry.

4

Thanks @magnusbaeck, @steffens. DNS was one of the ways we were planning to resolve this. Shall continue to research if there is any other way. Shall keep you posted on the approach that we go forward with.

5

Thanks. Really interested how this goes.

btw. you can try to reserve multiple logstash hosts being pre-configured (ls1, ls2, ls3) in beats and enable some form of load-balancing. Then you might be able to start/shutdown any number of instances as required and beats will start to balance batches of events among number of configured and available hosts only (ensure spooler_size = N * bulk_max_size). If logstash host is down (not needed to cope with current load), it will not participate in load-balancing until reconnect was successful. Having 2 instances active might smoothen the move of one host at a time. Just an idea, no idea how feasible this would be with mesos.

6

This topic was automatically closed after 21 days. New replies are no longer allowed.