Scaling out logstash

Elastic Stack Logstash
1

Looking for the easiest approach to scale ELK with Docker (host network mode)

Currently I have 2 servers and each runs a single elasticsearch, logstash and kibana container.

The elasticsearch containers on each server run in master mode and can find each other with unicast discovery:

    discovery.zen.minimum_master_nodes: 2
    discovery.zen.ping.unicast.hosts: server1, server2

Each logstash container is configured with it's local elasticsearch container running on same server.

    output {
        elasticsearch {
            hosts   => ["${HOST_IP}:9200"]
        }
    }

The Kibana container will also only talk to local elasticsearch but I'm assuming via discovery it'll find all others.
A user can jump onto Kibana on any server to search all logs etc...

ELASTICSEARCH_URL=http://${HOST_IP}:9200

As I add more servers they'll run using the same setup. 1 master elasticsearch container and 1 logstash container and 1 Kibana

Does this setup make sense?

2

It's certainly not crazy, but

  • ES, Logstash, and Kibana all have different performance characteristics and different ways and needs of scaling, and
  • ES and Logstash will compete for resources.

Having your configuration assume that all three services run on the same machine could paint you into a corner if you ever decide you want to break things up. It probably won't be that hard to untangle but I'd prefer not making such assumptions in the first place.