I'm opening this topic to get some advice from your side since I'm designing a ELK platform having some constraints.
In particular, we will have to stick using a single machine with 32 or 64 GB of RAM.
The architecture will be based on Docker containers.
Here are some questions I would like to ask:
- On a single machine, there is some advantage configuring multiple Elasticsearch nodes? From my understanding reading forum topics, the answer is no;
- On our testing machine, which has 16 GB of RAM, using a single Elasticsearch node we are starting to get GatewayTimeouts accessing dashboards from Kibana; we are currently working with tens of millions of records per day. Is this issue mostly related to RAM constraints?
- Is it considered safe to run Logstash on the same machine where Elasticsearch is running? Our approach - since we are going to use Docker containers - would be to restrict the amount of RAM and CPU available to Logstash.