I was getting some JVM Heap errors and I was looking up on how to set all of that properly. I saw in ES documentation to set it at 50% of available memory (I have 8G total) up to 32G. But which one do I set - LS or ES or both? Do they need to be identical or 50% of 50% in each?
Logstash shouldn't need as much heap, I have 2G, 8 pipelines (3 active) with about 500 events per second. It probably depends what you do in the pipelines.
Elasticsearch and Logstash have different memory requirements.
You're correct that the recommended JVM Heap allocation for Elasticsearch is 50% of available memory (up to 32GB) and it will use the remaining memory for caching, which improves query performance.
Logstash uses memory differently and usually requires less than Elasticsearch. Logstash is essentially a data processing queue. It holds small batches of events in memory before processing them and shipping them to Elasticsearch. Many installations use the default 1GB. You may need to increase this if you plan to hold more events in the queue at once, or if you plan to use a large translation dictionary.
See also this discussion: Logstash OOM - understanding heap sizing
thank you, I ended up - for now - setting both ES and LS with the same amount of memory and it resolved the heap errors.