# Logstash performance (choice) **URL:** \ **Category:** Logstash\ **Created:** [June 3, 2019, 8:02am UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907 "2019-06-03T08:02:35Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![jof300](https://avatars.discourse-cdn.com/v4/letter/j/3da27b/32.png) [@jof300](https://discuss.elastic.co/u/jof300)\ **Post date:** [June 3, 2019, 8:02am UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907/1 "2019-06-03T08:02:35Z")
Hello everybody, We are implementing an ELK stack to manage our logs. We have all kind of logs (Firewall, syslog, application logs, hardware devices, etc...). We are running logstash on Docker swarm. The estimation of the throughput is around 300Go per day. I was wondering how to size logstash when we'll be in production. I see 2 approaches : - one big logstash with X Go ram . But how many ? - using one logstash container per type of logs - 1 container to parse and index firewall logs - 1 container to parse and index application logs - ... We are using logstash in two different manner (shipper and indexer). Any advice ? Regards ---
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [June 3, 2019, 12:10pm UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907/2 "2019-06-03T12:10:34Z")
Logstash does not cluster by default and you will have single point of failure. You could think about Kawka in front of your Logstashes. If you want to segment your configuration per log source, you can use pipeline to pipeline pattern that will simplify your configs (eg. as distributor architecture pattern: [https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html) ) When it comes to the RAM, have in mind that you have the oop's pointer compression problem: [https://www.baeldung.com/jvm-compressed-oops](https://www.baeldung.com/jvm-compressed-oops) and you should not give your JVM more than 32 GB of ram (how your machine determines the sweet spot for this you must check, but 31GB of ram per Logstash instance is the maximum you would like to give). ---
**Author:** ![jof300](https://avatars.discourse-cdn.com/v4/letter/j/3da27b/32.png) [@jof300](https://discuss.elastic.co/u/jof300)\ **Post date:** [June 3, 2019, 1:36pm UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907/3 "2019-06-03T13:36:31Z")
I don't have complex workflows so I don't need pipeline to pipeline. More, this is a beta feature. The idea behind multiple logstash container is having multiple container that listen on a port (ex : 5000, 5001, 5002) and the VM hostname is the same (ex : logstash.local). All containers output would send to the same output The final flow would be something like : logstash (role shipper) OR beats =\> Kakfa =\> logstash (role indexer) =\> ES Firewall logs go to logstash.local:5000 (1st container) application logs go to logstash.local:5001 (2nd container) etc... I didn't know about the 31GB Ram. Thx for that Regards ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [June 3, 2019, 2:12pm UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907/4 "2019-06-03T14:12:23Z")
> [@jof300](#): > > I didn't know about the 31GB Ram. Thx for that Just on this point. I have a system with 96 gig RAM. I didn't know about this limit and assign 50gig ram to JVM and pretty much everyday system was hanging. I spend quite a time to debug java dump and after searching this forum found this limit. Move JVM back to 25gig and my problem disappear. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 1, 2019, 2:12pm UTC](https://discuss.elastic.co/t/logstash-performance-choice/183907/5 "2019-07-01T14:12:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.