I have 2 servers each has 4 cpus and each cpu has 4 cores and the servers have 16gb RAM. I receive every hour 2 8gb files and 2 3gb files. What are the best number of shards and pipelines workers and batch size and JVM heap size basically what should I configure to make the best index speed for these files.
What kind of disk/storage do you have?
Thank you for reply and a normal HDD
OK. Then I would recommend you review and follow these guidelines. Use as few shards as possible and consider using the rollover index API to target a specific shard size.
I will, but i also need help in setting logstash pipeline workers and batch size acfording to file sizes
I would recommend using the defaults as they are quite good.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.