Best index performance

I have 2 servers each has 4 cpus and each cpu has 4 cores and the servers have 16gb RAM. I receive every hour 2 8gb files and 2 3gb files. What are the best number of shards and pipelines workers and batch size and JVM heap size basically what should I configure to make the best index speed for these files.

What kind of disk/storage do you have?

Thank you for reply and a normal HDD

OK. Then I would recommend you review and follow these guidelines. Use as few shards as possible and consider using the rollover index API to target a specific shard size.

I will, but i also need help in setting logstash pipeline workers and batch size acfording to file sizes

I would recommend using the defaults as they are quite good.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.