I am new to ELK stack and I am currently testing on the performance of the Elasticsearch with 3 different configurations as follows:
- One source file: 1GB
- One source file: 3GB
- 3 source files: 1GB+1GB+1GB
It took me 3 to 4 hours to have the 3GB file ingested into Elasticsearch. It took me more than a day to finish 3 different 1GB files to be ingested. The speed was getting slower as the size of the index increases.
I have tried to increase the JVM heap size and it's getting slightly better, shortening the time by about one hour. All other settings are configured in the default settings.
I would like to know the additional ways on how to improve the ingestion speed on this? Thanks.