I'm very new to elasticsearch and running as 3 node cluster using docker. I have a large amount of data around 250 million records which is currently stored between xml files and a mysql database and found that elasticsearch would be create to use for my needs.
So i began following some video tutorials online, and manage to create/index 1,000,0000 fine, this was relatively quick too, using the localhost:9200 end point. I played around with a few queries and everything all ok. So i began to index (not sure if right term) a few more million records with the plan to do some more testing. However once i had around 1.6million documents indexed it seem to grind to a halt. Indexing a single document was/is taking around 7 seconds to do, where as before i could send 100 a second quiet easy. But the reading/queries seems fine.
Im using raid5 with 3x8TB drives, binding the volume data to the raid drive instead of the default /var/lib/docker folder.
I have done some reading but not 100% sure what i should be looking for.