Actually i am indexing 5 crore data into my vm ware elastic server from SQL server using log-stash ,creating the 9 index with 1 primary shard and 1 replica shard and running the Multipipeline in pipeline.yml file and each pipeline has 3 worker and other setting is default.
my problem which is facing that indexing freeze any time whether data size is 1 Lac or 75 Lac. i tried a lot even i used the single multipipeline for single indexed but still the same issue occured.
I had tried the logstash heap memory more than 1GB(default) to 4 GB(after changing) but still the same issue.
i also change the heap memory of Elasticsearh 1GB(default) to 6 GB but still getting the same issue.
Please suggest the optimal resolution as soon as possible.
my pipeline is :
-pipeline id :abc