I am using elastic search 2.3.0 hosted on AWS Cloud.
I have recently made code changes for using the bulk indexing option provided by elastic.
When i have tested this change on my PERF env , i was happy with the performance achieved.
But , when i pushed this code to PROD env , the performance was very poor compared to the performance observed in PERF environment.
- Both PERF and PROD environments have the same hardware configuration (m4.2xlarge EC2 Server)
- Data Volume in PERF is about 170 Million Docs Stored in a single index , whereas data volume in PROD is about 800 Million Docs Stored in a single index.
Rest all the configurations in elasticsearch.yml are same in both PERF and PROD environments
I am unable to figure out the poor performance. Please provide me with some pointers.