Hi Gents,
Can we keep this in 1 thread please, thx.
@warkolm can you merge these 2 threads? or close the other one.
@Alain_St_Pierre thanks for answering on the other thread that help me understand a few things, it sounded like someone was executing the force merge on their own etc..
I am not sure whether I can help you or not but if you are up for it I will ask some basic questions perhaps we can get to the bottom of this, but I am a volunteer here / I have a day job.
Elastic certainly has other users running enrich on multi node clusters. (if I have a chance I will work up a sample)
You are running on Elastic Cloud from elastic.co not AWS ES correct?
Have you added / changes any of the default settings for elasticsearch?
Can we try to debug on just 2 Hot Nodes?
(not the Hot / Warm Architecture, we can perhaps get to that later / Data should not be being ingested on Warm nodes but that is an additional topic related to index lifecycle management )
In the dev tools can you run the following and provide outputs after you have set up the 2 nodes with the correct enrich policy.
GET /_cat/indices/.e*?v
GET _cat/aliases/.e*
GET _cat/shards/.e*?v
Can you provide the enrich policy?
Can you provide the ingest pipeline?
What tools are used to ingest the main time series data?
How is it configured?
How are you calling the pipeline?