Multi index search taking long time to give results

Elastic Stack Elasticsearch
1

Hi all,

I have 2 indices, index1 and index2 each containing thousands of records. I am querying index1, getting 40k hits, then for each hit I need to query index2 which itself gives 10k hits each time. This way, I end up with a for loop with 40k iterations. Each iteration has a for loop with 10k iterations. Making a total of 40k * 10k=400000,000 iterations. As you can see this is a huge value and I have to wait for about 30 minutes for the results.
There surely must be a better way to implement this. May be a function I am not aware of or maybe a better design for such queries.
Any help would be appreciated!!

2

Please NEVER invoke es API in a for loop !!! Use bulk operation and sleep to optimize it .
Also you can try to loop by page(if page size is 40, you just need 1k iterations ) , and use a list of ids as parameter instead of single id .

Another way in elasticsearch is using parent-child relationship : https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child.html