I have an ElasticSearch 6.4 index with 5 shards, 1 replica for each shard and 1.5 billions of documents.
I use Elastic from textual search, getting the result's ids andusing then to query on MongoDB.
We need get a roof of 350.000 documents for each query, and my problem start here. Setting query size between 20 and 50.000 documents, my search take less then 10 seconds to respond with more or less 10MB of non compressed JSON. But when I increase the size, the time increase exponential, and I can't get the results.
Any suggestions to resolve my problem? I can accept queries with 30 seconds, but I need a roof of 350.000 documents.