We are currently using Elasticsearch 6.5 and getting intermittent request timeout for some dashboards. We are querying on approximately 550 GB data (76873087 documents) which is distributed among 3 indices. We have 6 data nodes of 64 GB each and 3 master nodes of 8 GB each.
Each index has 6 shards and 1 replica.
So should we increase the request timeout or decrease the amount of querying data or any other suggestion that you may have to resolve this?
Those are your options, along with: beefing up your servers, optimizing your indices, scaling out, etc. If you can query for less data without negatively impacting your use cases, that's the ideal starting point, I'd say.