My work environment contains 7 days of data, and when looking at elasticsearch, it contains 172 indices (1720shards, 500 million docs). Then, using reathighlevelcilent, we issue a query to get 7 days of data sorted by timestemp.
There is no timeout until 1st and 2nd, but as search time is gradually increased, it becomes slower and eventually timeout occurs in searchrespons.
I know elasticsearch is a very good device for processing large amounts of data, but I don't understand that timeout occurs for up to 500 million data.
There might be a way to extend the timeout of the timeout as a workaround, but it is not the underlying problem solution I want.
I believe that the timeout that occurs in elasticsearch can be solved through setting as well as a way to extend the time.
However, I am not sure about elasticsearch, so I am looking for help.
Clearly, elasticsearch is an analysis tool that can process large amounts of data quickly.