Error:
So when I load dashboard for a period of 7 days,I'm not having any issues. It loads dashboard perfectly.
However when I want to see data for 30 days or 90 days, it takes about 2 minutes to load dashboard & then throws error as above.
Troubleshooting:
It may be a memory issue, but I have set heap space in jvm options. as follows: -Xms4g -Xmx4g
Increased search max bucket size to 20,000 in cluster settings.
Have 1000 shards and 1000 replicas for my indices data.
Still the error shows up.
Used Memory shows as 2 GB out of available 4GB when checked.
Basically i think it is unable to handle data so it is throwing error.
What other changes need to be made to handle this error?
How much data do you have? I'm no expert but 1000 shards seems way excessive if you only have 4GB of memory. For example I have over 250GB of data in 2.5 billion documents with 15GB (half of that assigned to ES) of ram and only 460 shards.
Have too many shards will only decrease performance.
If the error is caused by the search taking too long there is not much you can do. You can increase the timeout (see the docs) on the Elasticsearch side but most likely you'll need to give more resources to your server (queries are mostly depending on CPU I believe) if you want to keep load time acceptable.
Another thing you could do is look at your visualizations. Having too many of them on a single dashboard can impact performance just as poorly designed visualizations can. I believe that specially things such as terms on a timestamp field in a table can destroy performance.
But most likely you'll just need to trow more processing power at it.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
