I'm just implemented elastic stack as a log analyzer and still learning to use it.
I have a problem with kibana on a dashboard with multiple visualizations. When I'm trying to show all visualizations from 2 months ago to present an error occured like this "[esaggs] > JSON.parse: unexpected character at line 1 column 1 of the JSON data" over and over again with multiplied "[esaggs]". But when I'm trying to show all visualizations from a month ago to present is successful.
Futhermore on the other dashboard with multiple visualizations is successful to show all visualizations even from 3 months ago to present.
My questions are:
What is the meaning of this error message?
Which one is the problem? kibana or elasticsearch?
Where I can find more detailed error message to solve this problem?
I have set elasticsearch.requestTimeout and elasticsearch.shardTimeout to 90000 and the result is successful to show all visualizations from 2 months ago
Futhermore I'm trying to show all visualizations from 3 months ago and the result is [esaggs] > JSON.parse: unexpected character at line 1 column 1 of the JSON data again, even though I have set elasticsearch.requestTimeout and elasticsearch.shardTimeout to 300000
Is this my server limits due to its specification?
Is there a way to turn off the timeout feature and make kibana wait for all visualizations process finish? I don't mind if the visualization process take half an hour or more..
Check your Kibana and Elasticsearch server logs whether you are running in other timeouts. I think starting in 7.8 or 7.9 there is a feature to let queries run past the timeout (there will be a modal with a button you can confirm).
Also, these runtimes are quite high. If you are not using the individual documents but are just interested in aggregates, consider using the rollup feature to store only the metrics you are interested in - this will result in faster queries as well as less disk space.
I just upgraded the elastic stack to 7.8.0 (I want to upgrade to 7.9.0 but still unavailable publicly) and as you said the new feature to let queries run beyond timeout is available.
Finally I can show all visualizations even from the beginning to present!
It was took only for several minutes to finish the whole visualizations in a dashboard, which is fine.
Thank you Joe for your assistance!
Have a nice day!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.