Kibana Report Generation Timeout

Elastic Stack Kibana
1

We currently have Elasticsearch, Logstash and Kibana installed (version 7.16.1).. We use scripts to generate and download data from Kibana/Elasticsearch using curl and a POST URL.

One intermittent issue we have been facing is that sometimes one of our reports comes up with no data and the contents of the downloaded file just says "processing". I am wondering if there is a timeout setting within Kibana or some other mechanism that could allow more time for the report to run?

This only seems to happen on days where there is a lot of data in the reports.

Any help would be greatly appreciated.

2

It sounds like Kibana is still processing the report. There should be a 503 as the response code when this is the case. Your scripts need to check the HTTP response codes to coordinate with Kibana and download the report when it is done.

See Automatically generate reports | Kibana Guide [8.11] | Elastic

3

Thanks for the response. Right now my script does not check the HTTP response code prior to attempting the download. We just generate the Kibana post url using curl -X POST, have our script wait a predefined amount of time, then attempt to download the report via another curl. We tried adjusting the wait time but no matter how long we set it, it never seems to finish on time.

Would you recommend restructuring our script to take the HTTP response code into consideration? Personally, I have never done this so I would need to research how this is done.

4

Heya, I'm not so comfortable reviewing your scripts. Instead, I'd like to just share a shell script which uses curl and jq to do the job:

#!/bin/bash
#set -x

HOST=https://example.com
AUTH=test_user:changeme
PREFIX=/kibana

POST_URL="${HOST}${PREFIX}/api/reporting/generate/csv_searchsource?jobParams=....."

DOWNLOAD_PATH=${HOST}$(curl --silent -XPOST "$POST_URL" -H "kbn-xsrf: get_report" -u "$AUTH" | jq -r ".path")

echo Download path: $DOWNLOAD_PATH

STATUS=''
while [[ -z $STATUS || $STATUS =~ .*503.* ]]
do
  STATUS=$(curl --silent --head "$DOWNLOAD_PATH" -u "$AUTH" | head -1)
  echo $STATUS
done

# download final report
curl "$DOWNLOAD_PATH" -u "$AUTH"
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.