When I run this command, the csv report that comes out just contains the work 'Processing'. Is there a way to wait until the csv report is ready? I am version 7.10 of the ELK stack.
Hey, sorry for the late answer. In your new script it seems like you aren't extracting the path.
The idea is the following:
Kick off the report generation with the POST url copied from the UI (send as POST request)
This returns 200 immediately, and the response contains the path
Poll the path via GET request, until the response code turns 200
Once the response code turns 200, the response body will be the final report
Your second script seems to poll the initial post request instead.
My shell scripting game is really low, but something like this:
for filename in ${KIBANA_QUERIES}/*.rison; do
echo "Processing query file: $filename"
CSV_REPORT=$(basename $filename.csv)
REPORT_PATH=$(curl -s --insecure -u "elastic:${ELASTIC_SECRET_UMS}" -H "kbn-version: 7.11.2" -XPOST $(cat $filename) | jq -r '.path')
echo "Report path: $REPORT_PATH"
REPORT_READY=400
while [ "${REPORT_READY}" != "200" ]
do
REPORT_READY=$(curl -I -s --insecure -u "elastic:${ELASTIC_SECRET_UMS}" https://${K8S_NODE_IP}:${KIBANA_NODE_PORT}${REPORT_PATH} | grep HTTP | awk '{print $2}')
if [ "$REPORT_READY" == "404" ]; then
echo "Exiting on 404"
break;
fi
sleep 1
echo "Waiting for report to be ready, status $REPORT_READY"
done
if [ "$REPORT_READY" == 200 ]; then
echo "Downloading report to: $CSV_REPORT"
curl -s --insecure -u "elastic:${ELASTIC_SECRET_UMS}" https://${K8S_NODE_IP}:${KIBANA_NODE_PORT}${REPORT_PATH} --output $HOME/Downloads/$CSV_REPORT
fi
done
It's downloading the final report twice, it's probably possible to optimize that, but I don't know how (you need to get the status code and the response body in the same call)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.