Hi ,
I have some raw data and after preprocessing it through python I am uploading it on ELK through logstash in the following way.
cat get_info.sh
#!/bin/bash
/home/abc/script/allocated_budget_info.py | /usr/share/logstash/bin/logstash -f /home/abc/script/budget.conf --path.settings=/etc/logstash --path.data=/home/abc/junk1
When I run this script (get_info.sh) and if it fails due to issue with python script (/home/abc/script/allocated_budget_info.py) It show me an error on terminal .
Now I want the check if there is any way (plugin) to display an error on terminal if it would not able to upload the data on ELK due to some error . Although I can see the error on logstash log file but that is not helpful for me as I trigger this script at some specific action and want to generate an alert email / msg if data not uploaded on ELK successfully.
I tried stdout plugin but it showing same msg on both pass/fail condition.
For testing purpose I passed the date in yyyy-m-d format and it didn't upload the data on ELK (as expected as in Elastic date format should be yyyy-mm-dd format). It show an error on logstash log file:
error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [startDate]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2019-1-31" is malformed at
But of terminal in both success and failure cases it showing the same msg .
2019-10-17T05:41:04.125Z dev02 %{message}
2019-10-17T05:41:04.133Z dev02 %{message}