Deleting the data in Index before indexing the new data from input

Elastic Stack Logstash
1

I have a batch monitoring log files which will keep logging the job status and I'm indexing that data from logs to ElasticSearch using FileBeats & Logstash.

Here is the data :

Run 1:
job="job_one" status="Not Started"
job="job_two" status="Not Started"
job="job_three" status="Not Started"

Run 2:
job="job_one" status="Started"
job="job_two" status="Started"
job="job_three" status="Started"

Run 3:
job="job_two" status="In Progress"
job="job_three" status="In Progress"

Run 4:
job="job_three" status="In Progress"

Run 5:
*NOTHING

The challenge that I'm having here is, when I look at the latest run, If the job is not available which means that the job is completed. Ideally, If we have status as 'completed' in the log file, it would be easier to filter using that field. But now, since it's not available, How can I show pending jobs based on the latest run ?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.