Logstash(6.5.4) elasticsearch output

My filebeat is on two servers. let's say app1 and app2

My logs are in the format

jobID status data

app1 log

5hgsxyt3838 SCHEDULED data

app2 log

5hgsxyt3838 COMPLETE data

Here both these logs have the same jobID.
I am using this jobID as the document _id on elasticsearch

elasticsearch {
			hosts => [ "localhost:9200" ]
			index => "import-export-logger-%{index-name}"
			document_id => "%{jobID}"

Now I want my final status in elasticsearch as COMPLETE but sometimes there is a high load on app1 so app2 logs are processed first then app1.
So the final status becomes SCHEDULED

Is there a way to prevent this from happening i.e I want my document to be updated only when the status is not COMPLETE?
When the status is COMPLETE document must not update itself

Are you running 6.X? If so that's very much EOL and you will need to upgrade.

This is what my org is using.
So I have to configure for this version only

I do not think this is possible, at least not from Logstash. If you were on a newer vetsion it may have been possible to create a summary index using transform but your version is too old AFAIK.

Is there any other way to solve this .

Not that I am aware of or can think of.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.