Logstash(6.5.4) elasticsearch output

Elastic Stack Elasticsearch
1

My filebeat is on two servers. let's say app1 and app2

My logs are in the format

jobID status data

Example:
app1 log

5hgsxyt3838 SCHEDULED data

app2 log

5hgsxyt3838 COMPLETE data

Here both these logs have the same jobID.
I am using this jobID as the document _id on elasticsearch

elasticsearch {
			hosts => [ "localhost:9200" ]
			index => "import-export-logger-%{index-name}"
			document_id => "%{jobID}"
		}

Now I want my final status in elasticsearch as COMPLETE but sometimes there is a high load on app1 so app2 logs are processed first then app1.
So the final status becomes SCHEDULED

Is there a way to prevent this from happening i.e I want my document to be updated only when the status is not COMPLETE?
When the status is COMPLETE document must not update itself

2

Are you running 6.X? If so that's very much EOL and you will need to upgrade.

3

This is what my org is using.
So I have to configure for this version only

4

I do not think this is possible, at least not from Logstash. If you were on a newer vetsion it may have been possible to create a summary index using transform but your version is too old AFAIK.

5

Is there any other way to solve this .

6

Not that I am aware of or can think of.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.