Status 404 on update

Elastic Stack Logstash
1

Hi,

I have a document that is inserted into elasticsearch under the index: index-%{+YYYY.MM.dd}
where %{+YYYY.MM.dd}=2017-08-09
A day later another event comes in and I have to update that document I inserted yesterday.

When trying to update I get a status 404 meaning it cannot find that record is my guess.
When the inserted document is updated on the same day and thus also under the same index it finds it and updates it without any errors.

How can I get hold of the index for the document I am trying to update so that I can dynamically use it in the output when updating?

Regards benny

2

You can use an elasticsearch filter to locate the original document. The filter has options to save metadata like the index name into fields that you can use later on in the elasticsearch output.

3

Hi Magnus,

I tried what you suggested but for some reason I get a "nil" when I copy that field to my new field current_index:

elasticsearch {
		hosts => ["localhost:9200"]
		index => "portal-*"
		query => "ORDER_ID:%{ORDER_ID}"
		fields => { "message" => "message" }
		fields => { "_index" => "current_index" }
}

This is the result when I send the output of that field to a log file: "current_index" => nil
I also tried it without the underscore.
Am I using the wrong identifier?

FYI: The message comes through no problem.

Regrads benny

4

This doesn't work because _index isn't a field in the original document. Unfortunately I mixed up the elasticsearch index with the elasticsearch filter. The former can add metadata fields like the index name to the event but the filter can't. There's an open pull request to add that , though:

5

Hi Magnus,

Do you mean that this is currently not possible and that it will be in a future release?
Is there a work-around for this problem?

Regards benny

6

Do you mean that this is currently not possible and that it will be in a future release?

Yes.

Is there a work-around for this problem?

None that I can think of now.

7

Okay, thanks for the reply Magnus!
I will investigate further.

Regards benny

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.