Document id upsert update fields in new object and keep others

Hi,
I am using Elasticsearch/logstash and have an issue when updating an existing index through logstash

imagine the _source object the first time is:

    "_source": {
            "id": "1234",
            "msg_receive_ts": "2017-07-07T14:08:50.115Z",
            "event_status": "SUCCESS",
            "code": "ABCD"
        },

Then another row/object comes in on that same document id (1234)

    "_source": {
            "id": "1234",
            "msg_receive_ts": "2017-07-07T16:08:50.115Z",
            "event_status": "FAILED"
        },

It seems like the whole object is getting replaced. Before I had a 'code' field but now it's gone with this update/upsert through logstash to elasticsearch.

What I want to happen is only the fields that are in the new object get replaced and the other old ones stay.
So what I would like to see is:

    "_source": {
            "id": "1234",
            "msg_receive_ts": "2017-07-07T16:08:50.115Z",
            "event_status": "FAILED",
            "code": "ABCD"
        },

Is this easily achievable? Please help.

My output is something like:

  output {
       elasticsearch{
           hosts=> ["x.x.x.x:9200"]
           index => "test"
           document_id => "%{id}"
       }
  }

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.