Hi,
How can I update a single field on an existing document in elastic from logstash?
This is the docment in elastic where I want to upate field: "FEEDBACK_RECIEVED" from "NO" to "YES"
{
"_index": "portal-2017.07.21",
"_type": "log",
"_id": "100000041",
"_version": 1,
"_score": null,
"_source": {
"offset": 26230,
"FEEDBACK_RECIEVED": "NO",
"APPLICATION_ID": "VNW2016000041",
"input_type": "log",
"source": "xxx.2017-06-30.log",
"message": "2017-05-30 10:56:30,118 INFO [http-bio-8080-exec-9][ServiceImpl:125] Checkout registered: Registering E-Form applicationId=[[VNW2016000041]], orderId=[[100000041]].",
"type": "log",
"tags": [
"beats_input_codec_plain_applied",
"UPDATE_DOCUMENT_ID",
"SEND_LINE"
],
"@timestamp": "2017-07-21T08:41:50.614Z",
"ORDER_ID": "100000041",
"@version": "1",
"beat": {
"hostname": "V2W1-HOST",
"name": "V2W1-HOST1",
"version": "5.4.0"
},
"host": "V2W1-HOST1",
"_TIMESTAMP": "2017-05-30 10:56:30,118",
"fields": {
"app_id": ".portal"
}
},
"fields": {
"@timestamp": [
1500626510614
]
},
"sort": [
1500626510614
]
}
After udating in logstash output I see that the message and the tags are updated which I want to be left as is.
But "FEEDBACK_RECIEVED" which I expected to have the value "YES" is not updated.
Below is de the result after upate:
{
"_index": "portal-2017.07.21",
"_type": "log",
"_id": "100000041",
"_version": 2,
"_score": null,
"_source": {
"offset": 26459,
"FEEDBACK_RECIEVED": "NO",
"APPLICATION_ID": "VNW2016000041",
"input_type": "log",
"source": "xxx.2017-06-30.log",
"message": "2017-05-30 10:56:30,118 INFO [http-bio-8080-exec-9][ServiceImpl:125] Checkout payment feedback: Processing payment feedback orderId=[[100000041]], transactionStatus",
"type": "log",
"tags": [
"beats_input_codec_plain_applied",
"UPDATE_FEEDBACK_RECIEVED",
"SEND_LINE"
],
"@timestamp": "2017-07-21T08:43:35.614Z",
"ORDER_ID": "100000041",
"@version": "1",
"beat": {
"hostname": "V2W1-HOST1",
"name": "V2W1-HOST1",
"version": "5.4.0"
},
"host": "V2W1-HOST1",
"_TIMESTAMP": "2017-05-30 10:56:30,118",
"fields": {
"app_id": ".portal"
},
"YES": "NO"
},
"fields": {
"@timestamp": [
1500626615614
]
},
"sort": [
1500626615614
]
}
This is the filter section where I find the old event:
elasticsearch {
hosts => ["localhost:9200"]
index => "portal-*"
query => "ORDER_ID:%{ORDER_ID}"
fields => {"FEEDBACK_RECIEVED" => "YES"}
}
And this is the output section:
elasticsearch {
action => "update"
document_id => "%{ORDER_ID}"
index => "portal-%{+YYYY.MM.dd}"
hosts => [ "localhost:9200" ]
}
Any help would greatly be appreciated.
Regards benny