i use kafka(input) and logstash (1.5.2) with (output) elasticsearch (ES 1.5.2 or 1.6.0)
- i change the code in plugin output elasticsearch to be able to update documents and for that i use the UpdateRequest API.
https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/188/files - until Elasticsearch 1.5.0 the function request.script(args[:_script]) works without problem but after in versions 1.5.2 or 1.6.0 this function doesn't work anymore
Even a simple action in the script like update a integer field fail :
{"document_id":"2754","script":"ctx._source.country = 76","action":"update","retry_on_conflict":3}
error log :
failedactionwithresponseof400,
droppingaction: [
"update",
{
: _id=>"21-2-1",
: _index=>"drilling_m02_2014_test",
: _type=>"detailed",
: _routing=>nil,
: _script=>" if (ctx._source.edt == null) { ctx._source.edt = eventData; ctx._source.vdu = []; ctx._source.pdt = []; } else { ctx._source.edt += eventData; } ",
: _scriptParams=>{
"label"=>"eventData",
"params"=><Java: : JavaUtil: : ArrayList: -610349907[
{
"filter"=>"city",
"value"=>"New York",
"vid"=>156442965,
"eid"=>16928
},
{
"filter"=>"image_id",
"value"=>"34eerts7",
"vid"=>156442965,
"eid"=>16928
}
]>
},
: _retry_on_conflict=>5
},
#<LogStash: : Event: 0x3a60323f@metadata={
"retry_count"=>0
},
@accessors=#<LogStash: : Util: : Accessors: 0x38494d4c@store={
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
@lut={
"@version"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"@version"
],
"index_date"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"index_date"
],
"document_id"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"document_id"
],
"action"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"action"
],
"script"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"script"
],
"scriptParams"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"scriptParams"
],
"retry_on_conflict"=>[
{
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
"retry_on_conflict"
]
}>,
@data={
"action"=>"update",
"document_id"=>"21-2-1",
"index_date"=>"drilling_m02_2014_test",
"@timestamp"=>"2015-07-13T09:58:13.297Z"
},
@metadata_accessors=#<LogStash: : Util: : Accessors: 0x3dc05ec4@store={
"retry_count"=>0
},
@lut={
}>,
@cancelled=false>
]{
: level=>: warn
}
someone know how i can fixe it please ?
issue open in github https://github.com/elastic/elasticsearch/issues/12205