Problem with elastic output plugin

Elastic Stack Logstash
1

Hi guys, how are you ?

I have a problem with logstash and elastic output plugin,
I'm sending a file to this logstash conf with json records on it, and i want to put them in elastic,
this is my conf

input{
        stdin{
                codec => json
        }
}

filter{

}

output{
#       stdout{
#               codec => dots
#       }
        stdout {
                codec => rubydebug { metadata => true}
        }

        elasticsearch{
                #elastic desarrollo 192.168.0.141
                hosts => ["192.168.0.141:9200"]
                user => root
                password => 12ll223ll
                index => "contactos_cr_3"
                document_type => "contactos_cr"
                document_id => "%{[account]}"
                action => "update"
                doc_as_upsert => true
                script => '
                        ctx._source.cliente = params.event.get("cliente");
                        ctx._source["@timestamp"] = params.event.get("@timestamp");
                        //Debug.explain(params.event.get("telefonos").getClass().getName());
                        ArrayList new_phones = params.event.get("telefonos");
                        //if(new_phone != "null"){
                        for (String new_phone : new_phones){
                                if(ctx._source.telefonos != null && ctx._source.telefonos instanceof ArrayList){
                                        if(!ctx._source.telefonos.contains(new_phone)){
                                                ctx._source.telefonos.add(new_phone);
                                        }
                                }else{
                                        if(ctx._source.telefonos instanceof String && ctx._source.telefonos != new_phone){
                                                String aux = ctx._source.telefonos;
                                                ctx._source.telefonos = null;
                                                ctx._source.telefonos = new ArrayList();
                                                ctx._source.telefonos.add(aux);
                                                ctx._source.telefonos.add(new_phone);
                                        }
                                }
                        }'
        }
}

This logstash is in a different server from the one that has elasticsearch installed, and the logstash log has this error when is executed:

[2017-12-22T07:57:13,115][INFO ][logstash.inputs.stdin    ] Automatically switching from json to json_lines codec {:plugin=>"stdin"}
[2017-12-22T07:57:13,568][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://~hidden~:~hidden~@192.168.0.141:9200"]}}
[2017-12-22T07:57:13,570][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-12-22T07:57:13,849][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Template file '' could not be found!", :class=>"ArgumentError"}
[2017-12-22T07:57:13,850][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["192.168.0.141:9200"]}
[2017-12-22T07:57:13,853][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2017-12-22T07:57:13,862][INFO ][logstash.pipeline        ] Pipeline main started
[2017-12-22T07:57:13,926][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9602}
[2017-12-22T07:57:14,071][ERROR][logstash.outputs.elasticsearch] Got a bad response code from server, but this code is not considered retryable. Request will be dropped {:code=>406}
[2017-12-22T07:57:16,889][WARN ][logstash.agent           ] stopping pipeline {:id=>"main"}
[2017-12-22T07:57:17,226][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:url=>#<URI::HTTP:0x605b1e6d URL:http://~hidden~:~hidden~@192.168.5.141:9200>, :healthcheck_path=>"/"}

Says got a bad response code from server code 406,
Do you know what can it be?

when trying the same conf inserting to an elasticsearch that is installed on the same server as the logstash it works fine, the problem is when is pointing to this other elasticsearch that is in another server.

I hope that you understand what I've said, My english is not very good.

Thanks you !!

Regards

2

did you recently update to elasticsearch 6.0?

3

yes i did, is that a problem ?

4

I wouldn't say it's a problem, but I'm wondering if that's what the issue is. One of the changes is that Elasticsearch will throw the error you are getting if the http request it receives doesn't explicitly set the Content-Type to application/json, so I was thinking maybe you updated Elasticsearch to 6.0 but not Logstash and Logstash isn't setting that header?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.