Unknown error occurred sending a bulk request to Elasticsearch

Elastic Stack Logstash
2

That's blowing up here. Per this comment on the same error in a different context it means you have non-UTF-8 characters in one of your fields.

OK, if in a ruby filter you do

            text = [0xD1].pack("C*")
            text.to_json

you will get

"\xD1" from ASCII-8BIT to UTF-8 {:class=>"Encoding::UndefinedConversionError"

You have a couple of options. If you know the encoding of the string you have you
could try this ruby code

text = event.get("problemField") 
text = text.force_encoding("iso-8859-1").encode("utf-8")
event.set("problemField", text)

which will get you

"problemField" => "Ñ",

I am not telling you that you have iso-8859-1 encoded text. It's plausible but you need to determine if this is true.

If the encoding is unknown or varies then a more aggressive approach would be

event.set("problemField", text.encode("UTF-8", "binary", :invalid => :replace, :undef => :replace)

which will get you

"problemField" => "�",

I recognize that losing data is bad, but your fields have to be valid UTF-8 to be sent to elasticsearch. It is not optional.

I assume you know which fields are likely to have non-UTF-8 data in them. If you do not know then you will have to iterate over the fields of the event. This code should give you some ideas.

I hope this helps you to understand the issue.

Another thing you could do in ruby is

            initialEvent = event.to_json # String.to_json crashes, Hash.to_json does not
            fixedEvent   = initialEvent.encode("UTF-8", "binary", "replace" => "x", :invalid => :replace, :undef => :replace)
            if fixedEvent != initialEvent
                event.tag("encodingProblem")
            end

then route to a different output based on "encodingProblem" in [tags], if you use a rubydebug output then start looking for \x, as in

"problemField" => "\xD1",
        "tags" => [
    [0] "encodingProblem"
],