Wrong logstash input filter invoked: csv filter is being invoked rather than http

Elastic Stack Logstash
1

When I PUT a json body to the configured http input url, the CSV filter is being invoked instead of http. What am I missing?
PUT:

curl --header "Content-Type: application/json" \
--request PUT \
--data '{"field1": "manual"}' \
http://my.hostname.here:6060/uri

Error:

[logstash.filters.csv     ] Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `each_index' for nil:NilClass>}

http-input plugin version: 3.3.0

Config:

    input {
        beats {
            port => 5044
         }
        http {
            host => "0.0.0.0"
            port => 6060
        }
}

filter {

    mutate {
        remove_field => [ "[beat]", "[host][name]", "[input][type]", "[prospector]", "[temp]", "[host]", "[source]", "[agent]" ]
        remove_tag => [ "beats_input_codec_plain_applied" ]

    }

    http {
        url => "http://my.hostname.here:6060/uri"
    }

    csv {
        separator => ","
        skip_header => true
        skip_empty_columns => true
        columns => [ "[col1]", "col2" ]
    }

}
2

Your configuration unconditionally runs a mutate filter, then an http filter, then a csv filter.

When you post application/json into an http input it is parsed, and there is no [message] field added. The csv filter will try to parse the [message]. I would expect that to be a no-op, but I guess not.

3

Oh I see. I thought that only that type of input would invoke the appropriate filter.

Thanks for pointing that out!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.