Dissect plugin reads too much into last field

Elastic Stack Logstash
1

Hey folks, I am using dissect plugin to slice Webserver log line into JSON document.

Logstash version is 6.0.1.

Here is NGINX log format:

log_format  main  '$remote_addr ($http_x_forwarded_for) - $remote_user [$time_iso8601] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" [$request_time] [$upstream_response_time] -';

Here how log acutally looks when above format is used:

10.0.2.2 (-) - - [2018-01-28T08:40:43+00:00] "POST /words HTTP/1.1" 200 699 "-" "Python-urllib/3.6" [0.012] [0.011] -
10.0.2.2 (-) - - [2018-01-28T08:40:43+00:00] "POST /words HTTP/1.1" 200 947 "-" "Python-urllib/3.6" [0.012] [0.011] -
10.0.2.2 (-) - - [2018-01-28T08:40:43+00:00] "POST /words HTTP/1.1" 200 944 "-" "Python-urllib/3.6" [0.012] [0.011] -
10.0.2.2 (-) - - [2018-01-28T08:40:43+00:00] "POST /words HTTP/1.1" 200 302 "-" "Python-urllib/3.6" [0.011] [0.010] -
10.0.2.2 (-) - - [2018-01-28T08:40:43+00:00] "POST /words HTTP/1.1" 200 647 "-" "Python-urllib/3.6" [0.012] [0.011] -

Here is pipleline used in Logstash to parse the above content:

filter {
    dissect {
        mapping => {
            'message' => '%{remote_addr} (%{http_x_forwarded_for}) %{} %{} [%{event_time}] "%{request}" %{status} %{body_bytes_sent} "%{http_referer}" "%{http_user_agent}" [%{request_time}] [%{upstream_response_time}] %{}'
            'request' => '%{http_method} %{request_path} %{http_version}'
        }

        convert_datatype => {
            request_time => 'float'
            upstream_response_time => 'float'
            status => 'int'
            body_bytes_sent => 'int'
        }
    }

    date {
        match => [ "event_time", "ISO8601" ]
        target => '@timestamp'
    }
}

And this is Logstash error I am getting:

[2018-01-28T08:34:40,531][WARN ][org.logstash.dissect.Dissector] Dissector datatype conversion, value cannot be coerced, field: upstream_response_time, value: 0.012] -

Please note that "request_time" field is being parsed correctly.

Thanks for your help.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.