Convert_datatype with dissect filter > error: "data conversion uncoercible"

Elastic Stack Logstash
1

Hello,

I have some squid access-log that I want to integrate in my Elastic indexes.

I am using the dissect filter to filter the different fields and then I use "convert_datatype" to convert the needed fields into integers.

My conf file :

filter {
dissect {
mapping => {

        "message" => "%{} %{} %{} %{host} %{} %{timestamp->} %{} %{user_id} %{req_status} %{status_code} %{user_ip} %{user_req_size} %{method} %{url} %{mime_type} %{reply_size} %{hierarchy} %{fwd_ip} %{srv_ip} %{srv_port} %{duration}"
      }

      remove_field => ["message"]

      convert_datatype => {
        "status_code" => "int"
        "user_req_size" => "int"
        "reply_size" => "int"
        "duration" => "int"
      }

    }

}

Some event examples :

Jun 12 11:37:21 infpvpxe1 (squid-5): 12/Jun/2019:11:37:21 +0200 - TCP_DENIED 403 10.106.193.248:44440 3369 GET http://intranoo.francetelecom.fr/ text/html 3331 HIER_NONE - 10.106.193.129 3128 0
Jun 12 11:37:21 infpvpxe1 (squid-2): 12/Jun/2019:11:37:21 +0200 iosw_hp TCP_MISS 200 10.117.56.2:32310 3969 CONNECT ws.chronopost.fr:443 - 3851 HIER_DIRECT ws.chronopost.fr 10.106.193.129 3128 44

With convert datatype, I obtain this error in the logstash stdout :

"tags" => [
[0] "_dataconversionuncoercible_duration_int"
],

while the second example return the correct "duration" field as "44"

The problem is : In log lines where the duration is "0", there is a (second) space juste before the 0, so convert_datatype is not able to convert a string with a space and a digit, into an int.

Is there a way to do so ?
Do I have no other choice than using a grok filter juste for the duration field ?

Thank you

2

You can use

%{srv_port->}

to consume optional padding to the right of the port number.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.