Logstash date field problem

I cannot get logstash to format custom field to be suitable for sort in kibana.
pipeline filter config is:

filter {
if ([fields][log_type] == "zira_prod_log") {
grok {
match => { "message" => "[%{TIMESTAMP_ISO8601:timestamp}] %{NOTSPACE:log_type}: %{NOTSPACE:field3} [%{NOTSPACE:verb}] %{NOTSPACE:api_uri} %{GREEDYDATA:jsonstring1} %{GREEDYDATA:jsonstring2}" }
patterns_dir => ["/etc/logstash/.patterns"]
}
mutate {
rename => { "source" => "file" }
replace => { "[type]" => "%{[fields][environment]}-zira_prod_log" }
}
date {
timezone => "Europe/Zagreb"
match => [ "timestamp", "YYYY-MM-dd HH:mm:ss.SSS", "ISO8601" ]
target => "@timestamp"
}
}
}
neither @timestamp is overwritten with timestamp nor is timestamp sortable (because it gets type text and not date)
input is beats, output elastic, i tried multiple variations, always delete es index, restart logstash on pipeline changes, and recreate index pattern, but no combo i've tried helped

how es shows field:

    "type" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    }

log example:

[2020-03-11 14:38:18.123456] zira_api.INFO: REQUEST [GET] https://x.y.z/examplepath/category?filter... {"payload":"[object] (Infrastructure\External\...: )"} {"tags":{"request_id":"d8886a16-..."}}

found the problem, changed:

match => [ "timestamp", "YYYY-MM-dd HH:mm:ss.SSS", "ISO8601" ]

to:

match => [ "timestamp", "YYYY-MM-dd HH:mm:ss.SSS", "YYYY-MM-dd HH:mm:ss.SSSSSS", "ISO8601" ]

now the data gets into @timestamp, all ok

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.