Reading date format in logstash date filter

[UsmanNiazi]

Hi,

I am unable to convert this string "03/31/2023 03:15 AM PDT" to date when using logstash date filter. Getting error dateparse failure.

I am using below script
date {
match => [ "start_time", "mm/dd/yyyy HH:mm Z","MMM d yyyy HH:mm:ss", "ISO8601" ]
target => "start_date"

Thanks

[Badger]

PDT is ambiguous. You are missing an "a" to consume the AM/PM, you have a timezone id (ZZZ) not a numeric offset, you want either hh or KK rather than HH, and months are MM (mm are minutes).

    mutate { gsub => [ "message", "PDT", "PST8PDT" ] }
    date { match => [ "message", "MM/dd/yyyy hh:mm a ZZZ" ] target => "start_date" }

will produce

   "message" => "03/31/2023 03:15 AM PST8PDT",
"start_date" => 2023-03-31T10:15:00.000Z

[UsmanNiazi]

Still getting the below error
"start_time" => "03/30/2023 11:35 PM PDT",
"tags" => [
[0] "_dateparsefailure"
],

[UsmanNiazi]

Here is my input section `filter {
csv {

separator => ","

	skip_header => "true"
    columns => [ "name","project_name","execution_source","requested_at","start_time","end_time","duration(sec)","status","execution_uid","workflow_uid" ]
 }

mutate {
convert => ["duration(sec)", "float"]

gsub => [ "start_time", "PDT", "PDT" ]

     remove_field => ["message","host","path","@version","@timestamp"]
}

mutate {
gsub => [ "message", "PDT", "PST8PDT" ] }
date {
match => [ "start_time", "MM/dd/yyyy hh:mm a ZZZ" ]
target => "start_date" }
}`

[UsmanNiazi]

It worked but its showing 5 hours back time in date.

[Badger]

5 hours back from what? The start_date field will always be UTC, so today it will be 7 hours ahead of PDT.

[UsmanNiazi]

In csv file time is 5am and in elastic its showing 10am

[Badger]

That would be expected if your server were in (for example) Asia/Karachi. If you need to tell the date filter what timezone the data in the csv is in then use the timezone option.