Timestamp not mapping with event time

Elastic Stack Logstash
1

My logs are exported in csv format and uploaded. The first column name called "Event Time" has following format event time followed by rest of the comma separated columns - "2020/11/10 00:00:00 CET"

I am not able to match @timestamp with the above log event time. Actually I want to replace the @timestamp with the above event time but the @timestamp is always showing as index creation or the data loaded time.

Below is what I have in filter that is not working. Can someone help I tried with several other combinations but was not working

filter {
    {
   date { match => ["Event Time", "yyyy/mm/dd HH:mm:ss Z"]
          target => "@timestamp"
        }
}
2

Try "yyyy/MM/dd HH:mm:ss Z"

3

No luck @Badger still not working, below is my code

filter {
    csv {
       separator => ","
       columns => ["Event Time", "Event2", "Event3"]
        }
   date { match => ["Event Time", "yyyy/MM/dd HH:mm:ss Z"] }
}

Log line sample

"2020/11/10 00:00:00 CET","2020/11/10 00:00:00 CET","192.168.1.10".....
4

Z is for timezone offsets without a colon, such as -0700. You have a timezone name. Try "yyyy/MM/dd HH:mm:ss ZZZ".

5

Post that change no data is visible. Got below error however the logstash is running

[2020-12-18T06:18:46,730][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-12-18T06:18:46,867][WARN ][logstash.filters.csv     ][main][f2b1efd928c7b7891ecf22895c965305a8a857d09e257a38caae1d86bd037669] Error parsing csv {:field=>"message", :source=>"A TCP connection between two hosts was deleted", :exception=>#<NoMethodError: undefined method `empty?' for nil:NilClass>}
[2020-12-18T06:18:46,904][WARN ][logstash.filters.csv     ][main][f2b1efd928c7b7891ecf22895c965305a8a857d09e257a38caae1d86bd037669] Error parsing csv {:field=>"message", :source=>"A UDP connection slot between two hosts was deleted", :exception=>#<NoMethodError: undefined method `empty?' for nil:NilClass>}
[2020-12-18T06:18:46,914][WARN ][logstash.filters.csv     ][main][f2b1efd928c7b7891ecf22895c965305a8a857d09e257a38caae1d86bd037669] Error parsing csv {:field=>"message", :source=>"A UDP connection slot between two hosts was deleted", :exception=>#<NoMethodError: undefined method `empty?' for nil:NilClass>}
[2020-12-18T06:18:46,922][WARN ][logstash.filters.csv     ][main][f2b1efd928c7b7891ecf22895c965305a8a857d09e257a38caae1d86bd037669] Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `empty?' for nil:NilClass>}
[2020-12-18T06:18:46,925][WARN ][logstash.filters.csv     ][main][1aba530ae982a28939d95fcd27f39e82a105f6039e1b332ef60e87326ffec1ff] Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `each_index' for nil:NilClass>}
[2020-12-18T06:18:46,930][WARN ][logstash.filters.csv     ][main][f2b1efd928c7b7891ecf22895c965305a8a857d09e257a38caae1d86bd037669] Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `empty?' for nil:NilClass>}
6

Those errors are just telling you that the [message] field does not contain a comma, so the CSV.parse_line function returns nil.

7

@Badger it did not worked. Still same issue not able to match the Event time with @timestamp. Am I missing anything in the configuration?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.