Split timestamp to date and time

Elastic Stack Logstash
1

Hi,

How do I split timestamp field and store the result into timestamp_date and timestamp_time in logstash.

Thank you.

2

I manage to get both date and time into saperate variables. Here is my script:

#Logstash filter
#/etc/logstash/patterns/ contains pfsense2-4-grok downloaded from https://github.com/patrickjennings/logstash-pfsense/blob/master/patterns/pfsense2-4.grok
filter{
    grok{
        patterns_dir => "/etc/logstash/patterns"
        match => [
            "message", "%{TIMESTAMP_ISO8601:timestamp} %{WORD:pf_host} filterlog: %{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}",
            "message", "%{TIMESTAMP_ISO8601:timestamp} %{WORD:pf_host} filterlog: %{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}"
        ]
    }
    mutate {
        # Add 2 fields for date and time and set their value to timestamp
        add_field => { "timestamp_date"=> "%{timestamp}" }
        add_field => { "timestamp_time"=> "%{timestamp}" }
    }
    mutate {
        # Remove time section
        gsub => ["timestamp_date", "T\d{2}:\d{2}:\d{2}((.\d{3}Z)|([+\-]\d{2}:\d{2}))", ""]
    }
    mutate {
        # Remove date section
        gsub => ["timestamp_time", "\d{4}-\d{2}-\d{2}T", ""]
    }
    date {
        match => ["timestamp", "ISO8601"]
    }
}

However, I notice that my @timestamp is wrong. The system took the current date instead of the logged date. I might have missed some config lines here.

Please assist.

Thanks.

3

What does the [timestamp] field look like?

4

The image was below taken from Kibana showing all the extracted fields.

image
image.png867×311 31.1 KB

From the above image, please note the following :

  • Text highlighted in RED is @timestamp field which refers to the execution time
  • Text highlighted in BLUE is timestamp field which refers to the actual event time extracted from the log

Question : How do I make @timestamp having the same value as timestamp ?

Thank you.

5

Please do not post pictures of text, just post the text. You can copy and paste it from the JSON tab in Kibana. Use markdown to make sure it is formatted correctly in the preview pane to the right of the edit pane.

6

Dear Badger,

Sorry about that. Will do as recommended in the future.

Thanks.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.