Change timestamp in logstash input

Elastic Stack Logstash
1

Hi, I have a log like this and want to change the timestamp to the time in the log.

Example Log

event1: aaaaaaa | event2: xxxxxxx | event3: ccccccc | date : 2022-11-07T21:03:48.9110;

The logstash

input {
  file {
    path => "D:/dump/*.log"
    start_position => "beginning"
    mode => "read"
    file_completed_action => "delete"
    sincedb_path => "NUL"
  }
}

filter {
    grok {
	  break_on_match => false
	  match => { "rawMessage" => [
			      "(?<Event1>(event1:)([^|]*))",
				  "(?<Event2>(event2:)([^|]*))",
				  "(?<Event3>(event3:)([^|]*))",
				  "(?<date>(date.:)([^;]*))"] }
    }

   date {
			match => "%{date}"
			target => "@timestamp" 
		}
}

output {
		elasticsearch { 
			hosts => ["localhost:9200"]
			index => "event_test"
		}
}
2

Try with this:

	 date {
      match => [ "date", "ISO8601" ]
	  target=> "@timestamp"
	 }

PS. Since this is separated with |, you can use csv or dissect filters.

3

It still gives results like the following. Log stays in time when sent.
image

4

Your grok is not OK. The date field has value:

{
  "date": [
    [
      "date : 2022-11-07T21:03:48.9110"
    ]
  ]

Try something like this:

grok {
	  match => { "rawMessage"  => "event1: %{DATA:event1} \| event2: %{DATA:event2} \| event3: %{DATA:event3} \| date : %{DATA:date};"
}
	 date {
      match => [ "date", "ISO8601" ]
	  target=> "@timestamp"
	 }
5

It works now then how do I change the timezone in my area? Because it's still using global time. Thank you

6

Time zones are here
ES and LS are based on UTC. Bu default, Kibana show data based on browser TimeZone.

  date {
      match => [ "date", "ISO8601" ]
      timezone => "Asia/Bangkok"
      target=> "@timestamp"
  }
7

Hi, i have problem with date filter, i run it on windows and it's work perfectly. But when i try in linux environtment it's doesn't work. And doesn't send any data to elastic. Please help

8

Can you dump data from ruby debug on Linux? Put as text, not as pic.

9

it's working now i change

match => [ "date", "ISO8601" ]
to
match => [ "date", "yyyy-MM-dd'T'HH:mm:ss.SSSS" ]

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.