Create two timestamp

Elastic Stack Logstash
1

I have following parse log and it has 2 types of log time and I need to build 2 timestamp using them.

{
"log_timestamp" => 2018-03-21T01:29:45.000Z,
         "year" => "2018",
          "day" => "Wed",
     "monthday" => "21",     
   "@timestamp" => 2018-03-21T06:59:45.482Z,
        "month" => "Mar",
         "time" => "06:59:45",
}

I tried to build a separate timestamp using "year", "month", "day" and "time" as below.

mutate {
	    add_field => ["trigger_timestamp", "%{year}-%{month}-%{day} %{time}"]
}

Finally I need to format that "trigger_timestamp in to standard ISO8601 time and I do following format configuration.

 date {
	match => [ "trigger_timestamp", "YYYY-MM-dd HH:mm:ss", "YYYY-MM-dd HH:mm:ss.ZZZ" ]
	target => "trigger_timestamp"
}

And my full filter script is as below after grok pattern match.

   mutate {
	    add_field => ["trigger_timestamp", "%{year}-%{month}-%{day} %{time}"]
	}
	
	date {
		match => [ "trigger_timestamp", "YYYY-MM-dd HH:mm:ss", "YYYY-MM-dd HH:mm:ss.ZZZ" ]
		target => "trigger_timestamp"
	}
	
	date {
		match => [ "log_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
		target => "log_timestamp"
	}

When I execute above script I received following error.

[2018-03-21T13:41:36,666][ERROR][logstash.filters.date    ] Invalid setting for date

filter plugin:

filter {
date {
  # This setting must be a string
  # Expected string, got ["log_timestamp", "trigger_timestamp"]
  target => ["log_timestamp", "trigger_timestamp"]
  ...
}
}
[2018-03-21T13:41:36,687][ERROR][logstash.agent           ] Cannot load an invalid 
configuration {:reason=>"Something is wrong with your configuration."}
[

Is this raise due to 2 date timestamps format config ?. Any idea to keep these 2 timestamp and fix this issue

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.