Parse specific field when JSON filter failed

Elastic Stack Logstash
1

I have some logs in JSON format with some offending key-value pairs that cause _jsonparsefailure. I wanted to leave the log as is and just parse the timestamp. I tried the below config but failed to parse the timestamp field. I don't even see the ts1 field in results. Please advise.

filter {
	json {
		source => "message"
	}	
	date {
		match => ["ts","YYYY-MM-dd'T'HH:mm:ss.SSSZ","ISO8601"]
		target => "@timestamp"
	}

	if "_jsonparsefailure" in [tags] { 
		mutate {
			add_field => { "ts1" => "" }
        }
		ruby {
			code => '
				t = event.get("[ts]")
				event.set("[ts1]", t)
			'
		}
		date {
			match => ["ts1","YYYY-MM-dd'T'HH:mm:ss.SSSZ","ISO8601"]
			target => "@timestamp"
		}
	
	}
}
2

JSON parsing is all-or-nothing. If it gets an error then a json filter will not add any fields to the event. You might be able to do something like

grok { match => { "message" => '"ts"\s*:\s*"%{TIMESTAMP_ISO8601:ts}"' } }

to extract the ts field.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.