Hey guys!
I'm receiving Logstash parsing error for this JSON -
{
"time_date": "2019-09-11T07:05:32+00:00",
"client": "22.222.2.222",
"host": "xxxx.com",
"scheme": "https",
"request_method": "GET",
"request_uri": "/rd/",
"request_id": "f44d35bcf367be11c2804fd2f7d2dd41",
"status": 499,
"upstream_addr": "111.11.111.111:80",
"-": -1,
"request_time": 0.295,
"upstream_response_time": 0.296,
"upstream_connect_time": 0.004,
"upstream_header_time": -1
}
Which is correct JSON.
The error is -
controller.go:276] Endpoints logstash-kube changed, syncing"; line: 1, column: 7]>}
2019-09-11T09:49:06.908134227Z [2019-09-11T09:49:06,907][WARN ][logstash.filters.json ] Error parsing json {:source=>"message", :raw=>"I0911 09:48:20.464574 1 controller.go:276]
That means, that it can't parse the date - raw=>"I0911.
Here is the Logstash config
apiVersion: v1
kind: ConfigMap
metadata:
name: logstash-kube-config
data:
logstash.conf: |-
input {
beats {
port => 5044
}
}
filter {
if [kubernetes][container][name] == "nginx-ingress" {
json {
source => "message"
}
}
else if [kubernetes][container][name] == "nginx" {
grok {
match => {
"message" => "%{IP:remote_ip} - \[%{HTTPDATE:[response][time]}\] \"%{DATA:url}\" %{NUMBER:[response][code]} %{NUMBER:[response][bytes]} %{QS:user_agent}"
}
remove_field => "message"
}
geoip {
source => "remote_ip"
target => "[geoip]"
}
}
else {
drop {}
}
date {
match => ["time", "ISO8601"]
remove_field => ["time"]
}
mutate {
remove_field => ["source", "host", "[beat][name]", "[beat][version]"]
}
}
output {
elasticsearch {
hosts => ["http://yyyy-xxxx-67284-dal10.lb.xxx.net:9200"]
index => "apps-prod-dal10-%{[kubernetes][namespace]}-deployment-%{[kubernetes][container][name]}-%{[kubernetes][replicaset][name]}%{+YYYY.MM.dd}"
}
}
What am I doing wrong here?