Hi,
The logs that I'm receiving for message, looking like that in Kibana -
message {"time_date": "2019-02-14T14:00:39+00:00","client": "10.xxx.xxx.xxx", "host": "xxx.com", "scheme": "https", "request_method": "GET", "request_uri": "/static/img/logo_new.png", "request_id": "xxxxxxxxxxxxxx", "status": 304, "upstream_addr": "xxx.xx.xx.xx:80", "upstream_status": 304, "request_time": 0.002, "upstream_response_time": 0.000, "upstream_connect_time": 0.000, "upstream_header_time": 0.000}
My Logstash ConfigFile -
input {
beats {
port => 50XX
}
}
filter {
if [kubernetes][container][name] == "nginx" {
grok {
match => {
"message" => "%{IP:remote_ip} - \[%{HTTPDATE:[response][time]}\] \"%{DATA:url}\" %{NUMBER:[response][code]} %{NUMBER:[response][bytes]} %{QS:user_agent}"
}
remove_field => "message"
}
geoip {
source => "remote_ip"
target => "[geoip]"
}
}
date {
match => ["time", "ISO8601"]
remove_field => ["time"]
}
mutate {
remove_field => ["source", "host", "[beat][name]", "[beat][version]"]
}
}
output {
elasticsearch {
hosts => ["es-xx-01.xxxx.pro:9200", "es-xx-02.xxxx.pro:9200"]
index => "apps-qa-%{[kubernetes][namespace]}-deployment-%{[kubernetes][pod][name]}-%{[kubernetes][labels][app]}-%{[kubernetes][container][name]}-%{+YYYY.MM.dd}"
}
}
How can I config the Logstash the right way, to geet the Message log parsed?
Thanks,
Aleksei