I changed the source from access to admin and restarted the filebeat service, when I use grok to filter only the admin source, it is empty, but if using grok to filter only the access, it working. It should be source admin not access, if there any other configuration part I missed? I checked the kibana, the log have field.source = admin
- type: log
enabled: true
paths:
- /etc/nginx/log/admin.log
fields:
source: admin
It is working while
if "access" in [source]{
mutate{
remove_field => ["event.original"]
}
grok {
match => { "message" => "%{IPORHOST:remote_ip} - %{DATA:user_name} \[%{HTTPDATE:time}\] \"%{WORD:method}%{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent:bytes} \"%{DATA:referrer}\" \"%{DATA:agent}\" \"%{NUMBER:request_time}\" \"%{DATA:http_x_forwarded_for}\" %{IPORHOST:http_host} " }
remove_field => "message"
}
}
Not working while
if "admin" in [source]{
mutate{
remove_field => ["event.original"]
}
grok {
match => { "message" => "%{IPORHOST:remote_ip} - %{DATA:user_name} \[%{HTTPDATE:time}\] \"%{WORD:method}%{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent:bytes} \"%{DATA:referrer}\" \"%{DATA:agent}\" \"%{NUMBER:request_time}\" \"%{DATA:http_x_forwarded_for}\" %{IPORHOST:http_host} " }
remove_field => "message"
remove_field => "event.original"
}
}