Hi all,
I am running 3 logstash instances in AWS EC2 with shared configuration (shared via AWS EFS) to balance load between them. Logs are sent by filebeat from the number of hosts.
I've created new logstash pipeline to gather Nginx access logs and see that grok filter actually works on one logstash node only. When the message is processed by one of the other 2 nodes, I don't see any errors or _grokparsefailure tag in the message, but it is not parsed.
Logstash versions are the same between all hosts, OS are Centos 7 with identical kernel version.
Any thoughts where I can look into to make it work on all 3 logstash nodes?
/usr/share/logstash/bin/logstash --version
logstash 6.4.1
Grok filter:
grok {
match => { "message" => "%{COMBINEDAPACHELOG} %{QS:ip_1} %{QS:ip_2} %{QS:ip_3} %{GREEDYDATA:session_id}" }
}
Example of input data:
10.0.0.2 - - [03/Mar/2019:03:45:20 +0000] "POST /my/awesome/service HTTP/1.1" 204 0 "https://my.awesome.sevice/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2)" "192.168.88.10:3355" "192.168.88.11" "192.168.88.11" 0KJSDKJK919191.abcd