Logstash 2.1.3 on CentOS 7.
I'm sending logs to ELK in syslog format (for now, though will switch to JSON at some point) from rsyslog. I'm ingesting some nginx access logs in NCSA combined format.
I do still see the log in Kibana, in this example, syslog_message is set to:
10.0.0.1 - - [07/Apr/2016:14:08:48 +0000] "GET
/foo/bar?access_token=XYZNYD&client=6
HTTP/1.1" 200 1574 "-" "-"
and syslog_program is nginx-access.
I tested my rule against %{COMBINEDAPACHELOG} at http://grokconstructor.appspot.com/do/match#result (using the value of syslog_message from what I see in Kibana). However, the log doesn't show up with the additional fields, making me think it's not getting parsed correctly.
# cat 05_filter.conf
filter {
if [type] == "syslog" {
grok {
match => { "message" => "(?m)<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
# cat 12_nginx_filter.conf filter {
if [syslog_program] == "nginx-access" {
grok {
match => { "syslog_message" => "%{COMBINEDAPACHELOG}" }
remove_tag => ["_grokparsefailure"]
add_tag => ["nginx_access"]
}
geoip {
source => "clientip"
}
}
}
# cat 99_output.conf
output {
# See http://kartar.net/2014/09/when-logstash-and-syslog-go-wrong/
# but basically, give us a way to easily find / analyze messages we
# aren't parsing.
if [type] == "syslog" and "_grokparsefailure" in [tags] {
file { path => "/var/log/logstash/failed_syslog_events" }
}
elasticsearch {
hosts => ["XXXX:9200"]
sniffing => true
}
}
Separately, if I use syntax closer to the example here:
https://miteshshah.github.io/linux/elk/how-to-monitor-nginx-logs-on-elk-stack/, when an access log (vs. an error log) comes in, the first grok seems to match, but then the second results in a _grokparsefailure; that's why I changed the conditionals a bit.