Logstash NGINX Generic and NAXSI Error pattern parse failure

Elastic Stack Logstash
1

Hi All,

We are using NGINX in combination with NAXSI and just started using Logstash inside an ELK Stack. When NAXSI blocks a specific request it logs this in the NGINX error log. These logs are shipped using Filebeat using Type: "nginx_ws_error" towards our Logstash host. On the Logstash host I am trying to parse the "generic" nginx error messages as well as the specific error messages generated by NAXSI.

The NGINX error messages are (other base URL inserted):

NGINX NAXSI

2016/05/05 09:58:23 [error] 765#765: *84260 NAXSI_FMT: ip=89.99.71.163&server=www.mydomain.com&uri=/api&learning=1&vers=0.54&total_processed=26&total_blocked=15&block=0&zone0=BODY&id0=2000&var_name0=action, client: 89.99.71.163, server: www.mydomain.com, request: "POST /api HTTP/1.1", host: "www.mydomain.com"

NGINX Generic

2016/05/05 06:50:16 [error] 7717#7717: *16536 "/var/www/mydomain/whatever/index.html" is not found (2: No such file or directory), client: 89.99.71.163, server: www.mydomain.com request: "GET /whatever/ HTTP/1.1", host: "www.mydomain.com"

To do this I am using the following basic configuration:

Logstash Filter for NGINX Error

filter {
if [type] == "nginx_ws_error" {
grok {
match => { "message" => "%{NGINX_ERROR}" }
}
}
}

Logstash Patterns

NGINX NAXSI

NGINX_ERROR %{DATESTAMP:messagetimestamp} [%{LOGLEVEL:severity}] (%{NUMBER:pid:int}#%{NUMBER}: *%{NUMBER}|*%{NUMBER}) %{WORD:naxsi_fmt}: ip=%{IP:client_ip}&server=%{IPORHOST:http_host}&uri=%{PATH:http_uri}&learning=%{WORD:naxsi_learning}&vers=%{NUMBER:naxsi_version}&total_processed=%{NUMBER:naxsi_total_processed}&total_blocked=%{NUMBER:naxsi_total_blocked}&block=%{NUMBER:naxsi_block}&zone0=%{WORD:naxsi_zone}&id0=%{NUMBER:naxsi_main_rule_id}&var_name0=%{WORD:naxsi_http_variable}, %{GREEDYDATA:naxsi_error_message}

NGINX Generic

NGINX_ERROR %{DATESTAMP:messagetimestamp} [%{LOGLEVEL:severity}] (%{NUMBER:pid:int}#%{NUMBER}: *%{NUMBER}|*%{NUMBER}) %{DATA:nginx_error_message}, client: %{IPORHOST:client_ip}, server: %{IPORHOST:http_server}, request: "%{WORD:http_verb} %{DATA:http_uri} HTTP/%{NUMBER:http_version}", host: "%{DATA:http_host}"

The results of this configuration are:

  • When configuring only the first pattern (NGINX NAXSI), Logstash parses this correctly (apart from not setting the timestamp correctly, but that is for another troubleshooting session)
  • When adding the second pattern (NGINX Generic), Logstash parses NGINX generic error messages correctly, but fails to parse any NGINX NAXSI log messages correctly.

So, the NGINX NAXSI pattern is no longer matched, and the NGINX Generic patterns is used to parse both types of NGINX Error messages, leading to a grok parse failure for NGINX NAXSI errors. I am fairly new to Grok and the ELK stack. I do not understand why the system would use the NGINX Generic pattern to parse the NGINX NAXSI log file when there is an "exact" pattern available to match on? Any explanations or pointers are very welcome, thanks!

Cheers,
Reinier

2

So, further testing reveals the following:

When switching positions of the patterns inside the nginx pattern file for Logstash the grok parse failure moves from NGINX NAXSI to the NGINX Generic. Meaning, Both types of log entry are now being parsed by the last pattern in the file, which now is my pattern for NGINX NAXSI.

Placing the now last pattern for NGINX NAXSI into a separate file by itself leads to the former situation, where the NGINX Generic pattern is used to parse both types of log entry, leading to a grok parse failure for NGINX NAXSI logs, while correctly parsing NGINX Generic logs.

3

Decided to solve this using Filebeat Include and Exclude under two separate collectors and document_types, combined with separate Logstash patterns for each type of log entry...

4