Parse Modsecurity json logs

Elastic Stack Logstash
1

Hi team,
First of all thanks for this awesome tool. I have a problem:
I'm trying to send Modsecurity's (JSON) logs to Elasticsearch through Filebeat and Logstash.

Log example:

{
  ...
   "audit_data":{
      "messages":[
         "Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:username: test@test.it' OR 1=1 -- \"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"]",
         "Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:password: password' OR 1 = 1 -- \"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"]",
         "Warning. Operator GE matched 5 at TX:anomaly_score. [file \"/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"91\"] [id \"949110\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 10)\"] [severity \"CRITICAL\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-generic\"]",
         "Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file \"/usr/share/modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf\"] [line \"86\"] [id \"980130\"] [msg \"Inbound Anomaly Score Exceeded (Total Inbound Score: 10 - SQLI=10,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 10, 0, 0, 0\"] [tag \"event-correlation\"]"
      ],
      "error_messages":[
         "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:username: test@test.it' OR 1=1 -- \"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/ajax/userManager.php\"] [unique_id \"XVgURKQ96dKvEiKPlPz8iAAAAAo\"]",
         "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:password: password' OR 1 = 1 -- \"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/ajax/userManager.php\"] [unique_id \"XVgURKQ96dKvEiKPlPz8iAAAAAo\"]",
         "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:anomaly_score. [file \"/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"91\"] [id \"949110\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 10)\"] [severity \"CRITICAL\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-generic\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/ajax/userManager.php\"] [unique_id \"XVgURKQ96dKvEiKPlPz8iAAAAAo\"]",
         "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file \"/usr/share/modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf\"] [line \"86\"] [id \"980130\"] [msg \"Inbound Anomaly Score Exceeded (Total Inbound Score: 10 - SQLI=10,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 10, 0, 0, 0\"] [tag \"event-correlation\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/ajax/userManager.php\"] [unique_id \"XVgURKQ96dKvEiKPlPz8iAAAAAo\"]"
      ],
      "handler":"application/x-httpd-php",
      "stopwatch":{
         "p1":414,
         "p2":4877,
         "p3":90,
         "p4":159,
         "p5":255,
         "sr":8,
         "sw":0,
         "l":0,
         "gc":0
      },
      "response_body_dechunked":true,
      "producer":[
         "ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/)",
         "OWASP_CRS/3.1.0"
      ],
      "server":"Apache/2.4.29 (Ubuntu)",
      "engine_mode":"DETECTION_ONLY"
   }
}

now, i tryed to get for each audit_data.messages throught the following filter:

filter {
	grok {
    match => { "audit_data.messages" => "ARGS:%{DATA}: %{DATA:m_data} \\\"]"}
    tag_on_failure => ["on_no_grok_fail"]
  }
}

but it fails every time...
What i've done wrong?

thanks in advance for any help guys!

2

You do not have an actual backslash in the string you are matching, it is escaping the double quote.

match => { "[audit_data][messages]" => 'ARGS:%{DATA}: %{GREEDYDATA:m_data} "]'}

will get you

    "m_data" => [
    [0] "test@test.it' OR 1=1 --",
    [1] "password' OR 1 = 1 --"
],
3

Thank you very much!

4

Thank you so much for answering;

I found that with this input it doesn't work:

"audit_data" => {
                    "engine_mode" => "DETECTION_ONLY",
                        "handler" => "application/x-httpd-php",
        "response_body_dechunked" => true,
                         "server" => "Apache/2.4.29 (Ubuntu)",
                       "messages" => [
            [0] "Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:prev: /tecnologie-web/Progetto/index.php?q=pass' OR 1 = 1 --\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"]",
            [1] "Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:query: pass2' AND 1=1 --\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"]",
            [2] "Warning. Operator GE matched 5 at TX:anomaly_score. [file \"/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"91\"] [id \"949110\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 10)\"] [severity \"CRITICAL\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-generic\"]",
            [3] "Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file \"/usr/share/modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf\"] [line \"86\"] [id \"980130\"] [msg \"Inbound Anomaly Score Exceeded (Total Inbound Score: 10 - SQLI=10,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 10, 0, 0, 0\"] [tag \"event-correlation\"]"
        ],
                 "error_messages" => [
            [0] "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:prev: /tecnologie-web/Progetto/index.php?q=pass' OR 1 = 1 --\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/user/login.php\"] [unique_id \"XVhIX3EWRd4AC4oFQvJ3dQAAAAA\"]",
            [1] "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. detected SQLi using libinjection with fingerprint 's&1c' [file \"/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf\"] [line \"66\"] [id \"942100\"] [msg \"SQL Injection Attack Detected via libinjection\"] [data \"Matched Data: s&1c found within ARGS:query: pass2' AND 1=1 --\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.1.0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-sqli\"] [tag \"OWASP_CRS/WEB_ATTACK/SQL_INJECTION\"] [tag \"WASCTC/WASC-19\"] [tag \"OWASP_TOP_10/A1\"] [tag \"OWASP_AppSensor/CIE1\"] [tag \"PCI/6.5.2\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/user/login.php\"] [unique_id \"XVhIX3EWRd4AC4oFQvJ3dQAAAAA\"]",
            [2] "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:anomaly_score. [file \"/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"91\"] [id \"949110\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 10)\"] [severity \"CRITICAL\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-generic\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/user/login.php\"] [unique_id \"XVhIX3EWRd4AC4oFQvJ3dQAAAAA\"]",
            [3] "[file \"apache2_util.c\"] [line 273] [level 3] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file \"/usr/share/modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf\"] [line \"86\"] [id \"980130\"] [msg \"Inbound Anomaly Score Exceeded (Total Inbound Score: 10 - SQLI=10,XSS=0,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 10, 0, 0, 0\"] [tag \"event-correlation\"] [hostname \"localhost\"] [uri \"/tecnologie-web/Progetto/php/user/login.php\"] [unique_id \"XVhIX3EWRd4AC4oFQvJ3dQAAAAA\"]"
 ],

how can I make it work in both cases?
Thanks in advance again

5

Remove the space at the end of the pattern and change GREEDYDATA to DATA

match => { "[audit_data][messages]" => 'ARGS:%{DATA}: %{DATA:m_data}"]'}

gets you

    "m_data" => [
    [0] "test@test.it' OR 1=1 -- ",
    [1] "password' OR 1 = 1 -- "
]

and

    "m_data" => [
    [0] "/tecnologie-web/Progetto/index.php?q=pass' OR 1 = 1 --",
    [1] "pass2' AND 1=1 --"
]
6

Sorry for the delay in answering.
Thank you very much and good job

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.