Hi team,
I'm using filebeat 6.2.4 to parse ModSecurity logs in Json format, so far so good using the following config:
filebeat.prospectors:
- type: log
paths:
- /var/log/httpd/modsec_audit.log
json.keys_under_root: true
json.add_error_key: true
output.elasticsearch:
hosts: ["192.168.1.128:9200"]
This is my sample log:
{"body":["j_username=%3Cscript%3Ealert%28%27xss%27%29%3C%2Fscript%3E&j_password=*******&next_url=%2Faction%2Fmy-portal"]},"response":{"protocol":"HTTP/1.1","status":403,"headers":{"X-Frame-Options":"SAMEORIGIN","Strict-Transport-Security":"max-age=3600","X-Xss-Protection":"1; mode=block","X-Content-Type-Options":"nosniff","X-Permitted-Cross-Domain-Policies":"none","Last-Modified":"Wed, 04 Apr 2018 18:40:51 GMT","ETag":"\"47d-5690a298d6a06\"","Accept-Ranges":"bytes","Content-Length":"1149","Keep-Alive":"timeout=5, max=100","Connection":"Keep-Alive","Content-Type":"text/html"}},"audit_data":{"messages":["Warning. detected XSS using libinjection. [file \"/etc/httpd/modsecurity/owasp-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf\"] [line \"64\"] [id \"941100\"] [rev \"2\"] [msg \"XSS Attack Detected via libinjection\"] [data \"Matched Data: upgrade-insecure-requests found within ARGS:j_username: <script>alert('xss')</script>\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.0.0\"] [maturity \"1\"] [accuracy \"9\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-xss\"] [tag \"OWASP_CRS/WEB_ATTACK/XSS\"] [tag \"WASCTC/WASC-8\"] [tag \"WASCTC/WASC-22\"] [tag \"OWASP_TOP_10/A3\"] [tag \"OWASP_AppSensor/IE1\"] [tag \"CAPEC-242\"]","Warning. Pattern match \"(?i)([<\\xef\\xbc\\x9c]script[^>\\xef\\xbc\\x9e]*[>\\xef\\xbc\\x9e][\\\\s\\\\S]*?)\" at ARGS:j_username. [file \"/etc/httpd/modsecurity/owasp-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf\"] [line \"99\"] [id \"941110\"] [rev \"2\"] [msg \"XSS Filter - Category 1: Script Tag Vector\"] [data \"Matched Data: <script> found within ARGS:j_username: <script>alert('xss')</script>\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.0.0\"] [maturity \"4\"] [accuracy \"9\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-xss\"] [tag \"OWASP_CRS/WEB_ATTACK/XSS\"] [tag \"WASCTC/WASC-8\"] [tag \"WASCTC/WASC-22\"] [tag \"OWASP_TOP_10/A3\"] [tag \"OWASP_AppSensor/IE1\"] [tag \"CAPEC-242\"]","Warning. Pattern match \"(?i)<[^\\\\w<>]*(?:[^<>\\\"'\\\\s]*:)?[^\\\\w<>]*(?:\\\\W*?s\\\\W*?c\\\\W*?r\\\\W*?i\\\\W*?p\\\\W*?t|\\\\W*?f\\\\W*?o\\\\W*?r\\\\W*?m|\\\\W*?s\\\\W*?t\\\\W*?y\\\\W*?l\\\\W*?e|\\\\W*?s\\\\W*?v\\\\W*?g|\\\\W*?m\\\\W*?a\\\\W*?r\\\\W*?q\\\\W*?u\\\\W*?e\\\\W*?e|(?:\\\\W*?l\\\\W*?i\\\\W*?n\\\\W*?k|\\\\W*?o\\\\W*?b\\\\W*?j\\\\W*?e\\ ...\" at ARGS:j_username. [file \"/etc/httpd/modsecurity/owasp-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf\"] [line \"236\"] [id \"941160\"] [rev \"2\"] [msg \"NoScript XSS InjectionChecker: HTML Injection\"] [data \"Matched Data: <script found within ARGS:j_username: <script>alert('xss')</script>\"] [severity \"CRITICAL\"] [ver \"OWASP_CRS/3.0.0\"] [maturity \"1\"] [accuracy \"8\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-xss\"] [tag \"OWASP_CRS/WEB_ATTACK/XSS\"] [tag \"WASCTC/WASC-8\"] [tag \"WASCTC/WASC-22\"] [tag \"OWASP_TOP_10/A3\"] [tag \"OWASP_AppSensor/IE1\"] [tag \"CAPEC-242\"]","Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file \"/etc/httpd/modsecurity/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"57\"] [id \"949110\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 15)\"] [severity \"CRITICAL\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-generic\"]","Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file \"/etc/httpd/modsecurity/owasp-crs/rules/RESPONSE-980-CORRELATION.conf\"] [line \"73\"] [id \"980130\"] [msg \"Inbound Anomaly Score Exceeded (Total Inbound Score: 15 - SQLI=0,XSS=15,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): NoScript XSS InjectionChecker: HTML Injection\"] [tag \"event-correlation\"]"],"
When filebeat gets audit_data.messages he gets the entire message as a single field, is there any way I can get sub fields in the audit_data.messages like id or msg so I can filter in Kibana with thoose fields?
Hope you can help me!