# Modsecurity Json log parsing

**URL:** <https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807>\
**Category:** Logstash\
**Created:** [March 19, 2021, 12:00pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807 "2021-03-19T12:00:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mirko.spezie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mirko.spezie/32/83705_2.png) [@mirko.spezie](https://discuss.elastic.co/u/mirko.spezie)\
**Post date:** [March 19, 2021, 12:00pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807/1 "2021-03-19T12:00:28Z")

</div>

Hi,  
I'm trying to parse with no success the modsecurity json log. This is the output:

```auto
    {
      "_index": "modsecurity-2021.03.17",
      "_type": "_doc",
      "_id": "oQRWQngB6w34lfYfAQp6",
      "_version": 1,
      "_score": null,
      "_source": {
        "@version": "1",
        "path": "none",
        "tags": [],
        "@timestamp": "2021-03-17T22:37:15.747Z",
        "host": " **************",
        "transaction": {
          "messages": [
            {
              "message": "HTTP Parameter Pollution (ARGS_NAMES)",
              "details": {
                "data": "Matched Data: TX:paramcounter_ARGS_NAMES found within MATCHED_VARS_NAMES:TX:paramcounter_ARGS_NAMES: TX:paramcounter_ARGS_NAMES",
                "ver": "OWASP_CRS/3.3.0",
                "maturity": "0",
                "rev": "",
                "accuracy": "0",
                "match": "Matched \"Operator `Rx' with parameter `TX:paramcounter_(.*)' against variable `MATCHED_VARS_NAMES:TX:paramcounter_ARGS_NAMES' (Value: `TX:paramcounter_ARGS_NAMES' )",
                "ruleId": "921180",
                "file": "/usr/share/modsecurity-crs/rules/REQUEST-921-PROTOCOL-ATTACK.conf",
                "reference": "o0,26o16,10v544,26",
                "lineNumber": "330",
                "tags": [
                  "application-multi",
                  "language-multi",
                  "platform-multi",
                  "attack-protocol",
                  "OWASP_CRS",
                  "capec/1000/152/137/15/460",
                  "paranoia-level/3"
                ],
                "severity": "2"
              }
            },
            {
              "message": "Inbound Anomaly Score Exceeded (Total Score: 5)",
              "details": {
                "data": "",
                "ver": "OWASP_CRS/3.3.0",
                "maturity": "0",
                "rev": "",
                "accuracy": "0",
                "match": "Matched \"Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' )",
                "ruleId": "949110",
                "file": "/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf",
                "reference": "",
                "lineNumber": "138",
                "tags": [
                  "application-multi",
                  "language-multi",
                  "platform-multi",
                  "attack-generic"
                ],
                "severity": "2"
              }
            },
            {
              "message": "The Application Returned a 500-Level Status Code",
              "details": {
                "data": "Matched Data: 503 found within RESPONSE_STATUS",
                "ver": "OWASP_CRS/3.3.0",
                "maturity": "0",
                "rev": "",
                "accuracy": "0",
                "match": "Matched \"Operator `Rx' with parameter `^5\\d{2}$' against variable `RESPONSE_STATUS' (Value: `503' )",
                "ruleId": "950100",
                "file": "/usr/share/modsecurity-crs/rules/RESPONSE-950-DATA-LEAKAGES.conf",
                "reference": "o0,3v544,3v544,3",
                "lineNumber": "96",
                "tags": [
                  "application-multi",
                  "language-multi",
                  "platform-multi",
                  "attack-disclosure",
                  "PCI/6.5.6",
                  "paranoia-level/2",
                  "OWASP_CRS",
                  "capec/1000/152"
                ],
                "severity": "3"
              }
            },
            {
              "message": "Outbound Anomaly Score Exceeded (Total Score: 4)",
              "details": {
                "data": "",
                "ver": "OWASP_CRS/3.3.0",
                "maturity": "0",
                "rev": "",
                "accuracy": "0",
                "match": "Matched \"Operator `Ge' with parameter `4' against variable `TX:OUTBOUND_ANOMALY_SCORE' (Value: `4' )",
                "ruleId": "959100",
                "file": "/usr/share/modsecurity-crs/rules/RESPONSE-959-BLOCKING-EVALUATION.conf",
                "reference": "",
                "lineNumber": "128",
                "tags": [
                  "anomaly-evaluation"
                ],
                "severity": "0"
              }
            }
          ],
          "host_port": 80,
          "unique_id": "1615466172",
          "server_id": "30f8d8e5499258cee7faa1a3490f04a35729b37c",
          "request": {
            "method": "GET",
            "uri": "/o/frontend-css-web/main.css?browserId=firefox&themeId=classic_WAR_classictheme&minifierType=css&languageId=en_US&b=7102&t=1546901591742",
            "headers": {
              "Host": " *******************",
              "Accept": "text/css,*/*;q=0.1",
              "Connection": "keep-alive",
              "Referer": "http:// ********************* /",
              "Accept-Encoding": "gzip, deflate",
              "Accept-Language": "it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3",
              "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0",
              "Cookie": "JSESSIONID=3C9F859C4DBC0C73164DBDE0CF202969; COOKIE_SUPPORT=true; GUEST_LANGUAGE_ID=en_US"
            },
            "http_version": 1.1
          },
          "response": {
            "headers": {
              "Date": "Thu, 11 Mar 2021 12:36:12 GMT",
              "Connection": "keep-alive",
              "Server": "nginx/1.18.0",
              "Content-Length": "206",
              "Content-Type": "text/html"
            },
            "body": "<html>\r\n<head><title>503 Service Temporarily Unavailable</title></head>\r\n<body>\r\n<center><h1>503 Service Temporarily Unavailable</h1></center>\r\n<hr><center>nginx/1.18.0 (Ubuntu)</center>\r\n</body>\r\n</html>\r\n",
            "http_code": 403
          },
          "client_port": 64900,
          "producer": {
            "components": [
              "OWASP_CRS/3.3.0\""
            ],
            "secrules_engine": "Enabled",
            "modsecurity": "ModSecurity v3.0.4 (Linux)",
            "connector": "ModSecurity-nginx v1.0.1"
          },
          "client_ip": " ****************",
          "time_stamp": "Thu Mar 11 13:36:12 2021",
          "host_ip": " ******************"
        }
      },
      "fields": {
        "@timestamp": [
          "2021-03-17T22:37:15.747Z"
        ]
      },
      "sort": [
        1616020635747
      ]
    }

```

The problem is related to what's inside the square brackets "messages: " : I've no idea how to manage this array. I've read another post about using the foreach ingest processor, but I haven't found a way to solve this thing.

This is the output in kibana

 ![immagine](https://us1.discourse-cdn.com/elastic/original/3X/2/c/2cb05af4b790df1f6b39cafff9335a32cac0d382.png)

Any idea?  
thanks

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 19, 2021, 12:14pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807/2 "2021-03-19T12:14:35Z")

</div>

What are you trying to parse or solve in the array?

---

<div class="post-metadata">

**Author:** ![mirko.spezie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mirko.spezie/32/83705_2.png) [@mirko.spezie](https://discuss.elastic.co/u/mirko.spezie)\
**Post date:** [March 19, 2021, 1:13pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807/3 "2021-03-19T13:13:12Z")

</div>

Most part of json is parsed automaticallyin elastic, expect for the array inside messages. I'd like to parse every field inside (or most of them)

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 19, 2021, 1:18pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807/4 "2021-03-19T13:18:36Z")

</div>

If you can post your Logstash configuration and an example of what your expected message out would look like I can help.

But I think you are looking to use the [split filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html).

---

<div class="post-metadata">

**Author:** ![mirko.spezie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mirko.spezie/32/83705_2.png) [@mirko.spezie](https://discuss.elastic.co/u/mirko.spezie)\
**Post date:** [March 19, 2021, 1:35pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807/5 "2021-03-19T13:35:01Z")

</div>

The logstash is very simple: a beat input, a filter for geoip and the output. This because the log arrives already in json format.  
you saved my day 🙂 It works  
thank you

Edit:  
Now I've this WARN in logstash log:

```
[2021-03-19T14:37:14,881][WARN][logstash.codecs.jsonlines][main][f9bbb66e1ecfd41dbc01d5d56a4f4173af5884bc3cb6298b39a534f98400012b] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected character ('.' (code 46)): Expected space separating root-level values at [Source: (String)"*.*.*.* - - [19/Mar/2021:14:36:03 +0100] "POST /api/jsonws/invoke HTTP/1.1" 503 4260 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36""; line: 1, column: 8]>, :data=>"*.*.*.* - - [19/Mar/2021:14:36:03 +0100] \"POST /api/jsonws/invoke HTTP/1.1\" 503 4260 \"-\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36\""}
[2021-03-19T14:37:14,992][WARN][logstash.filters.split][main][15bc70c2480e7c3b6a8e3d8f65817a878a227d71c29af9b490ca5c161b049831] Only String and Array types are splittable. field:[transaction][messages] is of type = NilClass
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 16, 2021, 1:36pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807/6 "2021-04-16T13:36:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.