# Parse lvmeng WAF log

**URL:** <https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073>\
**Category:** Logstash\
**Created:** [August 11, 2021, 1:25pm UTC](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073 "2021-08-11T13:25:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![star\_cool](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/star_cool/32/88697_2.png) [@star\_cool](https://discuss.elastic.co/u/star_cool)\
**Post date:** [August 11, 2021, 1:25pm UTC](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073/1 "2021-08-11T13:25:53Z")

</div>

hi

```auto
<11>Aug 11 19:14:55 localhost waf: tag:waf_log_websec site_id:1521041933 protect_id:2623109843 dst_ip:x.x.x.x dst_port:80 src_ip:183.209.173.69 src_port:17506 method:GET domain:www.domain.com uri:/%5fupload/tpl/01/57/343/template343/css/ajax%2dloader.gif alertlevel:HIGH event_type:Info_Leak stat_time:2021-08-11 19:14:52 policy_id:3407873 rule_id:0 action:Block block:No block_info:None http:HTTP/1.1 404 Not Found\\rServer: ****** \\rDate: Wed, 11 Aug 2021 11:14:48 GMT\\rContent-Type: text/html; charset=iso-8859-1\\rTransfer-Encoding: chunked\\rConnection: keep-alive\\rX-Frame-Options: SAMEORIGIN\\rFrame-Options: SAMEORIGIN\\rContent-Encoding: gzip\\r\\rac\\r\\u001F\\x8B\\b\\u0004\\u0003M\\x8E\\xBD\\u000E\\x820\\u0014\\x85\\xF7>?].\\u001A?\\x83\\u0002\\x91\\u0004\\x91\\x9828bzMI\\x90\\\"-\\u001A\\xDF\\xDE\\u0002\\x8B\\xE3\\xF9\\xFBr\\xF8&\\xB9\\u001C\\xE5\\xADJ\\xE1$\\xCF\\u0005T\\xF5\\xA1?\\u0010l\\u0011\\xF3Tf\\x88\\x89L\\xD6d\\u001FF\\x88i\\u0019\\b?{v\\x82kj\\x94\\u0017\\xAEu\\u001D\\x898\\x8A\\xA14\\u000E23\\xF5\\x8A\\xE3j2\\x8EK\\x89?\\xFAλ\\x9D\\xF8\\xEBx\\xC5\\xF8 \\xA4&\\u0018\\xE95\\x91u\\xA4\\xA0\\xBE\\u0016\\xF0i,\\xF4\\x9E\\xF5\\x98Y`zp\\xBA\\xB5`i|\\xD3\\u0018r\\u001C\\xFC\\f\\u0017\\xA2\\xC7\\xCFO\\xD8\\u000F\\xF3|\\u0015<\\xC4\\r0 alertinfo:None proxy_info:None characters:None count_num:1 protocol_type:HTTP wci:None wsi:None country:China correlation_id:6995129440198811088 site_name:vm-7.0 vsite_name:www \\n"

```

use

```auto
input {
  syslog {
    port => 5044
# codec => "json"
  }
}

filter {

      json {
        source => "message"
      }

# kv {
  # source => "message"
 # value_split => ":"
 # }
}

```

got error

```auto
 :exception=>#<LogStash::Json::ParserError: Unrecognized token 'tag': was expecting 'null', 'true', 'false' or NaN
 at [Source: (byte[])"tag:waf_log_webaccess site_id:1521041933 protect_id:2574925061 stat_time:2021-08-11 21:19:03 alertlevel:LOW

```

how to remove  
\<11\>Aug 11 19:14:55 localhost waf:  
and fix json error  
thanks

use kv maybe not work

```auto
 /usr/share/logstash/bin/logstash -e 'input{stdin{}}filter{kv{source => "message" value_split => ":"}}output{stdout{codec=>rubydebug}}'

```

```auto
tag:waf_log_websec site_id:1521041933 protect_id:2623109843 dst_ip:x.x.x.x dst_port:80 src_ip:183.209.173.69 src_port:17506 method:GET domain:www.domain.com uri:/%5fupload/tpl/01/57/343/template343/css/ajax%2dloader.gif alertlevel:HIGH event_type:Info_Leak stat_time:2021-08-11 19:14:52 policy_id:3407873 rule_id:0 action:Block block:No block_info:None http:HTTP/1.1 404 Not Found\\rServer: ****** \\rDate: Wed, 11 Aug 2021 11:14:48 GMT\\rContent-Type: text/html; { "alertlevel" => "HIGH", "host" => "localhost.localdomain", "block" => "No", "stat_time" => "2021-08-11", "domain" => "www.domain.com",
               "policy_id" => "3407873",
                  "action" => "Block",
                      "11" => "14:48",
                     "tag" => "waf_log_websec",
                "src_port" => "17506",
                    "http" => "HTTP/1.1",
                     "uri" => "/%5fupload/tpl/01/57/343/template343/css/ajax%2dloader.gif",
                "dst_port" => "80",
                  "dst_ip" => "x.x.x.x",
                  "src_ip" => "183.209.173.69",
              "protect_id" => "2623109843",
              "event_type" => "Info_Leak",
                 "message" => "tag:waf_log_websec site_id:1521041933 protect_id:2623109843 dst_ip:x.x.x.x dst_port:80 src_ip:183.209.173.69 src_port:17506 method:GET domain:www.domain.com uri:/%5fupload/tpl/01/57/343/template343/css/ajax%2dloader.gif alertlevel:HIGH event_type:Info_Leak stat_time:2021-08-11 19:14:52 policy_id:3407873 rule_id:0 action:Block block:No block_info:None http:HTTP/1.1 404 Not Found\\\\rServer: ****** \\\\rDate: Wed, 11 Aug 2021 11:14:48 GMT\\\\rContent-Type: text/html;",
        "Found\\\\rServer" => " ****** \\\\rDate:",
                 "rule_id" => "0",
                  "method" => "GET",
              "@timestamp" => 2021-08-11T14:09:56.192Z,
    "GMT\\\\rContent-Type" => "text/html;",
                      "19" => "14:52",
                 "site_id" => "1521041933",
                "@version" => "1",
              "block_info" => "None"
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 11, 2021, 3:56pm UTC](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073/2 "2021-08-11T15:56:15Z")

</div>

The log format is not JSON, so I would just remove the json filter.

---

<div class="post-metadata">

**Author:** ![star\_cool](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/star_cool/32/88697_2.png) [@star\_cool](https://discuss.elastic.co/u/star_cool)\
**Post date:** [August 11, 2021, 11:55pm UTC](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073/3 "2021-08-11T23:55:16Z")

</div>

yes, I use grok to fix it , thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 8, 2021, 11:55pm UTC](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073/4 "2021-09-08T23:55:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.