# Problem with parsing json in syslog format

**URL:** <https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042>\
**Category:** Logstash\
**Created:** [October 14, 2020, 12:30pm UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042 "2020-10-14T12:30:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![cyberzlo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cyberzlo/32/65490_2.png) [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\
**Post date:** [October 14, 2020, 12:30pm UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042/1 "2020-10-14T12:30:57Z")

</div>

Hi, can you please help me with logstash config?

```auto
    input {
      #tcp {
      # port => 5014
      # type => syslog
      #}
      udp {
        port => 5014
        type => syslog
      }
    }

    filter {
      if [type] == "syslog" {
        json {
            source => "message"
            #remove_field => "message"
        }

        #date {
        # match => ["timestamp", "UNIX_MS"]
        #}

        #mutate {
        # remove_field => ["host", "path", "_type"]
        #}
      }
    }

    output {
      file {
        path => "/tmp/logstash_%{host}_%{+YYYY-MM-dd}.log"
        codec => rubydebug
      }
      elasticsearch {
        hosts => ["localhost:9200"]
        index => "logstash-syslog-%{+YYYY-MM-dd}"
        manage_template => false
        ilm_enabled => false
      }
    }

```

I got alert to file:

```auto
    {
    "@timestamp" => 2020-10-14T12:22:11.424Z,
          "host" => "192.168.1.1",
      "@version" => "1",
          "tags" => [
        [0] "_jsonparsefailure"
    ],
          "type" => "syslog",
       "message" => "<12>1 2020-10-14T12:22:11.370Z test.local TestServer 1079 - - {\"event_type\":\"Threat_Event\",\"ipv4\":\"192.168.2.2\",\"hostname\":\"Test\",\"source_uuid\":\"504e731a-9e71-48b5-b275-cfaee4f5dab1\",\"occured\":\"14-Oct-2020 12:21:55\",\"severity\":\"Warning\",\"threat_type\":\"potentially unwanted application\",\"threat_name\":\"Win32/WebCompanion.B\",\"threat_flags\":\"Variant\",\"scanner_id\":\"Real-time file system protection\",\"scan_id\":\"virlog.dat\",\"engine_version\":\"22151 (20201014)\",\"object_type\":\"file\",\"object_uri\":\"file:///C:/installer.exe\",\"action_taken\":\"cleaned by deleting\",\"threat_handled\":true,\"need_restart\":false,\"username\":\"DESKTOP-ABCD\\\\DELL\",\"processname\":\"C:\\\\utweb_installer (1).exe\",\"circumstances\":\"Event occurred on a newly created file.\",\"firstseen\":\"14-Oct-2020 12:12:52\",\"hash\":\"2179FD861CB63D4B627AEC617\"}\n"
}

```

Some logs also looks like:

```auto
       "message" => [
        [0] "<12>1 2020-10-14T12:22:11.370Z test.local TestServer 1079 - - {\"event_type\"[...]\"}\n",
        [1] "{\"event_type\"[...]\"}\n"
    ]

```

^ Have one syslog-like intro and then 2 (or more?) jsons included.

Best would be if also timestamp can be taken from "occured", however even if I will be able just parse this will be great, please help me because after reading documentation I still don't have idea how fix this :(...

This is syslog format however there is JSON included in this syslog event. As alternative I can select LEEF format, but I think JSON is better for elastic?

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [October 14, 2020, 2:20pm UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042/2 "2020-10-14T14:20:34Z")

</div>

Here is an example [Parse JSON string contained in a Syslog message](https://discuss.elastic.co/t/parse-json-string-contained-in-a-syslog-message/54951)

This uses grok to parse off the syslog headers before using the json filter

```
    if [type] == "syslog" {
            grok {
                    match => { "message" => "%{SYSLOGBASE} %{GREEDYDATA:syslog_message}" }
            }

            json {
                    source => { source => "syslog_message" }
            }
    }
```

---

<div class="post-metadata">

**Author:** ![cyberzlo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cyberzlo/32/65490_2.png) [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\
**Post date:** [October 17, 2020, 11:53pm UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042/3 "2020-10-17T23:53:12Z")

</div>

Thanks but unfortunately looks like this is not common syslog ☹ i.a. because this `<12>` etc numbers at the begning.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 18, 2020, 12:49am UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042/4 "2020-10-18T00:49:44Z")

</div>

That could be a [syslog](https://tools.ietf.org/html/rfc3164) priority.

I would use a dissect filter on that.

---

<div class="post-metadata">

**Author:** ![cyberzlo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cyberzlo/32/65490_2.png) [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\
**Post date:** [October 21, 2020, 1:16am UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042/5 "2020-10-21T01:16:34Z")

</div>

But how? can you please elaborate more :)? I have no idea how deal with it...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 18, 2020, 1:16am UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042/6 "2020-11-18T01:16:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.