# Parse rabbitmq json log

**URL:** <https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009>\
**Category:** Logstash\
**Created:** [September 27, 2023, 7:36pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009 "2023-09-27T19:36:39Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\
**Post date:** [September 27, 2023, 7:36pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/1 "2023-09-27T19:36:39Z")

</div>

Hi, got json log message from rabbit as

```auto
{"timestamp":"2022-12-21 03:14:59.977922+02:00","level":"error","msg":"Error on AMQP connection <0.32551.1583>: enotconn (socket is not connected)","domain":"rabbitmq.connection","pid":"<0.32551.1583>"

```

JSONs are not parsed by default, i only got **message** field in ES, but I would like to extract 3 fields (actually 4)

\*time as a timestamp in ISO8601 format, not this default RFC 3339 format  
\*loglevel as a text or keyword  
\*pid as a keyword without \<\> braces  
\*msg as a "message"

It should look in Kibana like this:

```auto
"time": "2022-12-21T02:14:59+01:00"
"level": "error"
"message": "Error on AMQP connection <0.32551.1583>: enotconn (socket is not connected)"
"pid": "0.32551.1583"

```

Tried to play with grok and json filters without luck converting timestamp from RFC 3339 to ISO8601, any input would be appreciated. Thanks.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 27, 2023, 8:30pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/2 "2023-09-27T20:30:24Z")

</div>

What does your Logstash configuration file looks like?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 27, 2023, 8:41pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/3 "2023-09-27T20:41:02Z")

</div>

If your `json` log looks like this:

> {"timestamp":"2022-12-21 03:14:59.977922+02:00","level":"error","msg":"Error on AMQP connection \<0.32551.1583\>: enotconn (socket is not connected)","domain":"rabbitmq.connection","pid":"\<0.32551.1583\>"}

Then this filter will parse it:

```auto
filter {
    json {
        source => "message"
        remove_field => ["message"]
    }
    mutate {
        gsub => ["pid","<>",""]
        rename => {
            "msg" => "message"
        }
    }
    date {
        match => ["timestamp","yyyy-MM-dd HH:mm:ss.SSSSSSZZ"]
        target => "time"
        remove_field => ["timestamp"]
    }
}

```

The `gsub` will remove the `<>` from your `pid` field, and the `rename` will rename `msg` into `message`.

The `date` filter will parse your date format into ISO8601.

The output would be something like this:

```auto
{
    "@timestamp" => 2023-09-27T20:38:17.726537839Z,
      "@version" => "1",
           "pid" => "<0.32551.1583>",
        "domain" => "rabbitmq.connection",
          "time" => 2022-12-21T01:14:59.977Z,
         "level" => "error",
          "host" => "lab",
       "message" => "Error on AMQP connection <0.32551.1583>: enotconn (socket is not connected)"
}

```

---

<div class="post-metadata">

**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\
**Post date:** [September 27, 2023, 9:23pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/5 "2023-09-27T21:23:53Z")

</div>

why is it pid after gsub still with the brackets (`<0.32551.1583>`)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 27, 2023, 9:27pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/6 "2023-09-27T21:27:08Z")

</div>

Oh yeah, my mistake, I thought it worked on the first try.

Just need to change the gsub to this:

```auto
gsub => ["pid","[<>]",""]

```

And the `<>` will be removed.

---

<div class="post-metadata">

**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\
**Post date:** [September 27, 2023, 9:32pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/7 "2023-09-27T21:32:20Z")

</div>

thanks 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 25, 2023, 9:32pm UTC](https://discuss.elastic.co/t/parse-rabbitmq-json-log/344009/8 "2023-10-25T21:32:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.