# Logstash RabbitMQ Wrap Incoming Message

**URL:** <https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925>\
**Category:** Logstash\
**Created:** [October 31, 2017, 4:38pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925 "2017-10-31T16:38:55Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeff\_Bursik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeff_bursik/32/23626_2.png) [@Jeff\_Bursik](https://discuss.elastic.co/u/Jeff_Bursik)\
**Post date:** [October 31, 2017, 4:38pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/1 "2017-10-31T16:38:56Z")

</div>

I'm trying to wrap the incoming message off of Rabbit and then have some extra metadata outside of the wrapped message. Can this be done?

Hoping for an end result like:  
errorMessage:{ ... },  
trackingID:{ ... }  
headerProperties:{ ... }  
message:{...}

```
filter {
mutate {
rename => ["@message", "message"]
 
    add_field => {"errorMessage" => "%{[@metadata][rabbitmq_headers][errorMessage]}"}
    add_field => {"headerProperties" => "%{[@metadata][rabbitmq_properties]}"}
    add_field => {"trackingID" => "%{[@metadata][rabbitmq_headers][trackingId]}"}
}
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 31, 2017, 8:22pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/2 "2017-10-31T20:22:56Z")

</div>

What you have looks reasonable. In what way is it not working? What does your configuration look like? Show sample output from a `stdout { codec => rubydebug { metadata => true } }` output.

---

<div class="post-metadata">

**Author:** ![Jeff\_Bursik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeff_bursik/32/23626_2.png) [@Jeff\_Bursik](https://discuss.elastic.co/u/Jeff_Bursik)\
**Post date:** [November 1, 2017, 12:30am UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/3 "2017-11-01T00:30:17Z")

</div>

Thanks for the quick response.

Heres a message thats saved in elastic and is the same with the standard output:

```
{
      "reportsource": {
        "active": true,
        "_id": "1d38ed20-39c5-11e7-a4de-8b50391cf1ee",
        "updatedDate": "2017-10-31 16:10:08.260"
       },
      "errorMessage": "NodeNotConnectedException Node not connected]",
      "description": "N",
      "active": true,
      "sourceUpdated": 1469145600000,
      "updatedDate": "2017-10-31 12:51:42.656",
      "@timestamp": "2017-10-31T19:39:22.900Z",
      "@version": "1",
      "headerProperties": "{\"delivery-mode\":2,\"content-type\":\"application/json\",\"exchange\":\"error.exchange\",\"routing-key\":\"report\",\"consumer-tag\":\"amq.ctag-Q9vmGJ0GVH9zAcPkW9eCEg\",\"priority\":0}",
      "sourceCreated": 1468339661000,
      "class": "report"
    
  }

```

I just can't get the message to move into the new field. It treats the entire message off of rabbit as JSON and doesn't wrap it to the new field

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 1, 2017, 4:41am UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/4 "2017-11-01T04:41:56Z")

</div>

Not sure what you're trying to do here. What does your configuration look like? And what would you like the message above to look like instead?

---

<div class="post-metadata">

**Author:** ![Jeff\_Bursik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeff_bursik/32/23626_2.png) [@Jeff\_Bursik](https://discuss.elastic.co/u/Jeff_Bursik)\
**Post date:** [November 14, 2017, 3:17pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/5 "2017-11-14T15:17:41Z")

</div>

@magnusbaeck apologies for not responding until now. Here is a sample configuration:

```
input {
  rabbitmq {
  host => "local
  queue => "Error"
  durable => true
  threads => 3
  metadata_enabled => true
  prefetch_count => 50
  type => 'rabbit-errors'
  }
}

filter {
mutate {
rename => ["@message", "message"]
    rename => ["log", "message"]
    add_field => {"errorMessage" => "%{[@metadata][rabbitmq_headers][errorMessage]}"}
    add_field => {"headerProperties" => "%{[@metadata][rabbitmq_properties]}"}
    add_field => {"trackingID" => "%{[@metadata][rabbitmq_headers][trackingId]}"}
}
}

output {
  elasticsearch {
   }
}

```

which will print out the message shown above. I would prefer the errorMessage, and headers, to live outside of the actual message from the Rabbit queue. Something like:

```
{
      "errorMessage": "NodeNotConnectedException Node not connected]",
      "@timestamp": "2017-10-31T19:39:22.900Z",
      "@version": "1",
      "headerProperties": "{......}",
      messgage: {
          "reportsource": {
             "active": true,
             "_id": "1d38ed20-39c5-11e7-a4de-8b50391cf1ee",
             "updatedDate": "2017-10-31 16:10:08.260"
          },
          "description": "N",
          "active": true,
          "sourceUpdated": 1469145600000,
          "updatedDate": "2017-10-31 12:51:42.656",
          "sourceCreated": 1468339661000,
          "class": "report"
      }
  }
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 14, 2017, 3:21pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/6 "2017-11-14T15:21:14Z")

</div>

Okay. Use a mutate filter to move (rename) all the fields into being subfields of `message`. If you can't enumerate all the fields to move (because they're dynamic in nature) you need to use a ruby filter (i.e. there's no wildcard-based moving of fields).

---

<div class="post-metadata">

**Author:** ![Jeff\_Bursik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeff_bursik/32/23626_2.png) [@Jeff\_Bursik](https://discuss.elastic.co/u/Jeff_Bursik)\
**Post date:** [November 14, 2017, 3:34pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/7 "2017-11-14T15:34:17Z")

</div>

Got it! Thank you, I'll read more into the Ruby filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 12, 2017, 3:42pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-wrap-incoming-message/105925/8 "2017-12-12T15:42:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.