# Gsub not working

**URL:** <https://discuss.elastic.co/t/gsub-not-working/100176>\
**Category:** Logstash\
**Created:** [September 12, 2017, 8:48am UTC](https://discuss.elastic.co/t/gsub-not-working/100176 "2017-09-12T08:48:38Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\
**Post date:** [September 12, 2017, 8:48am UTC](https://discuss.elastic.co/t/gsub-not-working/100176/1 "2017-09-12T08:48:38Z")

</div>

Hello,

I'm trying to remove [] from my data as it is not supported in Kibana.  
Generally, I'm doing it very well with gsub (with http input).

Now I'm using rabbitmq input and gsub is not working. I can't figure out why, and I've tested different solutions (see below). Note that I've tried other mutate options like rename and it is working well. I also tried gsub to replace for example , with ; and it is not working.

```
input {
  rabbitmq {
    host => "x.x.x.x"
    subscription_retry_interval_seconds => 5
    exchange => "amq.topic"
    key => "#"
    auto_delete => "true"
    user => "X"
    password => "x"
    metadata_enabled => "true"
    codec => "json"
}
 }

filter {
   mutate {
      #gsub => ["metrics","\]",""]
      #gsub => ["metrics","\[",""]
      gsub => ["metrics","\\[|\\]",""]
     
}
}

output {

```

This is an example of the json i'm receiving

```
{
  "_index": "proteus-rabbitmq-2017.09.12",
  "_type": "logs",
  "_id": "AV51Q5agOmTEVnBW6Jvj",
  "_score": null,
  "_source": {
    "@timestamp": "2017-09-12T08:45:16.697Z",
    "@version": "1",
    "topic": "@metadata",
    "timestamp": "2017-09-12T08:45:16.680Z",
    "tags": [],
    "metrics": [
      {
        "dataType": "float",
        "name": "temperature",
        "value": 14,
        "timestamp": "2017-09-12T08:45:16.680Z"
      },
      {
        "dataType": "float",
        "name": "humidity",
        "value": 56,
        "timestamp": "2017-09-12T08:45:16.680Z"
      }
    ]
  },
  "fields": {
    "@timestamp": [
      1505205916697
    ],
    "metrics_test.timestamp": [
      1505205916680,
      1505205916680
    ],
    "timestamp": [
      1505205916680
    ]
  },
  "sort": [
    1505205916697
  ]
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 12, 2017, 10:49am UTC](https://discuss.elastic.co/t/gsub-not-working/100176/2 "2017-09-12T10:49:30Z")

</div>

As you have a json codec in the input, the metrics field is an object and not a string, which probably is why gsub fails.

---

<div class="post-metadata">

**Author:** ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\
**Post date:** [September 12, 2017, 11:43am UTC](https://discuss.elastic.co/t/gsub-not-working/100176/3 "2017-09-12T11:43:56Z")

</div>

Hi, thanks for your answer.

No, the example is one of my multiple attempts. I also tried without the json codec in the input...with the same result...

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 12, 2017, 12:09pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/4 "2017-09-12T12:09:17Z")

</div>

What does the input look like and what is the desired output?

---

<div class="post-metadata">

**Author:** ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\
**Post date:** [September 12, 2017, 12:16pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/5 "2017-09-12T12:16:16Z")

</div>

this is the input :

```
{
"timestamp": “2017-06-20T11:44:08.383357”,
"metrics": [{
"name": “temperature”,
"timestamp": “2017-06-20T11:44:08.383357”,
"dataType": “float”,
"value": “25.5”
},
{
"name": “humidity”,
"timestamp": “2017-06-20T11:44:08.383357”,
"dataType": “float”,
"value": “50”
}]
}

```

I want the square brackets removed, so it would be like this :

```
{
"timestamp": “2017-06-20T11:44:08.383357”,
"metrics": {
"name": “temperature”,
"timestamp": “2017-06-20T11:44:08.383357”,
"dataType": “float”,
"value": “25.5”
},
{
"name": “humidity”,
"timestamp": “2017-06-20T11:44:08.383357”,
"dataType": “float”,
"value": “50”
}
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 12, 2017, 12:29pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/6 "2017-09-12T12:29:03Z")

</div>

If you remove the square brackets it is no longer valid JSON. Is that really what you want?

Would it not make more sense to convert it to something like this:

```auto
{
    "timestamp": "2017-06-20T11:44:08.383357",
    "temperature": 25.5,
    "humidity": 50
}

```

---

<div class="post-metadata">

**Author:** ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\
**Post date:** [September 12, 2017, 12:33pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/7 "2017-09-12T12:33:16Z")

</div>

the problem is that kibana is not recognising anything inside the brackets.  
With another application, i simply removed the bracket, and it recognized 2 separate events.  
One for [metrics.name](http://metrics.name) = temperature and one for [metrics.name](http://metrics.name) = humidity. This is what i want

With the brackets, kibana recognized nothing but a string event inside metrics field.

Do you think of another way to do it ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 12, 2017, 12:36pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/8 "2017-09-12T12:36:33Z")

</div>

If you want them as separate events you might be able to use the [split filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html).

---

<div class="post-metadata">

**Author:** ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\
**Post date:** [September 12, 2017, 12:43pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/9 "2017-09-12T12:43:53Z")

</div>

Yes, i understand i can use split for separating in 2 events. But the brackets will still be there and they are not supported by kibana

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 12, 2017, 12:46pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/10 "2017-09-12T12:46:58Z")

</div>

Once you have split them, you know each event will have only one object in the array, and you can then use the mutate filter to copy the fields down to the main event and then delete the metrics field.

---

<div class="post-metadata">

**Author:** ![Charlotte\_EGM](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charlotte_egm/32/13503_2.png) [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\
**Post date:** [September 12, 2017, 12:53pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/11 "2017-09-12T12:53:43Z")

</div>

It worked !!!!

Thanks soooo much, you saved my day 😃

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 10, 2017, 12:54pm UTC](https://discuss.elastic.co/t/gsub-not-working/100176/12 "2017-10-10T12:54:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.