# Remove part of field name from json inputted fields

**URL:** https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412
**Category:** Logstash
**Created:** [September 23, 2016, 7:04pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412 "2016-09-23T19:04:18Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![sirmodok](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sirmodok](https://discuss.elastic.co/u/sirmodok)
#### Post date: [September 23, 2016, 7:04pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/1 "2016-09-23T19:04:18Z")

</div>

I have an input into elasticsearch from a large amount of json formatted data. The input looks like this

```auto
  file{
    codec => json
    path => "/opt/apps/logs/health/*.log"
    type => "health"
  }
}

```

Very simple.

The data in each event looks like this (paraphrased)

First event

````auto
Item.1234.pineapple.red
Item.1234.pegasus.purple.dogsled```

Second Event
```Item.5555.description
Item.5555.pineapple.red
Item.5555.pegasus.purple.dogsled```

I would like to get rid of this randomly generate number in the 2nd array slot. 
In other words I want the results of each event to look like this

```Item.description
Item.pineapple.red
Item.pegasus.purple.dogsled```

I'm trying to use the ruby filter to do this, however it's not even close to working. Can anyone help me accomplish this task?
````

---

<div class="post-metadata">

### Author: ![sirmodok](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sirmodok](https://discuss.elastic.co/u/sirmodok)
#### Post date: [September 23, 2016, 7:24pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/2 "2016-09-23T19:24:53Z")

</div>

I have tried using the information posted here

> [@Logstash mutate - rename field that occurs as an array](https://discuss.elastic.co/t/logstash-mutate-rename-field-that-occurs-as-an-array/50096):
>
> Hi, I'm trying to rename a field within an Elasticsearch type using the mutate filter. This particular field occurs within a zero-to-many array. Here is a mapping that closely resembles mine (the names have been changed to protect the innocent): "fieldRoot": { "properties": { "fieldCollection": { "properties": { "itemColor": { "index": "no", "type": "string" }, "itemDescription": { "index": "no", "type": "string" …

And I just end up with \_rubyparse errors in my tags as well as some additional "cannot convert string to integer " errors in my logstash.log

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 26, 2016, 7:57pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/3 "2016-09-26T19:57:04Z")

</div>

> I'm trying to use the ruby filter to do this, however it's not even close to working. Can anyone help me accomplish this task?

Please show us what you have so far.

---

<div class="post-metadata">

### Author: ![sirmodok](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sirmodok](https://discuss.elastic.co/u/sirmodok)
#### Post date: [September 26, 2016, 8:04pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/4 "2016-09-26T20:04:01Z")

</div>

Ok, but fair warning I don't understand what I'm doing here, just trying to adapt your code from the previous post to this one.

```auto
filter {
  if [type] == "health"{
    ruby {
      code => "event['Item'].each {|key|
                        event['testing'] = key['description']
                     }"
    }
  }
}

```

This was me trying to keep it simple and just create a field named "testing" from the field Item.5656.description. Even that does not work.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 27, 2016, 6:02am UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/5 "2016-09-27T06:02:11Z")

</div>

This should work better:

```nohighlight
filter {
  if [type] == "health"{
    ruby {
      code => "event['Item'].each_value {|value|
                        event['testing'] = value['description']
                     }"
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![sirmodok](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sirmodok](https://discuss.elastic.co/u/sirmodok)
#### Post date: [September 27, 2016, 4:16pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/6 "2016-09-27T16:16:01Z")

</div>

Thanks magnusbaeck, this worked but maybe I am approaching this wrong. Using this method I need to write a line for every field in the event

```auto
filter {
  if [type] == "health"{
    ruby {
      code => "event['Items'].each_value {|value|
                        event['description'] = value['description']
                        event['pineapple']['red'] = value['pineapple']['red']
                        event['pegasus']['purple']['dogsled'] = value['pegasus']['purple']['dogsled']
                     }"
    }
  }
}

```

Is this right? the whole reason I ended up here was because the mutate/rename plugin wouldn't support wildcards and I have 50 fields in each event that I need to remove the random number from. I can't seem to figure out how to automate the whole process.

I tried using slice, but no matter what I do with slice it tells me  
Ruby exception occurred: undefined method `slice!' for #\<Hash

---

<div class="post-metadata">

### Author: ![sirmodok](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sirmodok](https://discuss.elastic.co/u/sirmodok)
#### Post date: [September 27, 2016, 5:27pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/7 "2016-09-27T17:27:05Z")

</div>

Can't figure out why this doesn't work.

```auto
filter {
  if [type] == "health"{
    ruby {
      code => "event['Items'].each_value {|value|
                           event[value.delete_at(1)] = value.each
                     }"
    }
  }
}

```

Always results in  
Ruby exception occurred: undefined method `delete\_at' for #Hash:0x4453e3e5 {:level=\>:error}

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 28, 2016, 3:40am UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/8 "2016-09-28T03:40:57Z")

</div>

Inside the `each_value` loop, put another loop that iterates over the key/value pairs in `value` and copies them:

```nohighlight
value.each_pair { |k, v|
  event[k] = v
}

```

---

<div class="post-metadata">

### Author: ![sirmodok](https://avatars.discourse-cdn.com/v4/letter/s/c6cbf5/32.png) [@sirmodok](https://discuss.elastic.co/u/sirmodok)
#### Post date: [September 28, 2016, 3:45pm UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/9 "2016-09-28T15:45:38Z")

</div>

magnusblack  
This is working properly, and I added some additional code (that I stole from another of your posts) that removes the old fields too. Here is the whole thing together.

```auto
filter {
  if [type] == "health"{
    ruby {
      code => "
        event['Item'].each_value {|value|
          value.each_pair { |key, val|
            event[key]=val
           }
         }
      "
    }
    ruby {
      code => "
        event.to_hash.keys.each { |k|
          if k.start_with?('Item')
            event.remove(k)
          end
        }
      "
    }
  }
}

```

magnusbaeck Not only to you have a kick ass name, but you have been a tremendous help to me. Thank you very much.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:36am UTC](https://discuss.elastic.co/t/remove-part-of-field-name-from-json-inputted-fields/61412/10 "2017-07-06T04:36:32Z")

</div>


