# Logstash access dynamic field with ruby filter

**URL:** <https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489>\
**Category:** Logstash\
**Created:** [March 7, 2021, 9:41pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489 "2021-03-07T21:41:28Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ingram\_Gultom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ingram_gultom/32/56569_2.png) [@Ingram\_Gultom](https://discuss.elastic.co/u/Ingram_Gultom)\
**Post date:** [March 7, 2021, 9:41pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/1 "2021-03-07T21:41:28Z")

</div>

Hello

I have nested field looks like this

```auto
[field1][field2][field3]

```

I want to remove field1 and field2 so the expected output will be

```auto
[field3]

```

I already manage to remove field1 with this ruby filter

```auto
    ruby {
        code => '
            event.get("field1").each { |k, v|
                event.set(k,v)
            }
            event.remove("field1")
        '
    }

```

But I still cannot remove field2 because field2 is dynamic. How can I remove dynamic field2?

Thank you

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 7, 2021, 10:12pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/2 "2021-03-07T22:12:42Z")

</div>

Maybe I am not getting your question but why not just do something like.

```auto
mutate {
 rename => ["[field1][field2][field3]", "field3" ]
 remove_field => ["field1"]
}

```

---

<div class="post-metadata">

**Author:** ![Ingram\_Gultom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ingram_gultom/32/56569_2.png) [@Ingram\_Gultom](https://discuss.elastic.co/u/Ingram_Gultom)\
**Post date:** [March 7, 2021, 10:31pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/3 "2021-03-07T22:31:11Z")

</div>

Yes that will work for field1 but field2 is dynamic

How can I remove dynamic field?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 7, 2021, 11:29pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/4 "2021-03-07T23:29:29Z")

</div>

How about

```
        event.get("field1").each { |k, v|
            v.each { |j, u|
                event.set(j,u)
            }
        }
```

---

<div class="post-metadata">

**Author:** ![Ingram\_Gultom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ingram_gultom/32/56569_2.png) [@Ingram\_Gultom](https://discuss.elastic.co/u/Ingram_Gultom)\
**Post date:** [March 8, 2021, 12:42am UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/5 "2021-03-08T00:42:13Z")

</div>

Thanks It works

But some of my data looks like this

[field1][field2.1][field3] =\> "1"  
[field1][field2.2][field3] =\> "2"

So in one document I have the same name for [field3]

With my current config the output will be like this  
[field3] =\> "1"

value "2" will be replaced

So my expected output like  
[field3] =\> ["1", "2"] become like array

or automatically renamed  
[field3.1] =\> "1"  
[field3.2] =\> "2"

Thank you

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 8, 2021, 1:15am UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/6 "2021-03-08T01:15:20Z")

</div>

> [@Ingram\_Gultom](#):
>
> With my current config the output will be like this

Have you actually tested that? If a field exists then I would expect event.set to change it to be an array.

If necessary you could do

```
    event.get("field1").each { |k, v|
        a = []
        key = nil
        v.each { |j, u|
            key = j
            a << u
        }
        event.set(key, a)
    }

```

---

<div class="post-metadata">

**Author:** ![Ingram\_Gultom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ingram_gultom/32/56569_2.png) [@Ingram\_Gultom](https://discuss.elastic.co/u/Ingram_Gultom)\
**Post date:** [March 8, 2021, 5:37am UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/7 "2021-03-08T05:37:18Z")

</div>

Yes already tested, Here is the result

ruby filter

```auto
ruby {
         code => '
           event.get("field1").each { |k, v|
            v.each { |j, u|
                event.set(j,u)
            }
       }
        '
    }

```

Result field3 value replaced

```auto
{
  "_index": "email-2021.03.08",
  "_type": "_doc",
  "_id": "GMdTEHgBiOEFcJ6DutV8",
  "_version": 1,
  "_score": null,
  "_source": {
    "@version": "1",
    "field3": "2",
    "@timestamp": "2021-03-08T05:33:45.174Z",
    "host": "control-station7",
    "message": "{\"field1\":{\"field2A\":{\"field3\":\"1\"},\"field2B\":{\"field3\":\"2\"}}}",
    "path": "/home/elastic/logstash-7.9.3/email2.log",
    "field1": {
      "field2A": {
        "field3": "1"
      },
      "field2B": {
        "field3": "2"
      }
    }
  },
  "fields": {
    "@timestamp": [
      "2021-03-08T05:33:45.174Z"
    ]
  },
  "sort": [
    1615181625174
  ]
}

```

---

<div class="post-metadata">

**Author:** ![Ingram\_Gultom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ingram_gultom/32/56569_2.png) [@Ingram\_Gultom](https://discuss.elastic.co/u/Ingram_Gultom)\
**Post date:** [March 8, 2021, 5:39am UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/8 "2021-03-08T05:39:15Z")

</div>

And here is the result for second ruby filter

```auto
ruby {
     code => '
       event.get("field1").each { |k, v|
       a = []
       key = nil
       v.each { |j, u|
          key = j
          a << u
       }
      event.set(key, a)
 }
      '
}

```

Result field3 value replaced

```auto
{
  "_index": "email-2021.03.08",
  "_type": "_doc",
  "_id": "QsdLEHgBiOEFcJ6DYdQN",
  "_version": 1,
  "_score": null,
  "_source": {
    "field3": [
      "2"
    ],
    "field1": {
      "field2A": {
        "field3": "1"
      },
      "field2B": {
        "field3": "2"
      }
    },
    "@timestamp": "2021-03-08T05:24:37.942Z",
    "host": "control-station7",
    "message": "{\"field1\":{\"field2A\":{\"field3\":\"1\"},\"field2B\":{\"field3\":\"2\"}}}",
    "@version": "1",
    "path": "/home/elastic/logstash-7.9.3/email2.log"
  },
  "fields": {
    "@timestamp": [
      "2021-03-08T05:24:37.942Z"
    ]
  },
  "sort": [
    1615181077942
  ]
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 8, 2021, 5:45pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/9 "2021-03-08T17:45:19Z")

</div>

> [@Ingram\_Gultom](#):
>
> ```
> event.get("field1").each { |k, v|
> a = []
> key = nil
> 
> ```

Sorry, that should be

```
a = []
key = nil
event.get("field1").each { |k, v|

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 5, 2021, 5:46pm UTC](https://discuss.elastic.co/t/logstash-access-dynamic-field-with-ruby-filter/266489/10 "2021-04-05T17:46:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.