# Dynamically create fields from another field with json input file

**URL:** <https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508>\
**Category:** Logstash\
**Created:** [October 9, 2024, 9:10am UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508 "2024-10-09T09:10:13Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![remich](https://avatars.discourse-cdn.com/v4/letter/r/c89c15/32.png) [@remich](https://discuss.elastic.co/u/remich)\
**Post date:** [October 9, 2024, 9:10am UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/1 "2024-10-09T09:10:13Z")

</div>

Hi,  
I have json input that contains appname with their version and other field with standard string like :

```auto
{
  "app1": "1.2.3",
  "app2": "3.2.1",
  "arch": "virtual"
}

```

I would like to be able to use filter versions in kibana like "app1\<1.2.3" but I wasn't able to find how to do that.

So if thought maybe creating new fields like appname1\_major, appname1\_minor and appname1\_patch.  
But I cant manage to do it in logstash  
I have no way of knowing wich appname will be in the file, the version is always in the same format : 1.2.3

I managed to do it for only one field with grok but I can't do it fo every field that might appear or not

```auto
grok { 
  match => { "app1" => "%{INT:app1_major}\.%{INT:app1_minor}\.%{INT:app1_patch}" } 
}

```

This is ugly and takes more storage space but that's all I have for now  
Is there a good or better way to do this ?

Thanks

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 9, 2024, 9:31am UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/2 "2024-10-09T09:31:34Z")

</div>

Welcome!

IMHO you are doing it the right way.  
I'd probably do something like:

```auto
{
 "app1": {
  "major": 1,
  "minor": 2,
  "patch": 3
 }
}

```

instead of:

```auto
{
  "app1_major": 1,
  "app1_minor": 2,
  "app1_patch": 3
}

```

But that's a matter of taste 😉

---

<div class="post-metadata">

**Author:** ![remich](https://avatars.discourse-cdn.com/v4/letter/r/c89c15/32.png) [@remich](https://discuss.elastic.co/u/remich)\
**Post date:** [October 9, 2024, 9:55am UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/3 "2024-10-09T09:55:14Z")

</div>

yeah I agree but I would like to keep the full version too, it is nicer and way easier in a dashboard

Anyway, maybe my post wasn't clear enough but I don't know IF app1 exist or even what would really be app1's name

I made a regex to match a key value pair

```auto
/(?:\"|\')(?<key>[^"]*)(?:\"|\')(?=:)(?:\:\s*)(?:\"|\')?(?<major>\d+)\.(?<minor>\d+)\.(?<patch>\d+)/gm

```

This regex match the key name and each integer in version  
But I dont know how to tell logstash to create fields =\> value whit this regex  
like `${key}_major => ${major}`

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 9, 2024, 11:29am UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/4 "2024-10-09T11:29:19Z")

</div>

I see. I don't know if this can be done automatically without using [the ruby filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-ruby.html).

But actually I just realized that Elasticsearch supports as well the [version type](https://www.elastic.co/guide/en/elasticsearch/reference/current/version.html), so basically you don't need to do all that I guess...

---

<div class="post-metadata">

**Author:** ![remich](https://avatars.discourse-cdn.com/v4/letter/r/c89c15/32.png) [@remich](https://discuss.elastic.co/u/remich)\
**Post date:** [October 9, 2024, 1:23pm UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/5 "2024-10-09T13:23:03Z")

</div>

For that I need to specify each field statically, which I can't do

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 9, 2024, 2:11pm UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/6 "2024-10-09T14:11:17Z")

</div>

You could try something like

```
    ruby {
        code => '
            event.to_hash.each { |k, v|
                if v.is_a? String
                    matches = /^(\d+)\.(\d+)\.(\d+)$/.match(v)
                    if matches
                        newKey = k.gsub(/ /, "_").gsub(/[\[\]]/, "")
                        event.set("[apps][#{newKey}]", {
                            "major" => matches[1].to_i,
                            "minor" => matches[2].to_i,
                            "patch" => matches[3].to_i
                        })
                        # Alternatively
                        #event.set("#{newKey}_major", matches[1].to_i)
                        #event.set("#{newKey}_minor", matches[2].to_i)
                        #event.set("#{newKey}_patch", matches[3].to_i)
                    end
                end
            }
        '
    }

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 9, 2024, 2:19pm UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/7 "2024-10-09T14:19:05Z")

</div>

But do you have a prefix or something? In which case a dynamic template would work.

---

<div class="post-metadata">

**Author:** ![remich](https://avatars.discourse-cdn.com/v4/letter/r/c89c15/32.png) [@remich](https://discuss.elastic.co/u/remich)\
**Post date:** [October 9, 2024, 2:36pm UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/8 "2024-10-09T14:36:27Z")

</div>

I actually ended up with something like this (but yours is better)

```auto
ruby {
      code => "
        event.to_hash.each do |key, value|
        if value =~ /^(?<major>\d+)\.(?<minor>\d+)\.(?<patch>\d+)$/
          event.remove(key)
          event.set(key + '.full', value)
          event.set(key + '.major', $~[:major])
          event.set(key + '.minor', $~[:minor])
          event.set(key + '.patch', $~[:patch])
        end
      end
      "
      }

```

I still need to play with the remove event to remove the key or else I am not able to create key.major....  
But this is working as I want and easy enough to understand  
@dadoonet no prefix, just "randomappname": "1.2.3"

Thanks all for your help

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 9, 2024, 4:11pm UTC](https://discuss.elastic.co/t/dynamically-create-fields-from-another-field-with-json-input-file/368508/9 "2024-10-09T16:11:58Z")

</div>

I see. So if the `value` matches the `regex`, I'd just set the value to a field named `version_[key]` and use a dynamic template which matches any field where name starts with `version_`...

My 2 cents.