# Dynamically bring nested fields to root level

**URL:** <https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948>\
**Category:** Logstash\
**Created:** [April 22, 2024, 11:49pm UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948 "2024-04-22T23:49:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rubhamra](https://avatars.discourse-cdn.com/v4/letter/r/858c86/32.png) [@rubhamra](https://discuss.elastic.co/u/rubhamra)\
**Post date:** [April 22, 2024, 11:49pm UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948/1 "2024-04-22T23:49:28Z")

</div>

I need to extract fields, full\_name, roles, email and username from below output

```auto
"Mika.singh" => {
         "metadata" => {},
        "full_name" => "mika singh",
            "roles" => [
            [0] "teaml",
            [1] "team-interface",
            [2] "dev",
            [3] "support-read"
        ],
            "email" => "mika.singh@xyz",
          "enabled" => true,
         "username" => "mika.singh"
    },
"logstash_user" => {
     "metadata" => {},
    "full_name" => "",
        "roles" => [
        [0] "logstash_system",
        [1] "logstash_admin"
    ],
        "email" => "",
      "enabled" => true,
     "username" => "logstash_user"
},

```

But as you can see top level field is different every time, I tried with ruby filter but none of them worked,

Please help me on this.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 23, 2024, 1:51am UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948/2 "2024-04-23T01:51:29Z")

</div>

So you have a pair of hashes called logstash\_user and Mika.singh. What is the structure of the data in which they occur?

---

<div class="post-metadata">

**Author:** ![rubhamra](https://avatars.discourse-cdn.com/v4/letter/r/858c86/32.png) [@rubhamra](https://discuss.elastic.co/u/rubhamra)\
**Post date:** [April 23, 2024, 12:36pm UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948/3 "2024-04-23T12:36:24Z")

</div>

I am trying to pull kibana users through logstash pipeline and store it in index

```auto
input {
  http_poller {
    urls => {
      kibana_users => {
        # Supports all options supported by ruby's Manticore HTTP client
        method => get
        user => "elastic"
        password => "xyz123"
        url => "https://10.2.3.20:9200/_security/user"
        headers => {
          Accept => "application/json"
        }
     }
    }
    truststore => "/usr/share/logstash/downloaded_truststore.jks"
    truststore_password => "xyz123"
    request_timeout => 60
    # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
    schedule => { cron => "* * * * * UTC"}
    codec => "json"
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    metadata_target => "http_poller_metadata"
  }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 23, 2024, 1:47pm UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948/4 "2024-04-23T13:47:48Z")

</div>

You are saying you want to change the structure of your data, by moving fields to the top-level, but you have not explained what the existing structure of your data is. We cannot answer a question that has not been asked.

---

<div class="post-metadata">

**Author:** ![rubhamra](https://avatars.discourse-cdn.com/v4/letter/r/858c86/32.png) [@rubhamra](https://discuss.elastic.co/u/rubhamra)\
**Post date:** [April 23, 2024, 2:03pm UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948/5 "2024-04-23T14:03:12Z")

</div>

Sorry Badger, But I am trying to pull list of users inside Kibana through user Api and store in an index to create visulization , for that I need fields  
like  
username, email, roles and full name

I am using curl command to pull the users list.

```auto
curl -X GET --cacert /usr/share/logstash/ca.pem -u elastic:xyz123 https://10.2.3.20:9200/_security/user

```

Getting below results from query

```auto
 
{"elastic":{"username":"elastic","roles":["superuser"],"full_name":null,"email":null,"metadata":{"_reserved":true},"enabled":true},"kibana":{"username":"kibana","roles":["kibana_system"],"full_name":null,"email":null,"metadata":{"_reserved":true,"_deprecated_reason":"Please use the [kibana_system] user instead.","_deprecated":true},"enabled":true},"kibana_system":{"username":"kibana_system","roles":["kibana_system"],"full_name":null,"email":null,"metadata":{"_reserved":true},"enabled":true}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 23, 2024, 2:53pm UTC](https://discuss.elastic.co/t/dynamically-bring-nested-fields-to-root-level/357948/6 "2024-04-23T14:53:14Z")

</div>

You could try something like

```
    ruby {
        code => '
            users = []
            event.to_hash.each { |k, v|
                if v.is_a? Hash
                    users << v
                    event.remove(k)
                end
            }
            event.set("users", users)
        '
    }
    split { field => "users" }
    ruby {
        code => '
            event.remove("users").each { |k, v|
                event.set(k,v)
            }
        '
    }

```