# Split fields in different docs

**URL:** <https://discuss.elastic.co/t/split-fields-in-different-docs/295895>\
**Category:** Logstash\
**Created:** [February 1, 2022, 2:05am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895 "2022-02-01T02:05:13Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![cris](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cris/32/101855_2.png) [@cris](https://discuss.elastic.co/u/cris)\
**Post date:** [February 1, 2022, 2:05am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/1 "2022-02-01T02:05:13Z")

</div>

Hello friends. I am trying to reindex some data to another index but with a little differences.  
In the first index I have this kind of hit:

```auto
{
  "France": {
    "Testing": {
      "status": "passed"
    }
  },
  "Spain": {
    "Testing": {
      "status": "passed"
    }
  },
  "Brazil": {
    "Testing": {
      "status": "passed"
    }
  },
  "USA": {
    "Testing": {
      "status": "failed"
    }
  }
}

```

So I want to split each country in other doc each one some like this:

```auto
{
  "Country": {
    "Testing": {
      "status": "passed"
    }
  }
}

```

So I was doing this with logstash but I failed, I have this config. Where I am tryin to get all the values from the fields but I don't know how to get it. I want to do some like in kibana where use the "\*" but I don know how to use it in logstash

```auto

input {
  elasticsearch {
    hosts => "localhost:9200"
    index => "informationCountry"
    size => 500
    scroll => "5m"
    docinfo => true
  }
}
filter{
ruby {
           code => '
          f = event.get("[*][Testing][status]") 
        if f.is_a? Hash
            newF = []
            f.each { |k, v|
                newF << { "ENV": k }
            }
            event.set("[Country][Testing][status]", newF)
        end '
    }

}
output {
  elasticsearch {
    hosts => "localhost:9200"
    index => "testingInformation"
    document_id => "%{[@metadata][_id]}"
  }

}

```

Please help I tried with multiple ways but I can not got it

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [February 1, 2022, 3:21am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/2 "2022-02-01T03:21:51Z")

</div>

> [@cris](#):
>
> `f = event.get("[*][Testing][status]") `

works well?

---

<div class="post-metadata">

**Author:** ![cris](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cris/32/101855_2.png) [@cris](https://discuss.elastic.co/u/cris)\
**Post date:** [February 1, 2022, 3:56am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/3 "2022-02-01T03:56:23Z")

</div>

No ☹

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [February 1, 2022, 4:21am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/4 "2022-02-01T04:21:14Z")

</div>

You have to get the top level hash by `event.get("message")` and analyze it.

---

<div class="post-metadata">

**Author:** ![cris](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cris/32/101855_2.png) [@cris](https://discuss.elastic.co/u/cris)\
**Post date:** [February 1, 2022, 7:06am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/5 "2022-02-01T07:06:16Z")

</div>

How can I get it? because i tried with `puts event.get("message")` to print the value but I don get nothing is empty

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [February 1, 2022, 9:45am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/6 "2022-02-01T09:45:44Z")

</div>

Sorry, you had to use `event.to_hash` with Elasticsearch input plugin.

Use this filter:

```auto
input {
  elasticsearch {
    docinfo => true
  }
}
filter{
    ruby {
        code => '
            keys = event.to_hash.keys
            array = []
            keys.each{|k|
                if !(k.start_with?("@")) then
                    array << {"ENV": k}
                    event.remove(k)
                end
            }
            event.set("[Country][Testing][status]", array)
        '
    }
}
output {
    stdout {
        codec => rubydebug{metadata => true}
    }
}

```

You will get:

```auto
{
    "@timestamp" => 2022-02-01T09:44:25.393Z,
     "@metadata" => {
        "_index" => "test_split_fields",
           "_id" => "QhaitH4Bf0nakUP8oFTM",
         "_type" => "_doc"
    },
      "@version" => "1",
       "Country" => {
        "Testing" => {
            "status" => [
                [0] {
                    "ENV" => "Spain"
                },
                [1] {
                    "ENV" => "USA"
                },
                [2] {
                    "ENV" => "France"
                },
                [3] {
                    "ENV" => "Brazil"
                }
            ]
        }
    }
}

```

---

<div class="post-metadata">

**Author:** ![cris](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cris/32/101855_2.png) [@cris](https://discuss.elastic.co/u/cris)\
**Post date:** [February 1, 2022, 5:16pm UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/7 "2022-02-01T17:16:54Z")

</div>

Oh! yes I get this but how can add the data?, I was seeing the data on Kibana but the field are empty 🤔

```auto
{
    "@timestamp" => 2022-02-01T17:03:43.789Z,
     "@metadata" => {
        "_index" => "informationCountry",
         "_type" => "_doc",
           "_id" => "0Xhuq34BrOPMccolsPMW"
    },
      "@version" => "1",
       "Country" => {
        "Testing" => {
            "status" => [
                [0] {
                    "ENV" => "France"
                },
                [1] {
                    "ENV" => "Spain"
                },
                [2] {
                    "ENV" => "Brazil"
                },
                [3] {
                    "ENV" => "USA"
                },
                [4] {
                    "ENV" => "localtime"
                }
            ]
        }
    }
}

```

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [February 2, 2022, 1:57am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/8 "2022-02-02T01:57:54Z")

</div>

Are you using appropriate output plugin?

What does " the field are empty" mean? You found indexed documents but fields are empty? Or documents themselves are not indexed? In such debugging situation, you should use dev tools and REST API to exclude other problems.

---

<div class="post-metadata">

**Author:** ![cris](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cris/32/101855_2.png) [@cris](https://discuss.elastic.co/u/cris)\
**Post date:** [February 2, 2022, 6:23am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/9 "2022-02-02T06:23:54Z")

</div>

I am using this.

```auto
output {
  elasticsearch {
    hosts => "localhost:9200"
    index => "testingInformation"
    document_id => "%{[@metadata][_id]}"
  }

}

```

I get in the hit this:

```auto
 "Country" : {
            "Testing" : {
              "status" : [
                {
                  "ENV" : "Spain"
                },
                {
                  "ENV" : "USA"
                },
                {
                  "ENV" : "France"
                },
                {
                  "ENV" : "Brazil"
                }
              ]
            }
          }

```

but not the results of each country, the Env: passed or Env: failed. How can I get the values that are into each key?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 2, 2022, 6:24am UTC](https://discuss.elastic.co/t/split-fields-in-different-docs/295895/10 "2022-03-02T06:24:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.