# Trying to separate key-value pairs in nested array (help with RUBY)

**URL:** <https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035>\
**Category:** Logstash\
**Created:** [October 11, 2018, 11:31am UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035 "2018-10-11T11:31:33Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Harshet\_Jain](https://avatars.discourse-cdn.com/v4/letter/h/dc4da7/32.png) [@Harshet\_Jain](https://discuss.elastic.co/u/Harshet_Jain)\
**Post date:** [October 11, 2018, 11:31am UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/1 "2018-10-11T11:31:33Z")

</div>

I have been trying to import my mySQL database into Elasticsearch through Logstash but I am stuck (for two days now) to separate three columns into separate fields.

I want to split these into different fields so it displays as:

```
"dr_behaviour_Patient Healer": "5", 
"dr_behaviour_Couldn’t Care Less": "5"

```

```
filter {
  mutate {
    split => {
        "dr_behaviour_rate" => "," 
        }
    }
  kv {
    source => "dr_behaviour_rate"
    prefix => "dr_behaviour"
    field_split_pattern => "/[/]"
    include_brackets => true
  }
}

```

This is the actual data that is interpreted by Logstash/Elasticsearch

 ![Actual](https://us1.discourse-cdn.com/elastic/original/3X/9/c/9cc460e8605fcc65b881621fcd83e4956536691b.png)

This is how the data is being changed through the above query:

 ![Modified%20with%20SPLIT%20seperator](https://us1.discourse-cdn.com/elastic/original/3X/3/3/3315ea52d585f33aaab767c4c261e72e65590ce5.png)

---

<div class="post-metadata">

**Author:** ![OphyTe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ophyte/32/36444_2.png) [@OphyTe](https://discuss.elastic.co/u/OphyTe)\
**Post date:** [October 11, 2018, 3:29pm UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/2 "2018-10-11T15:29:03Z")

</div>

I think you have to play on the value-split options.

Try something like this :

```
 filter {
  kv {
    source => "dr_behaviour_rate"
    prefix => "dr_behaviour_"
    field_split => ","
    include_brackets => false
    value-split => "\["
    trim_key => "\s"
    trim_value => "\s"
  }
}
```

---

<div class="post-metadata">

**Author:** ![Harshet\_Jain](https://avatars.discourse-cdn.com/v4/letter/h/dc4da7/32.png) [@Harshet\_Jain](https://discuss.elastic.co/u/Harshet_Jain)\
**Post date:** [October 11, 2018, 3:44pm UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/3 "2018-10-11T15:44:42Z")

</div>

Thank you. This solved it. One issue remains. I am getting the data as:

```
"dr_behaviour_Patient Healer": "1]",

```

How do I remove the last "]"?

---

<div class="post-metadata">

**Author:** ![OphyTe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ophyte/32/36444_2.png) [@OphyTe](https://discuss.elastic.co/u/OphyTe)\
**Post date:** [October 11, 2018, 4:00pm UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/4 "2018-10-11T16:00:01Z")

</div>

I thought the " include\_brackets =\> false" will remove it but I think you can add these symbols in the trim options :

```
 filter {
  kv {
    source => "dr_behaviour_rate"
    prefix => "dr_behaviour_"
    field_split => ","
    include_brackets => false
    value-split => "\["
    trim_key => "\s\[\]"
    trim_value => "\s\[\]"
  }
}
```

---

<div class="post-metadata">

**Author:** ![OphyTe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ophyte/32/36444_2.png) [@OphyTe](https://discuss.elastic.co/u/OphyTe)\
**Post date:** [October 11, 2018, 4:04pm UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/5 "2018-10-11T16:04:32Z")

</div>

Keep in mind that if your key has a "[" into it, this won't work ...

---

<div class="post-metadata">

**Author:** ![Harshet\_Jain](https://avatars.discourse-cdn.com/v4/letter/h/dc4da7/32.png) [@Harshet\_Jain](https://discuss.elastic.co/u/Harshet_Jain)\
**Post date:** [October 11, 2018, 4:10pm UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/6 "2018-10-11T16:10:28Z")

</div>

Yes, you are right. It works. I will ensure that the key will not have square brackets. Thank you very much!!! 🙂

If I have other fields with similar issues, should I add another set of kv filter with that field as the source? In this way, I will have three sets of KV filters with different source and prefix. for example,

```
 filter {
  kv {
    source => "dr_orientation"
    prefix => "dr_orientation_"
    field_split => ","
    include_brackets => false
    value-split => "\["
    trim_key => "\s\[\]"
    trim_value => "\s\[\]"
  }
}
```

---

<div class="post-metadata">

**Author:** ![OphyTe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ophyte/32/36444_2.png) [@OphyTe](https://discuss.elastic.co/u/OphyTe)\
**Post date:** [October 12, 2018, 8:49am UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/7 "2018-10-12T08:49:21Z")

</div>

Yes it should work but with only one filter block !

```
filter {
  kv {
    ...
  }
  kv {
    ...
  }
  kv {
    ...
  }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 9, 2018, 8:49am UTC](https://discuss.elastic.co/t/trying-to-separate-key-value-pairs-in-nested-array-help-with-ruby/152035/8 "2018-11-09T08:49:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.