# Logstash manipulate json object before indexing to ES

**URL:** <https://discuss.elastic.co/t/logstash-manipulate-json-object-before-indexing-to-es/240721>\
**Category:** Logstash\
**Created:** [July 10, 2020, 3:17pm UTC](https://discuss.elastic.co/t/logstash-manipulate-json-object-before-indexing-to-es/240721 "2020-07-10T15:17:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![davehatma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davehatma/32/29322_2.png) [@davehatma](https://discuss.elastic.co/u/davehatma)\
**Post date:** [July 10, 2020, 3:17pm UTC](https://discuss.elastic.co/t/logstash-manipulate-json-object-before-indexing-to-es/240721/1 "2020-07-10T15:17:31Z")

</div>

A Kafka input plugin is getting messages in json format and being parsed by the json filter. One of the root keys -- transaction\_details -- can contain a very large number of fields that we don't want to flatten into discreet fields. Would like to be able to convert the transaction\_details root key from an object to a string that can be mapped as a text field and thus searchable as text but not selectable as a field in Kibana. We want to keep all the other json keys as they are when parsed. I can't seem to find a way (other than some kind of regular expression that seems very complicated and probably above my pay grade) to do this.

Sharing any thoughts would be most appreciated.

Thanks

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [July 10, 2020, 4:00pm UTC](https://discuss.elastic.co/t/logstash-manipulate-json-object-before-indexing-to-es/240721/2 "2020-07-10T16:00:40Z")

</div>

You could encode that as JSON again after parsing it. Or maybe mapping it as "flattened" in ES would make sense?

```auto
ruby {
    init => "require 'json'"
    code => "event.set('transaction_details', JSON.generate(event.get('transaction_details')))"
  }

```

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [July 12, 2020, 8:46am UTC](https://discuss.elastic.co/t/logstash-manipulate-json-object-before-indexing-to-es/240721/3 "2020-07-12T08:46:10Z")

</div>

Can you paste some samples data from kafka input ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2020, 8:46am UTC](https://discuss.elastic.co/t/logstash-manipulate-json-object-before-indexing-to-es/240721/4 "2020-08-09T08:46:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.