# Logstash adds fields to JSON by parsing another field

**URL:** <https://discuss.elastic.co/t/logstash-adds-fields-to-json-by-parsing-another-field/84032>\
**Category:** Logstash\
**Created:** [April 28, 2017, 3:42pm UTC](https://discuss.elastic.co/t/logstash-adds-fields-to-json-by-parsing-another-field/84032 "2017-04-28T15:42:20Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Li\_Cao](https://avatars.discourse-cdn.com/v4/letter/l/b9bd4f/32.png) [@Li\_Cao](https://discuss.elastic.co/u/Li_Cao)\
**Post date:** [April 28, 2017, 3:42pm UTC](https://discuss.elastic.co/t/logstash-adds-fields-to-json-by-parsing-another-field/84032/1 "2017-04-28T15:42:20Z")

</div>

Hi!

I am trying to set up a pipeline to ingest some logs from Kafka cluster using LogStash and into elasticsearch.

The message on Kafka is a JSON string like:

{"a"="123", "b"="456", msg="this is a test"}

Sometimes, when the msg field contains a certain String, it has some information that I need to parse to add the JSON itself so that it can be indexed in elasticSearch. The number of fields in the msg field is dynamic.

{"a"="123", "b"="456", msg="HEATLTH\_CHECK CPU=4 LOAD=123"}

I want to transform it to  
{"a"="123", "b"="456", msg="HEATLTH\_CHECK CPU=4 LOAD=123", "CPU"=4, "LOAD"=123}

I looked at GROK but am not sure if it can be done because the field name like CPU and LOAD may change and number of fields can change too. The only unchanged part is the keyword HEALTH\_CHECK and space-delimited format and the field name won't conflict with the "outer" fields like "a" and "b"

Anyone has similar issue? I am willing to write or customize a bit of Ruby code (I heard logstash plugins are in Ruby) if needed.

Thanks!

Li

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 28, 2017, 4:18pm UTC](https://discuss.elastic.co/t/logstash-adds-fields-to-json-by-parsing-another-field/84032/2 "2017-04-28T16:18:59Z")

</div>

Apply a grok filter to the `msg` field and have it extract the key-value list into a separate field. Then use the [kv filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html) to parse this,

---

<div class="post-metadata">

**Author:** ![Li\_Cao](https://avatars.discourse-cdn.com/v4/letter/l/b9bd4f/32.png) [@Li\_Cao](https://discuss.elastic.co/u/Li_Cao)\
**Post date:** [April 28, 2017, 5:28pm UTC](https://discuss.elastic.co/t/logstash-adds-fields-to-json-by-parsing-another-field/84032/4 "2017-04-28T17:28:41Z")

</div>

Thank you! I will try it out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 26, 2017, 5:43pm UTC](https://discuss.elastic.co/t/logstash-adds-fields-to-json-by-parsing-another-field/84032/5 "2017-05-26T17:43:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.