# How to extract key-value pairs automatically from mixed type of message

**URL:** <https://discuss.elastic.co/t/how-to-extract-key-value-pairs-automatically-from-mixed-type-of-message/247264>\
**Category:** Logstash\
**Created:** [September 2, 2020, 4:17pm UTC](https://discuss.elastic.co/t/how-to-extract-key-value-pairs-automatically-from-mixed-type-of-message/247264 "2020-09-02T16:17:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![kelk](https://avatars.discourse-cdn.com/v4/letter/k/13edae/32.png) [@kelk](https://discuss.elastic.co/u/kelk)\
**Post date:** [September 2, 2020, 4:17pm UTC](https://discuss.elastic.co/t/how-to-extract-key-value-pairs-automatically-from-mixed-type-of-message/247264/1 "2020-09-02T16:17:25Z")

</div>

hi  
We have a mixed set of message coming from filebeat. The message is some form of "YAML/json" mix

```auto
[WARN] 2020-09-02 16:03:55.966 [[pipline_id]>worker6] elasticsearch - Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"my_index", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x29cf370e>], :response=>{"index"=>{"_index"=>"my_index", "_type"=>"_doc", "_id"=>"qB6PT3QBbVYdMfr7deF1", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [error] tried to parse field [error] as object, but found a concrete value"}}}}

```

How to extract them into a hierarchy key-value basis automatically, without physically doing regex for each field?

So i'm looking for above message to be parsed and key-value's extracted as

```auto
message.status: 400
message.action.index._index: my_index
message.error.type: mapper_parsing_exception
...

```

Any idea how to do it?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 30, 2020, 4:17pm UTC](https://discuss.elastic.co/t/how-to-extract-key-value-pairs-automatically-from-mixed-type-of-message/247264/2 "2020-09-30T16:17:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.