# How extract specific fields from JSON input with Logstash filters?

**URL:** https://discuss.elastic.co/t/how-extract-specific-fields-from-json-input-with-logstash-filters/173066
**Category:** Logstash
**Created:** [March 20, 2019, 1:54am UTC](https://discuss.elastic.co/t/how-extract-specific-fields-from-json-input-with-logstash-filters/173066 "2019-03-20T01:54:03Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![mdobrin](https://avatars.discourse-cdn.com/v4/letter/m/f0a364/32.png) [@mdobrin](https://discuss.elastic.co/u/mdobrin)
#### Post date: [March 20, 2019, 1:54am UTC](https://discuss.elastic.co/t/how-extract-specific-fields-from-json-input-with-logstash-filters/173066/1 "2019-03-20T01:54:03Z")

</div>

I am currently trying to parse a JSON string using Logstash for sending to an Elastic Search DB. The input string looks like the following:

> ```
> "message": """{"resourceId":"201987-20190320-05307201","body":{"orderNumber":"201987-20190320-05307201","wrapping1":{"name":null,"price":null,"taxFlag":null},"wrapping2":{"name":null,"price":null,"taxFlag":null},"package":{"608738":{"goodsTax":222,"postagePrice":0,"deliveryPrice":0,"sender":{"isOrderer":true,"familyName":"Smith",action":"CREATE","timestampMs":1553044764100}"""
> 
> ```

I am trying a few different methods for extracting the "resourceId" field while removing all others, but have not been successful. I receive an error that reads:

> ```
> Could not index event to Elasticsearch.
> ...
> Limit of total fields [5000] in index [test2-rt_platform] has been exceeded
> 
> ```

My settings are as follows:

> ```
> input {
> kafka {
> # Target servers
> bootstrap_servers => "myMachine:9002"
> 
> # Topic and consumer settings
> topics => ["myTopic"]
> group_id => "theConsumer"
> consumer_threads => 1
> 
> decorate_events => true
> 
> # Output format settings
> codec => json
> 
> # Performance settings
> auto_commit_interval_ms => "10000"
> auto_offset_reset => "latest"
> request_timeout_ms => "7000"
> session_timeout_ms => "6000"
> heartbeat_interval_ms => "2000"
> poll_timeout_ms => 2000
> retry_backoff_ms => "1000"
> max_partition_fetch_bytes => "10485760"
> 
> }
> }
> 
> filter {
> mutate {
> add_field => {
> "[@metadata][index]" => "platform_test_%{[kafka][topic]}"
> "[@metadata][format]" => "%{[kafka][topic]}"
> }
> }
> json {
> source => "message"
> }
> 
> mutate {
> add_field => {
> "orderNumber22" => "%{[body][orderNumber]}"
> }
> 
> add_field => {
> "orderNumber33" => ["[body][orderNumber]" ]
> }
> add_field => {
> "orderNumber44" => "%{[message][body][orderNumber]}"
> }
> 
> add_field => {
> "orderNumber55" => ["[message][body][orderNumber]" ]
> }
> remove_field => ["[message]" ]
> }
> 
> fingerprint {
> method => "MD5"
> key => "%{[kafka][topic]}"
> target => "[@metadata][fingerprint]"
> }
> }
> 
> output {
> elasticsearch {
> action => "index"
> flush_size => 100
> document_id => "%{[@metadata][fingerprint]}"
> document_type => "%{[@metadata][format]}"
> hosts => ["myMachine:9002"]
> index => "test2-rt_platform"
> retry_max_interval => 5
> timeout => 10000
> user => "myUser"
> password => "myPassword"
> }
> stdout {
> codec => rubydebug { metadata => true }
> }
> 
> }
> 
> ```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 17, 2019, 1:54am UTC](https://discuss.elastic.co/t/how-extract-specific-fields-from-json-input-with-logstash-filters/173066/2 "2019-04-17T01:54:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
