# How to disable parsing of nested filelds

**URL:** <https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227>\
**Category:** Logstash\
**Created:** [August 20, 2018, 7:04pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227 "2018-08-20T19:04:55Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jibsonline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jibsonline/32/63154_2.png) [@jibsonline](https://discuss.elastic.co/u/jibsonline)\
**Post date:** [August 20, 2018, 7:04pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/1 "2018-08-20T19:04:55Z")

</div>

Below is my logstash configuration

```
input {
    beats {
            port => 5001
    }

    kafka {
            bootstrap_servers => "kafka-xxxxxxxxxxxxxxxx:9092"
            topics => ["java-prod", "java-prod-cron"]
            codec => "json"
            tags => "java-prod"
            consumer_threads => "20"
            decorate_events => true
}
}

filter {
  mutate {
gsub => [
  "message", "^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} ", ""
]
  }
  grok {
            match => { "message" => "\[%{DATA:thread}\]\s+%{LOGLEVEL:severity}\s+%{DATA:class}\s+%{DATA:linenum}\s+-\s%{GREEDYDATA:log}" }
   }

 if [message] =~ /http.wire-log.writeResponse/ {
 mutate {
add_field => { "http_type" => "response" }
  }
   json {
    source => "log"
    target => "response"
    skip_on_invalid_json => true
  }
  mutate {
convert => { "response.body" => "string" }
  }
  }

if [message] =~ /http.wire-log.writeRequest/ {
  mutate {
add_field => { "http_type" => "request" }
  }
   json {
    source => "log"
    target => "request"
    skip_on_invalid_json => true
  }
  }
}

output {

    stdout {
            #codec => rubydebug
            codec => json
    }
    if [@metadata][kafka][topic] == "java-prod" {
   elasticsearch {
            hosts => "elasticsearch-xxxxxxxxxxxxxxxx:9200"
            index => "java-prod-%{+YYYY.MM.dd}"
            manage_template => true
            template => '/etc/logstash/templates/java-prod.json'
            template_name => 'java'
            template_overwrite => true
    }
    }
    else if [@metadata][kafka][topic] == "java-prod-cron" {
    elasticsearch {
            hosts => "elasticsearch-xxxxxxxxxxxxxxxx:9200"
            index => "java-cron-prod-%{+YYYY.MM.dd}"
            manage_template => true
            template => '/etc/logstash/templates/java-prod.json'
            template_name => 'java'
            template_overwrite => true
    }
    }
}

```

The issue I am facing is that `response.body` contains json response which has nested json that has depth of 100 and more. I dont want those to be indexed as fields. How do I avoid parsing of `response.body`. I tried

```
mutate {
    convert => { "response.body" => "string" }
      }

```

But it doesnt seem to work. Tried json\_encode too. But that too didnt work. How can I handle it to consider as normal text and dont parse it as fields?

---

<div class="post-metadata">

**Author:** ![jibsonline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jibsonline/32/63154_2.png) [@jibsonline](https://discuss.elastic.co/u/jibsonline)\
**Post date:** [August 20, 2018, 7:15pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/2 "2018-08-20T19:15:09Z")

</div>

In short I want logstash to treat response.body as a string and not an json object

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 20, 2018, 8:17pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/3 "2018-08-20T20:17:01Z")

</div>

Try using `[response][body]` which is the correct way to address that nested field.

[https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references)

---

<div class="post-metadata">

**Author:** ![jibsonline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jibsonline/32/63154_2.png) [@jibsonline](https://discuss.elastic.co/u/jibsonline)\
**Post date:** [August 21, 2018, 4:30am UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/4 "2018-08-21T04:30:01Z")

</div>

This is how my configuration is now

```
json {
        source => "log"
        target => "response"
        skip_on_invalid_json => true
      }

   mutate {
    convert => { "[response][body]" => "string" }
  }

```

But I still get this error

```
[2018-08-21T09:52:18,527][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"java-prod-2018.08.21", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x68ff20cb>], :response=>{"index"=>{"_index"=>"java-prod-2018.08.21", "_type"=>"doc", "_id"=>"6iq4WmUBom8sbLqrFsN_", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [response.body.data.status]", "caused_by"=>{"type"=>"json_parse_exception", "reason"=>"Current token (VALUE_TRUE) not numeric, can not use numeric value accessors\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@5cc6fffc; line: 1, column: 1458]"}}}}}

```

Its still trying to parse `response.body.data.status`

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2018, 7:08am UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/5 "2018-08-21T07:08:16Z")

</div>

I'm not sure the mutate convert option works as you expect. I suggest you use the json\_encode filter instead.

Secondly I strongly recommend you use a `stdout { codec => rubydebug }` output rather than an elasticsearch output while you're debugging this. When the event looks and desired you can refocus your effort on ES.

---

<div class="post-metadata">

**Author:** ![jibsonline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jibsonline/32/63154_2.png) [@jibsonline](https://discuss.elastic.co/u/jibsonline)\
**Post date:** [August 21, 2018, 1:40pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/6 "2018-08-21T13:40:55Z")

</div>

The json\_encode seems to working. One doubt. What if a string comes once in a while as response.body. How do I handle it? Can I write a if condition?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2018, 1:47pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/7 "2018-08-21T13:47:45Z")

</div>

I suppose the json\_encode filter adds a tag to the event.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 18, 2018, 1:47pm UTC](https://discuss.elastic.co/t/how-to-disable-parsing-of-nested-filelds/145227/8 "2018-09-18T13:47:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.