# Logstash - JSON Filter to get inner json fields

**URL:** <https://discuss.elastic.co/t/logstash-json-filter-to-get-inner-json-fields/231022>\
**Category:** Logstash\
**Created:** [May 4, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-json-filter-to-get-inner-json-fields/231022 "2020-05-04T19:20:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rekha\_Dhinagaran](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rekha_dhinagaran/32/64349_2.png) [@Rekha\_Dhinagaran](https://discuss.elastic.co/u/Rekha_Dhinagaran)\
**Post date:** [May 4, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-json-filter-to-get-inner-json-fields/231022/1 "2020-05-04T19:20:22Z")

</div>

Hi,

This is my event log,

`{'_index': 'logstash-2020.05.03', '_type': '_doc', '_id': 'xXKW2XEB43wd9fLrXQnQ', '_score': 1.0, '_source': {'@version': '1', 'beat': {'version': '6.2.4', 'name': 'cb-rekha.local', 'hostname': 'cb-rekha.local'}, '@timestamp': '2020-05-03T08:10:40.400Z', 'thread': 'Thread-2', 'timestamp': '2020-05-02 22:20:56.155', 'level': 'INFO', 'message': 'Thread 13 is running', 'offset': 4352, 'context': 'default', 'mdc': {'cardNo': '123456789012934', 'domain': 'Mytest-domain', 'ThreadNumber': '6'}, 'source': '/Users/cb-rekha/logging-poc/logs/app1.json', 'logger': 'xxxxxxxx'}}`

I'm trying to parse only the "\_source" in this JSON and also to remove certain fields under \_source and then send it to Elastic search.

I want to post the document in that specific index and document id from the input event so that it would update already existing documents in ES.

Below is my logstash conf file.

```auto
input {
     kafka {
         bootstrap_servers => "localhost:9092"
         topics => ["esToKafka"]
   }
 }

 filter {
  json {
    remove_field => ["@version", "@timestamp"]
    source => "message"
    target => "myjson"
    remove_field => "message"
   }
 # json {
 # source => "_source"
 # remove_field => ["@beat", "@version", "@timestamp"]
 # }
 # prune { whitelist_names => ["_source"] 
 # }
  # json_encode { 
  # source => "_source"
  # remove_field => ["_score", "_type"]
  # } 
  }
 output {
  elasticsearch { 
  hosts => ["localhost:9200"]
  index => "%{_index}"
  document_id => "%{_id}"
  user => "admin"
  password => "admin"
  ssl => true
  ssl_certificate_verification => false
  ilm_enabled => false
  }
  stdout { }
  }

```

I tried out different filters but not able to get the source field and the value for index and document\_id correctly. Someone please help 🙂 thanks in advance.

---

<div class="post-metadata">

**Author:** ![InfiniteLoops](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/infiniteloops/32/47144_2.png) [@InfiniteLoops](https://discuss.elastic.co/u/InfiniteLoops)\
**Post date:** [May 5, 2020, 7:14pm UTC](https://discuss.elastic.co/t/logstash-json-filter-to-get-inner-json-fields/231022/2 "2020-05-05T19:14:12Z")

</div>

I think you need to set `decorate_events => true` in your input. Then you can access these fields with something like `[@metadata][_index]`

I haven't used this particular plugin so someone might correct me but was trying to do something similar with another setup and that seems to be the equivalent way in your setup.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 2, 2020, 7:14pm UTC](https://discuss.elastic.co/t/logstash-json-filter-to-get-inner-json-fields/231022/3 "2020-06-02T19:14:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.