# Logstash in k8s - parsing nested json from MongoDB and get every nested json as separated field

**URL:** <https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755>\
**Category:** Logstash\
**Created:** [August 27, 2023, 7:32am UTC](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755 "2023-08-27T07:32:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Denis\_Lezgin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denis_lezgin/32/117762_2.png) [@Denis\_Lezgin](https://discuss.elastic.co/u/Denis_Lezgin)\
**Post date:** [August 27, 2023, 7:32am UTC](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755/1 "2023-08-27T07:32:16Z")

</div>

Hi there,

I'm using Logstash to take documents from specific MongoDB collection, and save it to Elasticsearch.  
Nested fields are being saved to "log\_entry" as one JSON, starting with "BSON" or "ID", depends on manipulations I do using filter.

Here is example of the "log\_entry":

```auto
"log_entry": {\"_id\": \"122ghgh1111, \"msg_body\": {\"text_one\": 2, \"text_data\": [{\"position\": 1}, {....}, {...}]}}

```

there is a lot of text in log\_entry, so i don't post everything, just the structure.

Below is my config (ive tried different ways , so I'll share it all. None of them isn't doing what I'd like to achieve):

```auto
logstashPipeline:
   logstash.conf:
      input {
         mongodb {
            uri => 'mongodb://user:password@host:port/<db_name>?directConnection=true'
            placeholder_db_dir => '/opt/logstash-mongodb'
            placeholder_db_name => 'logstash_sqlite.db'
            collection => 'my_collection'
            codec => "json"
         }
      }

```

```auto
      // First try - still saving nested JSON as one

      filter {
         mutate {
            gsub => ["log_entry", "=>", ": "]
            rename => { "_id" => "mongo_id" }
            remove_filed => ["_id"]
         }
         mutate {
            gsub => ["log_entry", "BSON::ObjectID\('([0-9a-z]+'\)", '"\1']
            rename => { "_id" => "mongo_id" }
         }
      }

```

```auto
      // Second try - still saving nested JSON as one

      filter {
         mutate {
            gsub => ["log_entry", "=>", ": "]
            rename => { "_id" => "mongo_id" }
            remove_filed => ["_id"]
         }
         mutate {
            gsub => ["log_entry", "BSON::ObjectID\('([0-9a-z]+'\)", '"\1']
            rename => { "_id" => "mongo_id" }
         }
         grok {
            match => { "log_entry" => "%{DATA:log_entry}" }
         }
         json {
            source => "log_entry"
            remove_field => ["log_entry"]
         }
      }

```

```auto
      output { elasticsearch {
            action => "index"
            index => "mongo_log_data"
            hosts => ["https://<host>:9200"]
            ssl => false
            ssl_certificate_verification => false
            user => "elastic"
            password => "some_password"
         }
      }

```

Can you please help me to build a correct working filter to achieve what I need?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 27, 2023, 12:57pm UTC](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755/2 "2023-08-27T12:57:20Z")

</div>

You have told us that your existing filter does not do what you want, but have given no indication of what it is you want to do, which makes it hard to propose a solution. Can you provide an example of what an event looks like and what you would like it to look like?

---

<div class="post-metadata">

**Author:** ![Denis\_Lezgin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/denis_lezgin/32/117762_2.png) [@Denis\_Lezgin](https://discuss.elastic.co/u/Denis_Lezgin)\
**Post date:** [September 5, 2023, 7:11am UTC](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755/3 "2023-09-05T07:11:15Z")

</div>

Hi @Badger ,

This is what Im seeing in log\_entry:

```auto
"log_entry": {\"_id\": \"122ghgh1111, \"msg_body\": {\"text_one\": 2, \"text_data\": [{\"position\": 1}, {....}, {...}]}}

```

I would like it to be saved in ES as:

```auto

text_one: 2
text_data:_position: 1

etc

```

which means, each field of the nested JSON to be save as separated field.

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 3, 2023, 7:12am UTC](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755/4 "2023-10-03T07:12:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.