### Author:  [@rakesh15](https://discuss.elastic.co/u/rakesh15)
#### Post date: [August 17, 2020, 9:27am UTC](https://discuss.elastic.co/t/json-parsing-depth-in-logstash/245205/1 "2020-08-17T09:27:41Z")
Hi All,
Im trying to index the camel logs by using grok pattern and json filter in logstash and then push the same to ES.
Below is the sample log snippet that I'm trying to parse.
```auto
07:32:49.805 [Camel (MyCamel) thread #3 - seda://fetchSFDCRetailLogs] INFO loggerRoute - {"timestamp":"17-2020-08 07-32-49","processName":"EmailValidation","serviceName":"SFDCRETAIL_FE_ADAPTER","targetSystem":"CREDITVIDYA_BE_ADAPTER","messageType":"SourceRequest","username":"SFDCRetail","conversationId":"123345678","transactionId":"ID-sfdcretailfrontendadapter-263-8wsrv-1597305746529-0-4876","headerDetails":{"authorization":"Basic ODY5Y2FjZDk6c2ZkY3JldGFpbHVhdA==","conversationId":"123345678","sourceName":"SFDCRetail","recipient":"","otpRefNo":""},"payload":{
"firstName": "Jigar",
"middleName": "Nareshbhai",
"lastName": "Shah",
"pfNumber": "",
"designation": "",
"companyName": "UNION BANK OF INDIA",
"city": "Ahmedabad",
"uniqueId": "",
"officeStateCode": "",
"officeAddressLine1": "",
"email": "THusena.prj@tatacapital.com",
"clientReference": {
"transactionId": "",
"applicantType": "1",
"losId": "6059185",
"applicantId": "CUST11056089",
"loanType": "Personal Loan",
"leadId": "",
"webtopNumber": "272PZ0001137",
"sourceSystem": "sfdc_retail"
},
"officeAddressLine3": "",
"income": "34235",
"officeAddressLine2": "",
"imeiNo": "",
"iPAddress": "",
"mobileNumber": ""
},"status":""}
```
Below is the logstash configuration file configured for the same.
```auto
input{
beats{
port => 5044
}
}
filter{
if [kubernetes][namespace] == "dev"{
grok{
match => ["message", "%{TIME:timestamp:date} %{GREEDYDATA:Thread} %{WORD:LoggingLevel} %{WORD:RouteName} - (?(.|\r|\n)*)"]
}
json{
source => "logmessage"
target => "doc"
}
if "_grokparsefailure" in [tags]{
drop {}
}
}
}
output{
if [kubernetes][namespace] == "dev"{
elasticsearch{
hosts => ["http://ipaddress:port"]
index => "esblogs-%{+YYYY.MM.dd}"
user => "elastic"
password => "password"
}
}
}
```
I'm able to parse the logs using the grok pattern and json filter using the logstash conf file. But this creates all the fields that are part of the above json request.
What I would like to achieve is the json filter should parse logs only for the depth of 1. It should not create fields that are part of "payload" tag in json request.
Can some one please guide me on how to achieve the same.
Thanks and Regards,
Rakesh Chhabria
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 17, 2020, 2:25pm UTC](https://discuss.elastic.co/t/json-parsing-depth-in-logstash/245205/2 "2020-08-17T14:25:23Z")
I cannot think of a way to avoid parsing the JSON, but you can use
```
mutate { remove_field => ["payload"] }
```
to delete it once it has been parsed.
---
### Author:  [@rakesh15](https://discuss.elastic.co/u/rakesh15)
#### Post date: [August 18, 2020, 3:40pm UTC](https://discuss.elastic.co/t/json-parsing-depth-in-logstash/245205/3 "2020-08-18T15:40:24Z")
Hi Badger,
Thanks for your reply, but since payload contains ESB Logs, I cannot remove the field.
Do you have any pointers on how to add the below setting in logstash conf file mentioned above for all the new indices that being created daily.
```auto
index.mapping.depth.limit
```
Below is the link for the same.
[https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html#mapping-limit-settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html#mapping-limit-settings)
Looks like this can solve my issue.
Thanks and Regards,
Rakesh Chhabria
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 18, 2020, 4:11pm UTC](https://discuss.elastic.co/t/json-parsing-depth-in-logstash/245205/4 "2020-08-18T16:11:57Z")
You would not do that in logstash, it is an elasticsearch setting. You may be able to set it in an index template, but I am not sure of that.
---
### Author:  [@rakesh15](https://discuss.elastic.co/u/rakesh15)
#### Post date: [August 20, 2020, 11:02am UTC](https://discuss.elastic.co/t/json-parsing-depth-in-logstash/245205/5 "2020-08-20T11:02:37Z")
Hi Badger,
Thanks for providing the pointer to remove the payload field and it worked for me.
Since I was logging the entire message as part of "LogMessage" field I considered removing the [doc][payload] field.
Thanks and Regards,
Rakesh Chhabria
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 17, 2020, 11:02am UTC](https://discuss.elastic.co/t/json-parsing-depth-in-logstash/245205/6 "2020-09-17T11:02:38Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.