# Create json object with logstash

**URL:** <https://discuss.elastic.co/t/create-json-object-with-logstash/158581>\
**Category:** Logstash\
**Created:** [November 28, 2018, 1:53pm UTC](https://discuss.elastic.co/t/create-json-object-with-logstash/158581 "2018-11-28T13:53:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shawcs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shawcs/32/45490_2.png) [@Shawcs](https://discuss.elastic.co/u/Shawcs)\
**Post date:** [November 28, 2018, 1:53pm UTC](https://discuss.elastic.co/t/create-json-object-with-logstash/158581/1 "2018-11-28T13:53:36Z")

</div>

Hi

I face some trouble with JSON logs. I have a big JSON message comming in my Logstash and I made some parse in my file to create a new clean message. But now I want to create nested JSON field as I can have an array of nested json in my input.

here is my example:

this is my incomming message:

```
{
"timestamp": 1543400113822,
"correlationId": "b16afe5bf51593fa597b0faf",
"processInfo": {
	"hostname": "host1",
	"domainId": "75c0688d-ffab-458a-8744-f01b154d27f0",
	"groupId": "group-13",
	"groupName": "group23",
	"serviceId": "instance-57",
	"serviceName": "Manager",
	"version": "v7.5-Internal"
},
"circuitPath": [{
		"policy": "API OAuth 2.0 Security Device",
		"execTime": 2,
		"filters": [{
				"name": "<anonymous>",
				"type": "ValidateOAuthAccessTokenFilter",
				"class": "com.vordel.circuit.oauth.provider.ValidateOAuthAccessTokenFilter",
				"status": "Fail",
				"filterTime": 1543400113821,
				"execTime": 2
			}
		]
	}, {
		"policy": "API Broker",
		"execTime": 0,
		"filters": [{
				"name": "Set service context",
				"type": "ApiServiceContextFilter",
				"class": "com.vordel.coreapireg.runtime.broker.ApiServiceContextFilter",
				"status": "Pass",
				"filterTime": 1543400113821,
				"execTime": 0
			}, {
				"name": "Authentication Failure",
				"type": "ApiShuntFilter",
				"class": "com.vordel.coreapireg.runtime.broker.ApiShuntFilter",
				"status": "Fail",
				"filterTime": 1543400113821,
				"execTime": 0
			}
		]
	}
]
}

```

This is my logstash configuration

```
	json {
					source => "message"
					target => "json"
			}

			date {
					match => ["[json][timestamp]", "UNIX_MS"]
					target => "@timestamp"
			}
				mutate {
					add_field => {
							"api_correlationId" => "%{[json][correlationId]}"
							"api_hostname" => "%{[json][processInfo][hostname]}"
							"api_instance" => "%{[json][processInfo][groupName]}"
							"api_service_name" => "%{[json][processInfo][serviceName]}"
					}
			}

if ("circuitPath" in [json]) {
					split {
							field => "[json][circuitPath]"
					}

					mutate {
							add_field => {
							
#my goal here is to recreate the filters objects with only the following field in a nested object
				
									"[api_policy_name]" => "%{[json][circuitPath][policy]}"
									"[api_policy_Exec_Time]" => "%{[json][circuitPath][execTime]}"
									
									
									"[api_filter_name]" => "%{[json][circuitPath][N][filters][N][name]}"
									"[api_filter_type]" => "%{[json][circuitPath][N][filters][N][type]}"
									"[api_filter_status]" => "%{[json][circuitPath][N][filters][N][status]}"
									"[api_filter_exec_time]" => "%{[json][circuitPath][N][filters][N][execTime]}"

							}
					}
					mutate {
						remove_field => ["message"]
					}
			}	

```

So at the end my json should look like this:

```
{
api_correlationId
api_hostname
api_instance
api_service_name
circuit: [{
    api_policy_name:"API OAuth 2.0 Security Device"
    api_policy_Exec_Time:"2"
     [{
        api_filter_name:"anonymous"
        api_filter_type:"ValidateOAuthAccessTokenFilter"
        api_filter_status:"Fail"
        api_filter_exec_time:"2"
     }]
},
{
    api_policy_name:"API Broker"
    api_policy_Exec_Time:"0"
     [{
         api_filter_name:"Set service context"
         api_filter_type:"ApiServiceContextFilter"
         api_filter_status:"Pass"
         api_filter_exec_time:"0"
      },
     {
         api_filter_name:"Authentication Failure"
         api_filter_type:"ApiShuntFilter"
         api_filter_status:"Fail"
         api_filter_exec_time:"0"
   }]
}]
}

```

I can't find a way of doing it.

---

<div class="post-metadata">

**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\
**Post date:** [November 28, 2018, 2:55pm UTC](https://discuss.elastic.co/t/create-json-object-with-logstash/158581/2 "2018-11-28T14:55:11Z")

</div>

And what is the result you are currently getting?

---

<div class="post-metadata">

**Author:** ![Shawcs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shawcs/32/45490_2.png) [@Shawcs](https://discuss.elastic.co/u/Shawcs)\
**Post date:** [November 29, 2018, 10:42am UTC](https://discuss.elastic.co/t/create-json-object-with-logstash/158581/3 "2018-11-29T10:42:07Z")

</div>

Here is the configuration I finaly used

```
if ("circuitPath" in [json]) {
					split {
							field => "[json][circuitPath]"
					}

					mutate {
							add_field => {
							
									"[api_policy_name]" => "%{[json][circuitPath][policy]}"
									"[api_policy_Exec_Time]" => "%{[json][circuitPath][execTime]}"
							}
					}
					
					mutate {
							add_field => {
									#search for here		
									"[api_filter_name]" => "%{[json][circuitPath][filters][0][name]}"
									"[api_filter_type]" => "%{[json][circuitPath][filters][0][type]}"
									"[api_filter_status]" => "%{[json][circuitPath][filters][0][status]}"
									"[api_filter_exec_time]" => "%{[json][circuitPath][filters][0][execTime]}"

							}
					}
			}	

```

With this log in input :

```
 {
"timestamp": 1543487516616,
"correlationId": "1cc0ff5b3284091b4c191146",
"processInfo": {
	"hostname": "host1",
	"domainId": "75c0688d-ffab-458a-8744-f01b154d27f0",
	"groupId": "group-13",
	"groupName": "Manager1",
	"serviceId": "instance-57",
	"serviceName": "ManagerLow",
	"version": "v7.5.3-Internal"
},
"circuitPath": [{
		"policy": "API Broker",
		"execTime": 0,
		"filters": [{
				"name": "Not Found",
				"type": "ApiShuntFilter",
				"class": "com.vordel.coreapireg.runtime.broker.ApiShuntFilter",
				"status": "Fail",
				"filterTime": 1543487516614,
				"execTime": 0
			}, {
				"name": "Not Found",
				"type": "ApiShuntFailureFilter",
				"class": "com.vordel.coreapireg.runtime.broker.ApiShuntFailureFilter",
				"status": "Pass",
				"filterTime": 1543487516615,
				"execTime": 0
			}
		]
	}
]
 }

```

I got:

```
{
"timestamp": 1543487516616,
"api_correlationId": "1cc0ff5b3284091b4c191146",
"host": "host1",
"api_hostname": "Manager1",
"api_instance": "ManagerLow",
"api_policy_Exec_Time": "API Broker",
"api_filter_exec_time": 0,
"api_filter_name": "Not Found",
"api_filter_type": "ApiShuntFilter",
"api_filter_status": "Fail",
"api_filter_exec_time": 0	
}

```

So I can only take my first element

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 27, 2018, 10:42am UTC](https://discuss.elastic.co/t/create-json-object-with-logstash/158581/4 "2018-12-27T10:42:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.