# Parse json Array input

**URL:** <https://discuss.elastic.co/t/parse-json-array-input/70194>\
**Category:** Logstash\
**Created:** [December 29, 2016, 8:40am UTC](https://discuss.elastic.co/t/parse-json-array-input/70194 "2016-12-29T08:40:15Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mehdi-aouadi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehdi-aouadi/32/13644_2.png) [@mehdi-aouadi](https://discuss.elastic.co/u/mehdi-aouadi)\
**Post date:** [December 29, 2016, 8:40am UTC](https://discuss.elastic.co/t/parse-json-array-input/70194/1 "2016-12-29T08:40:15Z")

</div>

I am retrieving some json data from a REST API ussing the `http_poller` input pluging :

```
http_poller {
  
  urls => {
     "myurl" => "https://myAPI"
  }
  interval => 30
  type => "myType"
  add_field => {
     "tag" => "myTag"
  }

```

}

This returns a json formatted data :  
`{"data_from_cache": false, "logs": [{"protocol": "PESIT", "processed": false }]}`

I need this data to be indexed in `elasticsearch` and I already prepared a mapping :

```
{
    "my_mapping" : {
      "properties" : {
         "protocol" : { "type": "string" },
         "processed" : { "type": "boolean" },  		
         "tag" : { "type" : "String" }
        }
    }
}

```

The data is not wrapped like the mapping. This configuration put the data like the following in `elasticsearch` :

```
data_from_cache: "false"
logs: "{ "protocol": "PESIT", "processed": false} "

```

I need the fields "protocol" and "processed" to be mapped as mentioned in separate fields. I do not need the data\_from\_cache field, I just need the data within logs in separate fields. How can I do that ? Should I use a json filter or a json codec ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 31, 2016, 10:30am UTC](https://discuss.elastic.co/t/parse-json-array-input/70194/2 "2016-12-31T10:30:23Z")

</div>

It's unclear exactly what the data is stored like in Elasticsearch (use a `stdout { codec => rubydebug }` output to make thing unambiguous), but you may have to add `codec => json` to your http\_poller input. Additionally you need a mutate filter that renames the `protocol` and `processed` subfields to the top level and deletes the undesired `data_from_cache` field.

---

<div class="post-metadata">

**Author:** ![mehdi-aouadi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehdi-aouadi/32/13644_2.png) [@mehdi-aouadi](https://discuss.elastic.co/u/mehdi-aouadi)\
**Post date:** [January 2, 2017, 1:51pm UTC](https://discuss.elastic.co/t/parse-json-array-input/70194/3 "2017-01-02T13:51:03Z")

</div>

I already put a `json codec` . Finally I used a `split` filter to split the json data to fields and a `mutate` filter to delete the unused ones and rename the others in order to remove parent.child names of the fields (logs.protocol and logs.reprocessed). Here is my final config :

```
input {
  http_poller {      
		urls => {
		  "myurl" => "https://myRestAPIurl"
		}
		interval => 30
		type => "mytype"
		add_field => {
			"tag" => "myTag"
		 }
		 codec => "json" 
	}
}
filter {
 if [tag] == "myTag" {
		split {
			field => "logs"
		}
		mutate { 
			remove_field => ["data_from_cache"]
		}
		mutate {
			rename => { "[logs][protocol]" => "protocol" }
			rename => { "[logs][reprocessed]" => "reprocessed" }
		}
    }
output {
if [tag] == "myTag" {
		elasticsearch {
			hosts => ["localhost:9200"]
			index => "myIndex"
			document_type => "myType"
		}	
	}      
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 30, 2017, 1:51pm UTC](https://discuss.elastic.co/t/parse-json-array-input/70194/4 "2017-01-30T13:51:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.