# Logstash whitelist nestet json objects, filter only specific fields

**URL:** <https://discuss.elastic.co/t/logstash-whitelist-nestet-json-objects-filter-only-specific-fields/151216>\
**Category:** Logstash\
**Created:** [October 5, 2018, 1:22pm UTC](https://discuss.elastic.co/t/logstash-whitelist-nestet-json-objects-filter-only-specific-fields/151216 "2018-10-05T13:22:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Petr.Simik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/petr.simik/32/38082_2.png) [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\
**Post date:** [October 5, 2018, 1:22pm UTC](https://discuss.elastic.co/t/logstash-whitelist-nestet-json-objects-filter-only-specific-fields/151216/1 "2018-10-05T13:22:29Z")

</div>

I cant filter out nestet json objects

I tried to implement [network packets analysis](https://www.elastic.co/blog/analyzing-network-packets-with-wireshark-elasticsearch-and-kibana).

The result has too-many fields and I want to filter out only messages having content "http\_http\_file\_data" and import only specific fields.

I tried approach with prune or if statement but it always ignore the conditions and imports everything into index

I need to import only selected fields for instance (see example below)  
and I need to import only the messages having http\_http\_file\_data != null

thank you

`\> input {  
file {  
path =\> "/home/user/pcap/packets3.json"  
start\_position =\> "beginning"  
ignore\_older =\> 0  
}}

filter {  
# Drop Elasticsearch Bulk API control lines  
if ([message] =~ "{"index") {  
drop {}  
}

```
json {
    source => "message"
    remove_field => "message"
}

# Extract innermost network protocol
grok {
    match => {
        "[layers][frame][frame_frame_protocols]" => "%{WORD:protocol}$"

    }
}
#this prune does not work and this config = all is blacklisted, any condition is not matched
prune {
whitelist_names => [ "timestamp",
"[layers][http][http_http_file_data]",
"[layers][http][http_authorization_http_authbasic]",
"[layers][http][http_http_host]",
"[layers][http][http_http_request_full_uri]", 
"[layers][ip][ip_ip_addr]" ]
}

date {
    match => ["timestamp", "UNIX_MS"]
}

```

}

output {  
elasticsearch {  
hosts =\> ["127.0.0.1:9200"]  
index =\> "user-pcap-test"  
}  
stdout { }  
}  
`

the JSON I simplified looks like this:  
{"timestamp" : "1538279088714", "layers" : {"frame": {},"eth": {},"ip": {"ip\_ip\_src": "10.1.1.2"},"tcp": {"tcp\_tcp\_port": "34908"},"http": {"http\_text": "HTTP/1.1 200 OK\r\n","http\_http\_file\_data": "\<?xml IremovedThisXmlData\>"},"xml": {}}}

---

<div class="post-metadata">

**Author:** ![Petr.Simik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/petr.simik/32/38082_2.png) [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\
**Post date:** [October 23, 2018, 4:37am UTC](https://discuss.elastic.co/t/logstash-whitelist-nestet-json-objects-filter-only-specific-fields/151216/2 "2018-10-23T04:37:42Z")

</div>

I resolved the problem with mutate . Created parsed json and mutate add fields, (whitelist like) . Finally remove source and parsed object.

```
1) 
filter {
    #create parsed_json from input message
    json {
        source => "message"
        target => "parsed_json"
         }

2) 
mutate {
        add_field => {"timestamp" => "%{[parsed_json][timestamp]}"}
        add_field => {"http_data" => "%{[parsed_json][layers][http][http_http_file_data]}"}
        add_field => {"ip_src" => "%{[parsed_json][layers][ip][ip_ip_src_host]}"}
        add_field => {"ip_dst" => "%{[parsed_json][layers][ip][ip_ip_dst_host]}"}
        add_field => {"frame_time" => "%{[parsed_json][layers][frame][frame_frame_time]}"}
       
        remove_field => ["json", "message"]
        remove_field => ["json", "parsed_json"]
     }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 20, 2018, 4:37am UTC](https://discuss.elastic.co/t/logstash-whitelist-nestet-json-objects-filter-only-specific-fields/151216/3 "2018-11-20T04:37:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.