# Logstash remove\_field issues **URL:** \ **Category:** Logstash\ **Created:** [July 30, 2018, 1:27pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186 "2018-07-30T13:27:12Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [July 30, 2018, 1:27pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/1 "2018-07-30T13:27:12Z")
Hello, I am trying to remove some fields from Twitter plugin using Logstash filter. **Partial json from Kibana:** ```auto { "_index": "twitter", "_type": "tweets", "_id": "iedY62QB9sEBYW4Cw1c0", "_version": 1, "_score": null, "_source": { "user": { "followers_count": 5037, "profile_image_url_https": "https://pbs.twimg.com/profile_images/974284997396967430/BcsHO0dZ_normal.jpg", "protected": false, "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png", "default_profile": true, "translator_type": "none", "following": null, "listed_count": 3, "profile_use_background_image": true, "screen_name": "CryptoPartyBoun", "favourites_count": 1370, "profile_sidebar_border_color": "C0DEED", "profile_background_color": "C0DEED", "created_at": "Fri Dec 14 08:09:19 +0000 2012", "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png", "geo_enabled": true, "description": "#bitcoin #btc #ethereum #eth #blockchain #cryptocurrency #altcoin #ico #follow #followme #follow4follow #followback #подписка #ЧитаюВзаимно #ВзаимныйФолловинг", "profile_background_tile": false, "url": null, "statuses_count": 686, "profile_image_url": "http://pbs.twimg.com/profile_images/974284997396967430/BcsHO0dZ_normal.jpg", "friends_count": 4922, "contributors_enabled": false, "default_profile_image": false, "follow_request_sent": null, "id": 1010662458, "id_str": "1010662458", "utc_offset": null, "notifications": null, "location": "USA, Iowa", "is_translator": false, "profile_sidebar_fill_color": "DDEEF6", "profile_text_color": "333333", "verified": false, "time_zone": null, "name": "CryptoParty", "lang": "en", "profile_link_color": "1DA1F2" }, ``` **logstash filter expression:** ```auto filter { prune { whitelist_names => [ "^entities$", "^lang$", "^coordinates$", "^retweeted_status$", "^text$", "^extended_tweet$", "^@timestamp$", "^user$" ] } if [user] == "null" { mutate { remove_field => [ "[user][contributors_enabled]", "[user][created_at]", "[user][default_profile]", "[user][default_profile_image]", "[user][description]", "[user][favourites_count]", "[user][followers_count]", "[user][following]", "[user][follow_request_sent]", "[user][friends_count]", "[user][id]", "[user][is_translator]", "[user][listed_count]", "[user][location]", "[user][notifications]", "[user][profile_background_color]", "[user][profile_background_image_url]", "[user][profile_background_image_url_https]", "[user][profile_background_tile]", "[user][profile_image_url_https]", "[user][profile_link_color]", "[user][profile_sidebar_border_color]", "[user][profile_sidebar_fill_color]", "[user][profile_text_color]", "[user][protected]", "[user][screen_name]", "[user][statuses_count]", "[user][time_zone]", "[user][translator_type]", "[user][url]", "[user][utc_offset]", "[user][verified]" ] }} } ``` Logstash does not throw any error and works, ingesting data, but the remove\_field filter does not filter the fields at all (whitelist\_names filter works as expected). What am I doing wrong here? Should I pipe it differently? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 30, 2018, 1:29pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/2 "2018-07-30T13:29:56Z")
You only try to remove\_field if [user] is null. Did you mean that to be != perhaps? ---
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [July 30, 2018, 1:39pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/3 "2018-07-30T13:39:09Z")
You are right. Changed to !=, I still do not have this data removed. I have also tried without if statement before, but the result was the same. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 30, 2018, 2:40pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/4 "2018-07-30T14:40:47Z")
I don't know. It works for me. ``` "user" => { "profile_use_background_image" => true, "name" => "CryptoParty", "lang" => "en", "profile_image_url" => "http://pbs.twimg.com/profile_images/974284997396967430/BcsHO0dZ_normal.jpg", "id_str" => "1010662458", "geo_enabled" => true } ``` ---
**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Post date:** [July 30, 2018, 3:38pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/5 "2018-07-30T15:38:30Z")
Whats the result? Is it a some sort or debug? To what you were outputting to? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 30, 2018, 4:12pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/6 "2018-07-30T16:12:43Z")
That is from output { stdout { codec =\> rubydebug} } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 27, 2018, 4:12pm UTC](https://discuss.elastic.co/t/logstash-remove-field-issues/142186/7 "2018-08-27T16:12:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.