# Remove fields that match regex

**URL:** <https://discuss.elastic.co/t/remove-fields-that-match-regex/132830>\
**Category:** Logstash\
**Created:** [May 22, 2018, 1:17pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830 "2018-05-22T13:17:28Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [May 22, 2018, 1:17pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/1 "2018-05-22T13:17:28Z")

</div>

Hi!  
Could someone tell me , can I delete a few fields that match regex expression?

I get snmp traps and in my filter I set all information that I need to a specific fields, so I don't need anymore fields that starts with "SNMPv2-SMI::" How I can delete them all?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 22, 2018, 1:20pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/2 "2018-05-22T13:20:19Z")

</div>

I believe you will need to use the [ruby filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-ruby.html) with some custom code for that.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 22, 2018, 1:22pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/3 "2018-05-22T13:22:40Z")

</div>

Use a prune filter.

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [May 22, 2018, 1:47pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/4 "2018-05-22T13:47:29Z")

</div>

So, if I have fields like  
SNMPv2-SMI::enterprises.oid1  
SNMPv2-SMI::enterprises.oid2  
SNMPv2-SMI::enterprises.oid3

I will able to delete them with this :

prune  
{  
blacklist\_names =\> ["SNMPv2-SMI.\*"]  
}

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [May 22, 2018, 2:50pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/5 "2018-05-22T14:50:13Z")

</div>

@Christian_Dahlqvist could you please write an example of filter in this case?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 22, 2018, 7:45pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/6 "2018-05-22T19:45:37Z")

</div>

> So, if I have fields like  
> SNMPv2-SMI::enterprises.oid1  
> SNMPv2-SMI::enterprises.oid2  
> SNMPv2-SMI::enterprises.oid3
> 
> I will able to delete them with this :

Why don't you try it out? But you should start your regexp with `^` to only match fields that _start_ with SNMPv2-SMI.

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [May 24, 2018, 12:43pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/7 "2018-05-24T12:43:54Z")

</div>

Hi, Magnus for my sorry, I couldn't use prune filter, I have older version of logstash that 6.0

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 24, 2018, 2:00pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/8 "2018-05-24T14:00:11Z")

</div>

The prune filter has been around since Logstash 1.x. What problems were you having?

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [May 24, 2018, 2:29pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/9 "2018-05-24T14:29:09Z")

</div>

My problem is bad knowledge of english 😕.  
Ok, I founded on github [https://github.com/logstash-plugins/logstash-filter-prune/releases/tag/v3.0.3](https://github.com/logstash-plugins/logstash-filter-prune/releases/tag/v3.0.3)  
the latest version of filter. But I couldn't install it.

_[root@user]# /usr/share/logstash/bin/logstash-plugin install file:///home/mon/downloads/ELK/logstash-filter-prune-3.0.2.zip_  
_Installing file: /home/mon/downloads/ELK/logstash-filter-prune-3.0.2.zip_  
_\*\*ERROR: Invalid pack for: file:///home/mon/downloads/ELK/logstash-filter-prune-3.0.2.zip, reason: The pack must contains at least one plugin, message: The pack must contains at least one plugin_\*\*

Maybe I downloaded wrong file?

Thanks for patience 😉

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 24, 2018, 2:43pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/10 "2018-05-24T14:43:17Z")

</div>

Why not install the plugin via the logstash-plugin command, as described in the documentation?

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [May 24, 2018, 2:46pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/11 "2018-05-24T14:46:51Z")

</div>

Cause I don't have network acess to [artifacts.elastic.co](http://artifacts.elastic.co)

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [June 14, 2018, 3:12pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/12 "2018-06-14T15:12:02Z")

</div>

Well that works for :

ruby  
{  
code =\>  
'event.to\_hash.keys.each { |k| if k.start\_with?("some text") then event.remove(k) end }'  
}

---

<div class="post-metadata">

**Author:** ![M\_B\_I](https://avatars.discourse-cdn.com/v4/letter/m/c37758/32.png) [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\
**Post date:** [June 14, 2018, 3:14pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/13 "2018-06-14T15:14:49Z")

</div>

Well that works for :

ruby  
{  
code =\>  
'event.to\_hash.keys.each { |k| if k.start\_with?("some text") then event.remove(k) end }'  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2018, 3:14pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830/14 "2018-07-12T15:14:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
