Remove fields that match regex

[M_B_I (Vladislav)]

Hi!
Could someone tell me , can I delete a few fields that match regex expression?

I get snmp traps and in my filter I set all information that I need to a specific fields, so I don't need anymore fields that starts with "SNMPv2-SMI::" How I can delete them all?

[Christian_Dahlqvist]

I believe you will need to use the ruby filter with some custom code for that.

[magnusbaeck (Magnus Bäck)]

Use a prune filter.

[M_B_I (Vladislav)]

So, if I have fields like
SNMPv2-SMI::enterprises.oid1
SNMPv2-SMI::enterprises.oid2
SNMPv2-SMI::enterprises.oid3

I will able to delete them with this :

prune
{
blacklist_names => [ "SNMPv2-SMI.*" ]
}

[M_B_I (Vladislav)]

@Christian_Dahlqvist could you please write an example of filter in this case?

[magnusbaeck (Magnus Bäck)]

So, if I have fields like
SNMPv2-SMI::enterprises.oid1
SNMPv2-SMI::enterprises.oid2
SNMPv2-SMI::enterprises.oid3

I will able to delete them with this :

Why don't you try it out? But you should start your regexp with ^ to only match fields that start with SNMPv2-SMI.

[M_B_I (Vladislav)]

Hi, Magnus for my sorry, I couldn't use prune filter, I have older version of logstash that 6.0

[magnusbaeck (Magnus Bäck)]

The prune filter has been around since Logstash 1.x. What problems were you having?

[M_B_I (Vladislav)]

My problem is bad knowledge of english .
Ok, I founded on github https://github.com/logstash-plugins/logstash-filter-prune/releases/tag/v3.0.3
the latest version of filter. But I couldn't install it.

[root@user]# /usr/share/logstash/bin/logstash-plugin install file:///home/mon/downloads/ELK/logstash-filter-prune-3.0.2.zip
Installing file: /home/mon/downloads/ELK/logstash-filter-prune-3.0.2.zip
**ERROR: Invalid pack for: file:///home/mon/downloads/ELK/logstash-filter-prune-3.0.2.zip, reason: The pack must contains at least one plugin, message: The pack must contains at least one plugin**

Maybe I downloaded wrong file?

Thanks for patience

[magnusbaeck (Magnus Bäck)]

Why not install the plugin via the logstash-plugin command, as described in the documentation?

[M_B_I (Vladislav)]

Cause I don't have network acess to artifacts.elastic.co

[M_B_I (Vladislav)]

Well that works for :

ruby
{
code =>
'event.to_hash.keys.each { |k| if k.start_with?("some text") then event.remove(k) end }'
}

[M_B_I (Vladislav)]

Well that works for :

ruby
{
code =>
'event.to_hash.keys.each { |k| if k.start_with?("some text") then event.remove(k) end }'
}