How to dynamic parse fieldname and replace?

Elastic Stack Logstash
1

Hello,

I am trying data that looks like this

unwanted.field1
unwanted.field2
categories.colors.blue.t.type

First I need to remove every unwanted field, then I want to change categories.colors to only blue.t.type.

I can use a kv filter for to remove unwanted fileds, but I am unable to find an easy way to remove
catories.colors dynamically

I want

unwanted.field1
unwanted.field2
categories.colors.t.blue.type
categories.colors.t.red.type

to become
t.blue.type
t.red.type

My code

//ruby {
code => "
begin
keys = event.to_hash.keys
keys.each{|key|

if ( key =~ /m./ )

         if key.start_with? '[categories][colors]'
          #I don't know what should go in the event.set, how do I can a field name matching a pattern and doing a substring to get everything after the ''.t.''
          event.set('key.substring something', event.remove(key))

I amd having a hard time doing a start, ruby doesnt seem to like [field][field] and I can't escape de dots either with a backslash..

I am starting to wonder if i am on the right path or is there an easier way?

Thank you,

2

If you field names contain periods then you could use

    ruby {
        code => '
            event.to_hash.each { |k, v|
                if k.start_with?("unwanted")
                    event.remove(k)
                elsif k.start_with?("categories.colors.")
                    event.set(k.sub("categories.colors.", ""), v)
                    event.remove(k)
                end
            }
        '
    }

If they are objects then you do not need ruby.

    mutate {
        remove_field => [ "unwanted", "[categories]" ]
        rename => { "categories][colors][t]" => "t" }
    }
3

Thakn you, I will try this out! Will post back with results

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.