How to dynamic parse fieldname and replace?


I am trying data that looks like this


First I need to remove every unwanted field, then I want to change categories.colors to only blue.t.type.

I can use a kv filter for to remove unwanted fileds, but I am unable to find an easy way to remove
catories.colors dynamically

I want


to become

My code

//ruby {
code => "
keys = event.to_hash.keys

if ( key =~ /m./ )

         if key.start_with? '[categories][colors]'
          #I don't know what should go in the event.set, how do I can a field name matching a pattern and doing a substring to get everything after the ''.t.''
          event.set('key.substring something', event.remove(key))

I amd having a hard time doing a start, ruby doesnt seem to like [field][field] and I can't escape de dots either with a backslash..

I am starting to wonder if i am on the right path or is there an easier way?

Thank you,

If you field names contain periods then you could use

    ruby {
        code => '
            event.to_hash.each { |k, v|
                if k.start_with?("unwanted")
                elsif k.start_with?("categories.colors.")
                    event.set(k.sub("categories.colors.", ""), v)

If they are objects then you do not need ruby.

    mutate {
        remove_field => [ "unwanted", "[categories]" ]
        rename => { "categories][colors][t]" => "t" }

Thakn you, I will try this out! Will post back with results

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.