How do I use prune inside of a specific field?

I have a bunch of csv files from which I need to extract the "user" and "hwid" columns from. The csv filter will pull out all of the columns and put them in a field called "data". I then use the prune filter to keep only these 2 fields. The prune filter will work just fine if the target is not defined, however once it is defined it refuses to work. I have looked through the documentation page and there doesnt seem to be a "target field" parameter for the prune filter. How can I prune only the fields inside of the "data" field?

input {
    file {
        path => "C://file.csv"
        start_position => "beginning"
	sincedb_path => "NUL"
    }
}
filter {
    csv {
	autodetect_column_names => true
        target => "data"
    }
    prune {
        interpolate => true
        whitelist_names=> ["user", "hwid"]
    }
}
output{
	stdout {}
}

The prune filter only works on top-level fields, it won't work with nested fields, this is in the documentation.

This filter currently only support operations on top-level fields, i.e. whitelisting and blacklisting of subfields based on name or value does not work.

If you only want to output data.user and data.hwid from all the fields in your csv you can do that combining a couple of mutates.

Something like this:

filter {
  csv {
    autodetect_column_names => true
    target => "_tmp"
  }
  mutate {
    add_field => {
      "[data][user]" => "%{[_tmp][user]}"
      "[data][hwid]" => "%{[_tmp][hwid]}"
    }
  }
  mutate {
    remove_field => ["_tmp"]
  }
}
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.