Safely remove tcp input plugin's "host" and "port" fields

The tcp input plugin adds host and port fields to received events.

Ideally we don't want to add these but since this isn't documented, there doesn't seem to be a way to remove them.

What is the best filter to safely remove them? For example, we can't blindly:

filter {
  mutate {
    remove_field => [
      "host",
      "port"
    ]
  }
}

since some of our received events are using ECS and this will destroy anything under host (e.g. host.hostname)

If there a way to test in logstash for this? Or should we use a ruby filter?

EDIT:

I would like to do something like:

filter {
   ruby {
      code => "event.delete('host') if String === event.get('host')"
   }  
}

but the ruby API only provides get and set; you can't delete a field.

EDIT:

unless I hear "there's a better way to do this" here's what I'm settling on:

filter {
  ruby {
    code => "event.set('host', {}) if String === event.get('host')" # set by input plugins, we don't care
    id => "remove_input_noise"
    remove_field => [
      "port" # set by input plugins, we don't care
    ]
  }
}

if elastic receives the blank hash under host, it won't index anything so there's no extra noise in the event. It's still in the original document though.

Not so, there is an event.remove method that deletes a field.

Is it unsupported? It's not in the ruby API documentation.

They may be undocumented but event.remove or event.cancel are both used a lot.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.