Not to add 'host', 'path' field

Elastic Stack Logstash
1

Hi.

With this configuration,

input {
    file {
        path   => ...
    }
}
output {
    elasticsearch {
        ...
    }
}

logstash add files 'host' and 'path' to the document.

  "_source": {
   ...
    "path": "/home1/.logstash/input/default_agg_daily/0505.json",
    "host": "xyzzzz"
  },

And I want to remove path and host by

mutate {remove_fileds => [...] }

But some case the origin document may contain field 'path' and/or 'host'.
So I can not use remove_filds simply.

Is there a way make logstash not to add 'host' and 'path' field?

2

The right configuration is :

mutate {
  remove_field => ["path", "host"]
}

And for information, if field path and/or host doesn't exist, there's no problem. The plugin will remove field if field exists, and just do nothing if field does not exist.

3

thanks!

but I knows remove fields only works when that fields exist.

the problem is
some of input of fileinputplugin contains host field.
in that case logstash will not add/overwrite host field.

but when origin input does not have host field it will add hist field.

in short
I want logstash not to add host field by itself.

4

OK, so I suppose your file input plugin use "json" codec ?
I mean, if it reads plain lines, there is no "host" field in input of file input plugin.

Presently, there is no way to disable "host" field in file input plugin.

You could open an issue for that :

And otherwise, you can use a trick :

  • read file with plain codec
  • then use mutate filter to remove "host" and "path" fields
  • then use "json" filter to convert message field to json document
1 Like
5

Thank you for the clear response.

You are right.
input file configured as json codec.