Rename filebeat default fields

Elastic Stack Beats
1

Is there a way for me to rename "beat.hostname" to "hostname" in filebeat output?

the documentation mentioned filter, but this seems like a filter setup in logstash, not in filebeat, right?
per https://www.elastic.co/guide/en/beats/filebeat/current/migration-changed-fields.html,
I included the following in filebeat.yml, filebeat complained about the syntax "Exiting: error loading config file: yaml: line 71: could not find expected ':'"

filter {
    mutate {
        rename => {
            "filebeat.hostname" => "hostname"
        }
    }
}
2

It can be done using Logstash or using the Ingest Node feature in Elasticsearch.

Right, that is a Logstash filter example.

1 Like
3

Thanks for the clarification. that is a bummer, we would like to deploy filebeat only at the edge server, and remove those unnecessary fields before sending them out.

Btw, just to make sure I understand this, log stash is heavy weight, not supposed to deployed at the edge server (where our production service is running), right?

4

You can remove the fields with Filebeat alone. You just cannot rename them.

processors:
- drop_fields:
    fields: ['beat.hostname']

With an ingest node pipeline in Elasticsearch you can rename the fields and this only requires Filebeat and Elasticsearch.

You could deploy LS on your edge nodes. It will likely have a larger memory footprint than Filebeat. If your only task is sending a logs then I would recommend Filebeat.

1 Like
5

@andrewkroh is it possible to remove "@timestamp" and "type"?

thanks!
yan

6

Those fields are protected from being removed because they are used by some of the outputs in beats. But you can drop them with the ingest node remove processor.

1 Like
7

I currently set up filebeat and logstash on the same machine in my local dev box, config filebeat to send output to logstash. In my logstash output, beat.hostname and host have the same value. I read the following in the filebeat documentation. I would like to confirm that host is indeed directly copied from beat.hostname, such that
when i deploy filebeat and logstash on different servers, they would still have the same values and I can safely remove "beat" field in logstash filter. I would like to keep the hostname of the edge server where filebeat is located.
Could someone please correct me if i mis-interpreted the following? If my understanding is correct, why duplicated fields are kept by default?

Filebeat uses "beat.hostname" for Sending the Hostname of the Server. While the Logstash Forwarder sends the hostname of the server it’s running on in the host field, Filebeat uses the beat.hostname field for the same purpose.
Because host is commonly used in the Logstash plugin ecosystem, the Beats input plugin automatically copies beat.hostname into host.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.