# Filebeat @metadata object, can we remove this object before publishing? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 10, 2019, 6:44pm UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847 "2019-07-10T18:44:20Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\ **Post date:** [July 10, 2019, 6:44pm UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/1 "2019-07-10T18:44:20Z")
I'm using filebeat 7.1.1 with a simple log file input ``` filebeat.inputs: - type: log ``` When the events are published, there is a "@metadata" object inserted in the event. ``` { "@timestamp": "2019-07-10T18:24:56.312Z", "@metadata": { "beat": "", "type": "_doc", "version": "" }, ... "agent": { "type": "filebeat", "ephemeral_id": "97f29009-c620-4746-9598-66bb651b4707", "hostname": "pop-os", "id": "38a65a07-1a0a-4e0a-a651-bc83065f73be", "version": "7.1.1" }, ... }, ``` One observation is that the 'beat' and 'version' field are empty. But the data is already in the "agent" object. How can I remove the "@metadata" object from the events before they are published to the output? I tried ``` processors: - drop_fields: fields: ["@metadata"] ``` without much success. I also could not find the documentation on the "@metadata" at [https://www.elastic.co/guide/en/beats/filebeat/7.1/exported-fields-beat.html](https://www.elastic.co/guide/en/beats/filebeat/7.1/exported-fields-beat.html) ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [July 11, 2019, 12:05pm UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/2 "2019-07-11T12:05:39Z")
You can not remove it. Internally it is dropped by the Elasticsearch output. Normally it is used in conjunction with Logstash (e.g. filebeat -\> kafka -\> logstash -\> elasticsearch) and should ship information about the index to use. It can actually be different from `agent`. But it should not be empty, though. ---
**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\ **Post date:** [July 11, 2019, 6:01pm UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/3 "2019-07-11T18:01:16Z")
Thanks for the explanation. I never used LogStash. Would be nice to have this once removed and save some bytes that have to go over the network 😉 ---
**Author:** ![martinr\_ubi](https://avatars.discourse-cdn.com/v4/letter/m/b5e925/32.png) [@martinr\_ubi](https://discuss.elastic.co/u/martinr_ubi)\ **Post date:** [July 12, 2019, 2:38am UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/4 "2019-07-12T02:38:46Z")
Maybe there is a misunderstanding here. What will be the output that you will really use in production? You say save bytes over the wire but your question is about the local file output of filebeat. Which means your outputting to the local filesystem. No wire here. You dont use logstash either, so were you inspecting the events with the file output but in reality you’ll use the elasticsearch output? If you ship to ES with the ES output, those fields are removed and not sent to ES. So the bytes are saved if you use the ES output. And because those fields are for logstash they need to be sent when the logstash output is used. If you use the file output but then send the events over the wire it means you’re configuring filebeat to write to files on disk and then using something else to read them and finally send them over the wire? So that other soft could remove the meta field before shipping the events then. I’m curious to understand your setup in the context of “over the wire” while you report using the file output... What am I missing? ---
**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\ **Post date:** [July 12, 2019, 3:29am UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/5 "2019-07-12T03:29:10Z")
I'm just looking at the journalbeat logs from filebeat. It 'suggested' that those event values were being send to the output. Glad to hear that this is not the case. ---
**Author:** ![meister](https://avatars.discourse-cdn.com/v4/letter/m/90db22/32.png) [@meister](https://discuss.elastic.co/u/meister)\ **Post date:** [July 23, 2019, 7:03am UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/6 "2019-07-23T07:03:32Z")
How about sending to Kafka? I want to drop those metadata and just send the `message` part. My usecase is: Read logs via filebeat Send logs to kafka Kafka is managed by other team with configurations we can't touched. They can't parse our data because of unnecessary metadata. We'd like to avoid using Logstash for this. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 20, 2019, 7:03am UTC](https://discuss.elastic.co/t/filebeat-metadata-object-can-we-remove-this-object-before-publishing/189847/7 "2019-08-20T07:03:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.