# Logstash input-beat not receiving @metadata field

**URL:** <https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 28, 2016, 1:42pm UTC](https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365 "2016-01-28T13:42:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bruno\_Lavoie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruno_lavoie/32/8408_2.png) [@Bruno\_Lavoie](https://discuss.elastic.co/u/Bruno_Lavoie)\
**Post date:** [January 28, 2016, 1:42pm UTC](https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365/1 "2016-01-28T13:42:01Z")

</div>

Hello,

I searched all over the web and on this forum but haven't found.

I configured a filebeat to parse server logs and send it to logstash, when I want to refer to @metadata fields in filters and Elasticsearch-output parameters ([as shown here](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-details.html#logstash-output)), the field doesn't seems to be transmitted.

When I output filebeat to stdout I can see the @metadata field structure, but as soon it enters Logstash, it's not there. I checked it with rubydebug output...

Any ideas?

LS 2.1.1  
filebeat 1.0.1

Thanks  
Bruno

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 28, 2016, 5:08pm UTC](https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365/2 "2016-01-28T17:08:55Z")

</div>

can you share config files and some more details about your setup? LS remove '@metadata' field itself in output plugins.

---

<div class="post-metadata">

**Author:** ![Bruno\_Lavoie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruno_lavoie/32/8408_2.png) [@Bruno\_Lavoie](https://discuss.elastic.co/u/Bruno_Lavoie)\
**Post date:** [January 28, 2016, 6:28pm UTC](https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365/3 "2016-01-28T18:28:42Z")

</div>

Ok, it makes things clearer...

My test setup is very simple:  
input-beats:  
`beats { port => 5102 codec => line }`

no filters...

output-stdout  
`output { stdout { codec => rubydebug } }`

When I run filebeat with stdout, I can see the field:  
`{ "@metadata": { "beat": "filebeat", "type": "my-defined-type" }, ... }`

But not with LS outputed stdout with rubydebug codec..

IIRC what you said, output stdout remove @metadata?  
However, why output plugins are deleting these fields?

The point is that we use front LS nodes (proxy) that just open ports and then forward to a Redis broker....  
Then, input-redis is used by a LS indexing cluster to apply the complex stuff (filters)

So, as soon an output is used we lose the @metadata, it makes it impossible to use buffering broker and then output to Elasticsearch with config like this:

`index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}"`

Am I correctly understand?

Thanks  
Bruno

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 29, 2016, 3:21pm UTC](https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365/4 "2016-01-29T15:21:56Z")

</div>

No idea why logstash is removing '@metadata'. I guess '@metadata' field is supposed to be local to logstash instance.

See [this response](https://discuss.elastic.co/t/beats-deployment-with-logstash-relays-to-a-central-ls-instance/36107/4) for using filters to transfer metadata.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:56pm UTC](https://discuss.elastic.co/t/logstash-input-beat-not-receiving-metadata-field/40365/5 "2017-07-05T21:56:10Z")

</div>


