Configure metadata sent to logstash

I am writing a custom beat that will be sending messages to logstash. I would like to be able to add additional fields to the @metadata object received by logstash. Is this possible? I am currently using libbeat v 5.5.

I followed the developer guide to create a new beat, and attempted adding the following code to what was generated in order to try to set additional fields in the @metadata object:

event := common.MapStr{
	"@timestamp": common.Time(time.Now()),
	"type":       b.Name,
	"counter":    counter,
eventMeta := common.MapStr{
	"index": "test",
	"otherData": "1234",
metaOp := publisher.Metadata(eventMeta)
bt.client.PublishEvent(event, metaOp)

Unfortunately, this did not work, and the @metadata object in logstash is still

Any help would be greatly appreciated.


In 5.x branch this is currently not possible. You can add another field to your event and do some event manipulation in logstash to get a similar result.

In master and 6.x branch the publisher pipeline has been refactored and the event type has become:

type Event struct {
  Timestamp time.Time
  Meta common.MapStr
  Fields common.MapStr

Whatever you put into Meta will be send/used by the outputs. e.g. adding event.Meta["pipeline"] selects the ingest node pipeline if the Elasticsearch output is used. All other outputs (file, console, kafka, redis, logstash) will put fields in Meta into the @metadata field when encoding to json.

Please note, 6.0 just entered the beta-phase and we haven't had time to update the developer docs, as there might be some more internal changes/cleanups before 6.0 GA, potentially affecting the docs.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.