So I'm at a loss on how to go about publishing an event and being sure it was a success. If I use publisher.Client.PublishEvent(m, publisher.Sync) and the output (in my test case logstash) isn't up, it blocks forever instead of timing out and returing false. This hangs the beat requiring a SIGKILL to stop it. Am I doing something wrong or is this a bug?
The client has a Close() method which must be used in the case of sync publisher. See here the implementation in Filebeat: https://github.com/elastic/beats/blob/master/filebeat/beater/publish.go#L125
Here are the docs for the client: https://godoc.org/github.com/elastic/beats/libbeat/publisher#Client
options understood by
publisher.Sync: block PublishEvent until either event has been send or event is dropped due to max_retries
publisher.Guaranteed: Never drop event. Keep event in output pipeline until ACKed by output. Can be used without sync for async guaranteed publishing.
publisher.Signal: Callback for publish requests. E.g. used in filebeat in conjunction with
publisher.Guaranteedto update file state offsets once events have been ACKed by output.
Filebeat supports both mode. Sync publishing: https://github.com/elastic/beats/blob/master/filebeat/beater/publish.go#L110
and async publishing: https://github.com/elastic/beats/blob/master/filebeat/beater/publish.go#L166
If output (logstash/elasticsearch/...) is unresponsive and buffers in publisher pipeline (both, in sync and async mode) are full, PublishEvent(s) will block. In order to guarantee proper shutdown one has to dicsonnect from publisher pipeline via
Close is called, the publisher pipeline will not give any guarantees to events being queued. Events might be send or not.
This topic was automatically closed after 21 days. New replies are no longer allowed.