I'm trying to create a custom beat that accepts input log input via stdin. For example you might feed a single completed log file into this beat like this:
cat myapp.log | mybeat
The problem that I'm having is that my beat exits before the publisher.Client has a chance to flush and send log events to the destination (Elasticsearch, for example). If I put a time.Sleep(5 * time.Second) after my custom beat completes, but before main exits, my events make it to Elasticsearch. Is there a cleaner way for me to ensure that the publisher.Client has flushed before exiting the application?
Great to hear that you are creating a beat. If you need a beat that accepts stdin, have a look at Filebeat. It accepts input_type: stdin.
For your shutdown question: Best is to have a look at the filebeat code as there filebeat is waiting until the publisher is closed. In case you need further help, please share the link to your code. That will make it much easier to comment.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.