I am using journalbeat to ship journal events to a remote elasticsearch (ie over the public internet). We have observed that journalbeat frequently stops shipping logs, but continues running in this broken state.
There are no errors in the journalbeat log (even at "info" level). There is some indication of read errors from the metrics endpoint in the libbeat.output.read.errors metric, but as these errors are not logged, it's hard to debug further.
After finding this comment on a previous issue I used lsof to investigate a failing journalbeat process and a healthy one and i see the same problem: the failing process has lost its TCP connection to elasticsearch.
I am running journalbeat 6.6.0 on ubuntu 18.04.2 on AWS EC2.
Shall I file a github issue?