I have this pipeline:
Apache Webserver + Filebeats -> Logstash1 (no filters) -> Kafka -> Logstash2 (Filters) -> ES + Kibana
Via my Kafka consumer, I can see the logs going to the Kafka channel. And in Kibana I can see the heart beat being indexed in ES. But for some reason, there is no longer any logs going Kafka->LS->ES. I look at the Logstash2's log file and there are no errors, and the config is ok. I have removed all the filters and the ips and ports are good. I created a new channel in Kafka and temporarily had logs arriving in ES but then it stopped again (the Heartbeats continued).
Anyone have any idea what is going on? Thanks