I configured Filebeats to add new Logfiles in the harvestors' path and restarted the filebeat service on those instances from where the Logs were to be harvested.
On the Logstash side, I configured the Logstash-output-email plugin to send out an email once a 500 error is encountered in the message. What I didn't apprehend beforehand was that the harvesting would start from the beginning of the Logfiles. What I am trying to achieve is to harvest only for new entries and not the existing logs.
Now the real problem is that, Once the emails started coming in , they are not stopping at all eventhough I have commented the email output code in Logstash. I am assuming the emails because of the older logs have queued in and are coming out in humongous amounts. Is there any way to flush the email queue ?