We are in the process of designing a system in AWS that automatically creates and stands up a new instance of an application server and after a configured amount of time tear it down and rebuild a new one. We're doing that to make sure that the system has the latest updated information available.
To that end we want to have filebeat installed on these new 'ephemeral' instances writing application log information to a back end Elasticsearch environment but we need to make sure that filebeat has finished processing any log files on the source server before completing the tear down.
Looking in the documentation I see settings for queueing and flushing but not seeing anything related to a finished or acknowledge signal that gets generated and sent out letting the filebeat service know that it is safe to shutdown or that it is safe to tear the machine down.
Is this possible and what would be the best approach?