Not sure if ELK stack is the right way to solve this problem, but let's just talk through it. Here is a new use case I have:
We get a daily playload with data which contains account and invoice data. We record this accounts data throughout the month into ES. As we
get closer to month end, we need to validate these accumulated accounts against a
remote database (JDBC query). Some of these accounts will be valid
(found), others will be invalid and we need to mark them as such in ES
so we can report on it. Ideally, I would like to re-trigger this
validation periodically, so as to get a more accurate picture of the
account statuses as we get closer ot month end billing run (around 21st
of each month).
How would we try to solve that using an ELK stack tools?! Is it possible to do that? Would it be practical? We have Java/JDBC skills on the team, as well as up and coming ELK skills. Please comment on the possible approach.