I have two data streams that I'm tracking: one has billing requests coming in with a structure of Senderid > Account > Invoice#
The other steam is monthly billing data - which processes all of the requests. I would like to capture the data when it comes in for billing and then update the same data when it's processed at the end of the month to reflect status details of each invoice.
I have trouble deciding how to organize my data in ES. I mean this is clearly a parent child data structure, where Sender has multiple Accounts and each account has multiple Invoices. I have not setup Parent:Child relationship because I am afraid that I won't be able to search/display such data in Kibana and that's important for our users.
So, what I've done is I've flattened the document structure and I'm keeping every record at Invoice level. So each document contains SenderId, Account# and Invoice# (not normalized).
Thus each document is a unique Invoice. Now for my challenge:
The data that I process from the monthly billing comes to me in two different granularities: one at Customer Account level and one at Invoice level. The invoice level is better because I know I should update the unique record in ES for that invoice. But when I get a customer account level data (example: customer requires manual billing), I have a number of records with that account id in ES and I don't know how to update them all....
I hope that makes sense - can I update a field in multiple records in ES where customer accountid = %{value} ?