I have LS 2.0.x ES2.0 Shield 2.0 and Kibana 4.2
My configuration reads incoming application log data. The data represents all of the work flow events that our orders go thorugh - a series of pre-defined steps (approx 7 steps) that arrive at different times.
I want my Index, to contain only one document for each order. I have created in my mappings individual fields (buckets) in that index that keep track of the different work flow steps. Thus, everytime a new event comes in for that order, I want to update a different "bucket" in the document to reflect the completion of that step. Thus WFStep1 timestamp, WFStep2 timestamp, WFStep3...
I control the document_id to facilitate the updates so I get one document per order.
Thus I want to populate the document gradually as the different WorkFlow steps are completed for each order. At the end of all WF steps I want all of the 'buckets'to be populated as a result.
It almost works... the problem is that the final document only contains the very last WorkFlow step update and doesn't 'remember' any other steps that happened before it. Even though there were 7 transactions for that same order only the last one is retained in the document. Why aren't the other fields populated by prior updates preserved in the document? How can I gradually populate that document with data?