Hello,
I'm trying to implement an alert system in Elasticsearch. I'll describe the key elements of the problem and my solution attempts until now.
Scenario
We are ingesting information into an Elasticsearch index using Filebeat. I can define a relevant set of features among the existing variables. These are: device_id, timestamp, and used_flag. used_flag may take different values, let's say five values. Each device sends information into Elasticsearch informing its timestamp and the used flag at the time.
We are interested in notifying a user each time a device changes the used flag.
Considered approach
I've taken some ideas from this previous question, and my current approach consists of the following procedure (not yet implemented). Index the information using the device_id as document ID with the fields timestamp and used_flag. Each time new data is received for device ID, search the document and identify if the previous used_flag is the same as the current one. If it is not, alert and update the document.
Issues
My approach has many issues, most of them related to my ability to implement many operations. First, I don't know how can I implement the operation of searching the previous state of a document upon reception of an update. Second, I don't know how to send some kind of alert in Elasticsearch. I've mostly used the Python API to interact with Elasticsearch and perform more complex operations, however, as I'm sending the documents using Filebeat, this procedure does not seem appropriate.
I'd appreciate any insights regarding this problem. If you consider that could use another approach, it would be also appreciated.
Best regards,