Hi folks,
I'm dealing with geo-timeseries data (IOT use-case). The data is indexed in an ES and queried in various ways.
So far, so good, but we have some bad data points (that we detect using outlier detection algorithms, heuristics, or even manual annotation), that we need to UPDATE. Sometimes, we just flag the data, other times, we do some imputation to keep aggregates working well.
I'm not very fond of UPDATING what should be otherwise immutable data. This causes other complexities in the architecture, or when re-indexing. So I'm looking for design recommendations, REX, best practices, to deal with this kind of problematic.
Any pointer or comment will be appreciated !
Thanks
Mat