We have a system that from one side writes information, and on the other sides reads the data.
I synchronize the data based on a timestamp. I read data from a point in time, arrange it according to the timestamp and continue reading from the last timestamp I got.
I noticed that if I always read the latest data, i miss some data which is written out of order, so instead I added a delay of 1 minute, and now everything is fine.
My question is, what is the minimal delay I should use? In other words, what is the "settling time" of written records, and does it depend on the throughput of data?
With Best Regards,