Look at the log entries in discover mode, and inspect the JSON. What do the timestamp(s) look like? How are you processing timestamps in your ingest pipeline?
Based on your description another possible cause could be that your documents are being delayed throughout the ingest process. What does your pipeline look like?