While investigating on an event I noticed there was a huge difference between event.created and event.ingested which created a confusion on the real event time.
Later digging in docs resulted
event.ingested: timestamp when an event arrived in the central data store.
This is different from @timestamp, which is when the event originally occurred. It’s also different from event.created, which is meant to capture the first time an agent saw the event.
Now how do I prove that why elasticsearch is ingesting logs so slowly or there is something wrong with elastic agent?
Will be happy to provide all the necessary details.
The lag seems very, very long. Is this something you see consistently for this type of event? Which timezone are you in? Is it possible this could be a timezone issue?
No, it isn't time zone issue I crosschecked the NTP.
It could be a timezone issue based on how the timestamps are parsed even if NTP is OK.
Please answer the questions I asked.
I didn't noticed it eariler but checking now it looks like there is at least a time difference of 2-3 hours for every event.
How are you ingesting data? What does the flow of these events look like?
It's the basic architecture.
ELK and fleet on a single host nothing fancy.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.