Recently, I set up a filebeat to receive palo alto and sophos data. Filebeat module parsed the data as the timestamp.
Instead of using the timestamp in log, i want to use the received time as @timestamp.
Does there any way to override the @timestamp parsed by filebeat module using receive time.
You'll have to update the ingest pipelines.
I understand that we can override @timstamp using "target field:@timestamp" with a specific time fields.
However, how can I get the server current time or the log received time to override the @timestamp
U can use _ingest.timestamp for the time it gets into Elasticsearch. Is there a reason u want to do this? The@timestamp fields is meant for when the event took place regardless of when it got to Elasticsearch.
As Palo Alto Log do not show the timezone of it.
Therefore, I planned to standardize the timestamp to storage time.
Are u sending multiple Palo alto logs to the same filebeat instance and are they in different time zones? If so, one thing u could do is start multiple instances of the module and set a field with the respective timezone or add script processor to filebeat to set the timezone based off of log source IP... There are a bunch of things u could do. I personally wouldn't use the ingest time as ur source of truth for event timestamps as it could be widely inaccurate.