In Filebeat 7.3.0 the way timezone conversion is applied is different. Now, you have to use processors to add, remove or change the "event.timezone" variable. Now, I can't get it to work, at least with the panw module. It seems that the ingest processor refuses to overwrite the date values.
If I don't do anything, event.timezone gets the value -02:00, and all the timestamps are added 2 hours
If I add in filebeat.yml:
processors:
- add_fields:
target: "event"
fields:
timezone: "UTC"
event.timezone is changed to "UTC", but the timestamps are still 2 hours ahead.
I've been playing with the elasticsearch ingest pipeline, It seems that it is unable to overwrite the timestamp:
In "_ingest/pipeline/filebeat-7.3.0-panw-panos-pipeline", If I add the target_field:
{
"date" : {
"formats" : [
"ISO8601"
],
"timezone" : "{{ event.timezone }}",
"on_failure" : [
{
"append" : {
"value" : "{{ _ingest.on_failure_message }}",
"field" : "error.message"
}
}
],
"if" : "ctx.event.timezone != null",
"field" : "@timestamp"
"target_field" : "AAAAAAAA"
}
},
The AAAAAAAA field is added, and the value is correct.
But if I remove the target_field, meaning it should overwrite "@timestamp" in this case, It does not work.
The same happens with the other date fields, If target_field is a new field, It gets added and the value is correct, but If not, It can not overwrite the same field.
Is there something missing?