When filebeat/metricbeat configured to send output to kafka, offset for timestamp functionality in kafka is not working.
This is likely due to bug in Sarama (the underlying library filebeat/metricbeat use) not setting MaxTimestamp in a batch when the topic's message.timestamp.type is CreateTime. Whereas the fix is available in Sarama v1.45.1, filebeat/metricbeat is using Elastic's Sarama fork, based on old sarama version 1.19.1
Sarama upstream fix: produce: set MaxTimestamp by rockwotj · Pull Request #3108 · IBM/sarama · GitHub
mod.go:
GitHub - IBM/sarama: Sarama is a Go library for Apache Kafka. => GitHub - elastic/sarama: Sarama is a Go library for Apache Kafka 0.8, and up. v1.19.1-0.20220310193331-ebc2b0d8eef3