Can filebeat's callback method `Publish(_ context.Context, batch publisher.Batch) error` batch events belong to one file

Can filebeat's callback method Publish(_ context.Context, batch publisher.Batch) error batch events belong to one file.
For example, harvesting three log a.log, b.log, c.log.Call the method with param Batch only file contents that belong to one of a.log, b.log and c.log

Filbeat creates inputs/streams per file, so events will be grouped/batched per file.

Thanks for your reply. Did you mean that although the Event in Publish(_ context.Context, batch publisher.Batch)'s batch, both have their own metadata.
But they are exactly from one file?

Thank you very much.

@ChrsMark Could you please take a look, thank you

Hey,

filebeat internally uses an internal queue (Configure the internal queue | Filebeat Reference [7.14] | Elastic), however I don't think that you can access the batch variable through configuration.

To the best of my knowledge the queue is common for a filebeat's process and hence once the queue has enough events to send the batch will be sent. Hope that helps. If you have a more specific problem to solve then maybe we can help.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.