Completion of file

Elastic Stack Beats
1

Is there a way to find out how much a file has already been processed by Filebeat? (e.g. completion percentage)

I have a script that does sincedb <-> inode mapping for Logstash and calculates a completion value, but I have no idea how to do this for Filebeat?

Any suggestions?

2

Sorry, there is no official way to do this. Can you tell us more about your use case? I take it you're processing large files?

3

Yes, couple of GB per file. Once I start up Filebeat it would be nice to see how much it already processed.

By the way: Is this really such a strange requirement? imho thousands of developers would be keen to have such kind of information, no?

4

Well, typically Filebeat is used to handle files that are constantly being appended to. It sounds like you have fixed size files that occasionally are dropped into a spot for Filebeat. Is that correct? Can you say more about your use case?

This sounds like a potential feature for x-pack monitoring, where you could drill down to a filebeat instance and see which files it is currently working on.

Would be able to open a feature request on https://github.com/elastic/beats/issues to start a discussion about what you'd like in terms of this feature?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.