Filebeat docker running on windows not allowing application to rotate the log!

Elastic Stack Beats
1

I am using filebeat image - docker.elastic.co/beats/filebeat:5.4.1 to setup filebeat container locally on windows to read logs from an application and send data to logstash. The application which generate logs will rotate it after '1MB' size.

  • I have set close_inactive = 2M. So that filebeat release the handle after 2 minutes.
  • When application stops writing data and all the data is read from the file, file handle is released. This is as expected.

My concern is,

  • While filebeat container is reading the file, application is not able to rotate the log file after 1MB size.

If i run filebeat as a service (Not inside docker) it works as expected i.e logs are rotated after 1MB.

How can I avoid this ?

2

Please see this post and try using the sysinternals tool to see who has the file open. Filebeat locks application logs on Windows

What docker driver are you using on Windows? Is it the Hyper-V driver?

3

Hi Andrew,

Thank you for replying to my query!

File is being held by VBoxHeadless.exe and java process(application which is writing to the file).
I am using virtualbox driverfor docker.

docker-machine create --driver virtualbox machine1

Docker version 1.12.5, build 7392c3b

4

Do you have detail on the share mask used by the two processes (like RWD) in opening the file?

5

image.png1487×638 96.1 KB

6

Could you please tell me how can i get share mask used by the process ?

Did you mean,

7

That does not show the share mask, try the handle.exe tool from sysinternals. See File open mode for an example.

8

May be this helps.

9

With that output it's not possible to attribute the handles to a process.

Based on your info that it's VBoxHeadless and java you should be able to get the PID with:

tasklist | findstr VBoxHeadless

Then use the PID in this command:

handle -a -u -p <pid> | findstr test-application.log

Repeat that for java.exe. Then we can know what share mask both applications are using.

10

Ok thanks for detailed steps.

11

Based on that output where VBoxHeadless.exe has the log file open with RW-, I'd say that the shared filesystem used by VirtualBox is to blame. It is not opening the file with FILE_SHARE_DELETE so other applications cannot delete or rename the file.

12

Makes sense.FYI, I also tried this scenario in Linux box. I didn't see this issue.

  • Is there something you aware of which i can do to avoid this issue ?
  • Should I be using different driver for docker on windows ?

Thanks!

13

I'm not aware of way to avoid this, but it's also not something I've encountered. Trying the Hyper-V driver would be an interesting test. Also trying Windows native containers would be interesting (note that Elastic doesn't provide a Windows container image).

14

This topic was automatically closed after 21 days. New replies are no longer allowed.