# Filebeat cannot detect modified ingested files

**URL:** <https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [January 10, 2022, 2:04am UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837 "2022-01-10T02:04:56Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![park\_rage](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/park_rage/32/100038_2.png) [@park\_rage](https://discuss.elastic.co/u/park_rage)\
**Post date:** [January 10, 2022, 2:04am UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837/1 "2022-01-10T02:04:56Z")

</div>

I am configuring the ELK + filebeat environment using docker-compose , but the environment and configuration are as follows.

- Environment: MacOS

- Functions to be implemented: Collect the /Users/mymac/dev/logs/app.log file in the MacOS host path from filebeat and send it to logstash

- composition

- configuration file

If you set it up as above and run it with docker-compose ..

After the server is running, the app.log data is sent to logstash immediately, but no changes are sent after that.

The items checked for this are as follows.

1. Are you connected to logstash normally?

-\> When running the container, check the log of logstash to confirm that it was received normally

1. Is the filebeat.yml file in the filebeat container applied properly?

-\> Connect to the filebeat container and check the /usr/share/filebeat/filebeat.yml file. As a result, the file set by the host is normally applied.

1. Does filebeat's log folder and files exist?

-\> Connect to the filebeat container and check the existence of the /var/log/server/app.log file

-\> However, even if the host file information is changed or updated, it is confirmed that the date and contents do not change in this container.

-\> By forcibly modifying the contents or changing the file permission, the update date and contents are detected as changed and sent to logstash

1. When filebeat is run in the background in a MacOS host environment other than docker

-\> Collection and delivery normally. make sure there is no problem

Can anyone help me with my problem?  
I'm looking for a solution and waiting for help.

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [January 10, 2022, 9:44am UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837/2 "2022-01-10T09:44:43Z")

</div>

Hi!

So it seems that `/var/log/server/app.log` is being populated but Filebeat is not capable to identify these changes and collect any new logs?  
When you you mention about `changing the file permission` what exactly happens here? Maybe this is the problem for Filebeat not detecting the changes?  
Unfortunately it seems that your problem is quite environment specific 😞 .

C.

---

<div class="post-metadata">

**Author:** ![park\_rage](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/park_rage/32/100038_2.png) [@park\_rage](https://discuss.elastic.co/u/park_rage)\
**Post date:** [January 10, 2022, 2:48pm UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837/3 "2022-01-10T14:48:26Z")

</div>

Helllo!  
First, thank you for your reply.

So it seems that /var/log/server/app.log is being populated but Filebeat is not capable to identify these changes and collect any new logs?  
-\> That's right. The Host Machine's log is constantly changing, but the mounted files in FileBeat don't change at all. (The date does not change, and no content is added.)

When you you mention about changing the file permission what exactly happens here? Maybe this is the problem for Filebeat not detecting the changes?  
-\> If you access the FileBeat container's shell and perform modifications such as adding the permissions and contents of the mounted app.log file, the collector immediately detects the change and sends the log to LogStash.  
The fundamental problem now is that even if the log file of the Host Machine is changed, the log file mounted on FileBeat does not change.

Unfortunately it seems that your problem is quite environment specific 😞  
-\> I've been looking for the same situation in many places as I am, but it doesn't seem to be a problem with my settings. So it's very confusing. I need to find a way to get help by organizing the source and uploading it to Git.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [January 10, 2022, 3:38pm UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837/4 "2022-01-10T15:38:16Z")

</div>

> [@park\_rage](#):
>
> That's right. The Host Machine's log is constantly changing, but the mounted files in FileBeat don't change at all. (The date does not change, and no content is added.)

If the file in the container does not change when the file in the host changes, then this is a Docker issue, not related to Filebeat.

You need to check in the Docker community forum how to solve this issue.

---

<div class="post-metadata">

**Author:** ![park\_rage](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/park_rage/32/100038_2.png) [@park\_rage](https://discuss.elastic.co/u/park_rage)\
**Post date:** [January 10, 2022, 10:04pm UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837/5 "2022-01-10T22:04:23Z")

</div>

You are right ..!  
I thought it was a FileBeat problem, so I tried to find the answer here. However, after further checking, it was determined that the problem was with Docker, and now we are looking for the problem together with the Docker community.  
Thank you so much for your interest.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2022, 12:04am UTC](https://discuss.elastic.co/t/filebeat-cannot-detect-modified-ingested-files/293837/6 "2022-02-08T00:04:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
