I'm new to Elasticsearch and have been facing an issue with filebeats where I've been observing that logs from certain paths do not seem to get picked when the file in that path is updated. Running filebeat in debugger mode also doesn't show up anything relevant to that path.
However, upon deleting the registry file, filebeat detects the file. This I can say as I see a log message that the harvester has started on the file.
A snippet of how the YAML configuration for the concerned path appear is below.
- type: log
I would check the settings for how u have the
close_* options or the ignore_older. Log input | Filebeat Reference [7.12] | Elastic.
Thanks for sharing this.
I'm not sure if
ignore_older is the issue as I'm manually changing the file in the concerned path while monitoring real time filebeat logs. Still I don't see the harvester detecting the changes.
Not sure, but is it possible that the state for the file is causing an issue here and if so using
clean_inactive can solve it? However, would using
clean_inactive also mean that this would cause the entire file to be ingested again including the lines which were already ingested before?
that i don't know, i would have to refer you to the docs on that.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.