# Filebeat Fails to Process Symlinked Log Files Despite Symlink Configuration Enabled

**URL:** <https://discuss.elastic.co/t/filebeat-fails-to-process-symlinked-log-files-despite-symlink-configuration-enabled/367159>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 26, 2024, 7:56am UTC](https://discuss.elastic.co/t/filebeat-fails-to-process-symlinked-log-files-despite-symlink-configuration-enabled/367159 "2024-09-26T07:56:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhautik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bhautik/32/137843_2.png) [@bhautik](https://discuss.elastic.co/u/bhautik)\
**Post date:** [September 26, 2024, 7:56am UTC](https://discuss.elastic.co/t/filebeat-fails-to-process-symlinked-log-files-despite-symlink-configuration-enabled/367159/1 "2024-09-26T07:56:00Z")

</div>

I am encountering an issue with Filebeat where it fails to process symlinked log files, even though I have configured the symlink handling to be enabled. Below are the details of my configuration and the error messages I’m receiving.

```yaml
filebeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true
      hints.default_config:
        type: filestream
        id: kubernetes-container-logs-${data.kubernetes.node.name}-${data.kubernetes.pod.name}-${data.kubernetes.container.id}-${data.kubernetes.pod.uid}
        paths:
          - /var/log/containers/*-${data.kubernetes.container.id}.log
        parsers:
          - container: ~
        prospector:
        scanner:
          fingerprint.enabled: false
          symlinks: true
        file_identity.fingerprint: ~
        clean_removed: true

```

I have checked the log files in /var/log/containers and confirmed that they are symlinks. However, I still receive the following error message:

```auto
{"log.level":"debug","@timestamp":"2024-09-26T07:31:38.571Z","log.logger":"scanner","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileScanner).GetFiles","file.name":"filestream/fswatch.go","file.line":388},"message":"cannot create an ingest target for file \"/var/log/containers/log-gen-6cfd5d4cfc-gkbmp_monitoring_log-gen-80a3482ec2656007c1308be932a56dc3420d901cce4bbf5336e0ee184dc88d3d.log\": file \"/var/log/containers/log-gen-6cfd5d4cfc-gkbmp_monitoring_log-gen-80a3482ec2656007c1308be932a56dc3420d901cce4bbf5336e0ee184dc88d3d.log\" is a symlink and they're disabled","service.name":"filebeat","ecs.version":"1.6.0"}

```

**Additional Information:**

• I have ensured that the symlink handling is enabled in my Filebeat configuration.  
• I have deleted the meta.json and log.json files to see if it resolves the issue, but the problem persists.  
• I have verified the permissions and existence of the target files for the symlinks.

I appreciate your help!

---

<div class="post-metadata">

**Author:** ![bhautik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bhautik/32/137843_2.png) [@bhautik](https://discuss.elastic.co/u/bhautik)\
**Post date:** [September 26, 2024, 1:01pm UTC](https://discuss.elastic.co/t/filebeat-fails-to-process-symlinked-log-files-despite-symlink-configuration-enabled/367159/2 "2024-09-26T13:01:18Z")

</div>

Set to `setup.template.enabled: false` because we cannot use same template

```yaml
# logging.level: debug

filebeat.inputs:
  - type: filestream
    id: auditbeat
    fields:
      auditbeat: true
    paths:
      - /var/log/auditbeat/*
    parsers:
      - ndjson:
          target: ""
          overwrite_keys: true

filebeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true
      hints.default_config:
        type: container
        id: kubernetes-container-logs-${data.kubernetes.node.name}-${data.kubernetes.pod.name}-${data.kubernetes.container.id}-${data.kubernetes.pod.uid}
        paths:
          - /var/log/containers/*-${data.kubernetes.container.id}.log

output.elasticsearch:
  logging.level: debug
  logging.selectors: ["*"]
  hosts: ["${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}"]
  compression_level: 9
  indices:
    - index: "auditbeat-%{[agent.version]}-%{+yyyy.MM.dd}"
      when.equals:
        fields.auditbeat: true
    - index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"
      when.not.equals:
        fields.auditbeat: true

# For debug purpose
# output.console:
# enabled: true
# codec.json:
# # Pretty-print JSON event
# pretty: false

setup.template.enabled: false
setup.template.name: "filebeat"
setup.template.pattern: "filebeat-*"
setup.ilm.enabled: true
setup.ilm.mode: auto

```