# Filebeat in k8s pod cannot read log file : No such file or directory **URL:** \ **Category:** Beats\ **Tags:** docker, filebeat\ **Created:** [December 28, 2022, 7:57am UTC](https://discuss.elastic.co/t/filebeat-in-k8s-pod-cannot-read-log-file-no-such-file-or-directory/322074 "2022-12-28T07:57:32Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![cat-rat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cat-rat/32/115292_2.png) [@cat-rat](https://discuss.elastic.co/u/cat-rat)\ **Post date:** [December 28, 2022, 7:57am UTC](https://discuss.elastic.co/t/filebeat-in-k8s-pod-cannot-read-log-file-no-such-file-or-directory/322074/1 "2022-12-28T07:57:32Z")
```auto --- apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: kube-system labels: k8s-app: filebeat data: filebeat.yml: |- filebeat.inputs: - type: container fields_under_root: true fields: evn: lcdp-dev log-type: lcdp-app server: lcdp language: java tail_files: true paths: - /var/log/containers/*_apaas_xjsd*.log processors: - add_kubernetes_metadata: host: ${NODE_NAME} matchers: - logs_path: logs_path: "/var/log/containers/" - drop_fields: fields: - host - ecs - stream ignore_missing: true # To enable hints based autodiscover, remove `filebeat.inputs` configuration and uncomment this: #filebeat.autodiscover: # providers: # - type: kubernetes # node: ${NODE_NAME} # hints.enabled: true # hints.default_config: # type: container # paths: # - /var/log/containers/*${data.kubernetes.container.id}.log processors: - add_cloud_metadata: - add_host_metadata: output.logstash: hosts: ['${LOGSTASH_HOST:logstash}:${LOGSTASH_PORT:514}'] --- apiVersion: apps/v1 kind: DaemonSet metadata: name: filebeat namespace: kube-system labels: k8s-app: filebeat spec: selector: matchLabels: k8s-app: filebeat template: metadata: labels: k8s-app: filebeat spec: serviceAccountName: filebeat terminationGracePeriodSeconds: 30 hostNetwork: true dnsPolicy: ClusterFirstWithHostNet containers: - name: filebeat image: docker.elastic.co/beats/filebeat:7.17.8 args: [ "-c", "/etc/filebeat.yml", "-e", ] env: - name: LOGSTASH_HOST value: "10.20.13.35" - name: LOGSTASH_PORT value: "5151" - name: NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName securityContext: runAsUser: 0 # If using Red Hat OpenShift uncomment this: #privileged: true resources: limits: memory: 200Mi requests: cpu: 100m memory: 100Mi volumeMounts: - name: config mountPath: /etc/filebeat.yml readOnly: true subPath: filebeat.yml - name: data mountPath: /usr/share/filebeat/data - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true - name: varlog mountPath: /var/log readOnly: true volumes: - name: config configMap: defaultMode: 0640 name: filebeat-config - name: varlibdockercontainers hostPath: path: /data/docker/containers - name: varlog hostPath: path: /var/log # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart - name: data hostPath: # When filebeat runs as non-root user, this directory needs to be writable by group (g+w). path: /var/lib/filebeat-data type: DirectoryOrCreate ``` ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/7/d7fb4b3095a86f6f5e2d48d5c9a40ffd68658b8c.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/d/1de4d033f504251ec720f97eecafa0e80a93ef17.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/6/d65cb489c6a4c66ba650e97d9b2e6c8c271eed4d.png) filebeat pod log ```auto 2022-12-28T06:06:31.952Z INFO instance/beat.go:697 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs] Hostfs Path: [/] 2022-12-28T06:06:31.952Z INFO instance/beat.go:705 Beat ID: a188c828-2598-40ed-b376-ad1db88d8927 2022-12-28T06:06:34.953Z WARN [add_cloud_metadata] add_cloud_metadata/provider_aws_ec2.go:79 read token request for getting IMDSv2 token returns empty: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used. 2022-12-28T06:06:34.958Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed 2022-12-28T06:06:34.958Z INFO [beat] instance/beat.go:1051 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "a188c828-2598-40ed-b376-ad1db88d8927"}}} 2022-12-28T06:06:34.958Z INFO [beat] instance/beat.go:1060 Build info {"system_info": {"build": {"commit": "692b4aac606e457bd2f5ef092d2d23c2fa950828", "libbeat": "7.17.8", "time": "2022-12-03T00:31:52.000Z", "version": "7.17.8"}}} 2022-12-28T06:06:34.958Z INFO [beat] instance/beat.go:1063 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":16,"version":"go1.18.5"}}} 2022-12-28T06:06:34.961Z INFO [beat] instance/beat.go:1067 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2022-08-30T05:57:35Z","containerized":true,"name":"l-sj-lcdp-work-dev01","ip":["127.0.0.1/8","::1/128","10.20.13.13/24","fe80::250:56ff:fea7:3c32/64","172.17.0.1/16","fe80::42:9bff:fe36:87c2/64","10.42.1.0/32","fe80::8dd:9cff:fe71:f59b/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64","fe80::ecee:eeff:feee:eeee/64"],"kernel_version":"3.10.0-1160.71.1.el7.x86_64","mac":["00:50:56:a7:3c:32","02:42:9b:36:87:c2","0a:dd:9c:71:f5:9b","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee","ee:ee:ee:ee:ee:ee"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.5 LTS (Focal Fossa)","major":20,"minor":4,"patch":5,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0}}} 2022-12-28T06:06:34.961Z INFO [beat] instance/beat.go:1096 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 8, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2022-12-28T06:06:31.530Z"}}} 2022-12-28T06:06:34.961Z INFO instance/beat.go:291 Setup Beat: filebeat; Version: 7.17.8 2022-12-28T06:06:34.962Z INFO [publisher] pipeline/module.go:113 Beat name: l-sj-lcdp-work-dev01 2022-12-28T06:06:34.963Z WARN beater/filebeat.go:202 Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning. 2022-12-28T06:06:34.963Z INFO [monitoring] log/log.go:142 Starting metrics logging every 30s 2022-12-28T06:06:34.963Z INFO instance/beat.go:456 filebeat start running. 2022-12-28T06:06:34.964Z INFO memlog/store.go:119 Loading data file of '/usr/share/filebeat/data/registry/filebeat' succeeded. Active transaction id=0 2022-12-28T06:06:34.964Z INFO memlog/store.go:124 Finished loading transaction log file for '/usr/share/filebeat/data/registry/filebeat'. Active transaction id=0 2022-12-28T06:06:34.964Z WARN beater/filebeat.go:411 Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning. 2022-12-28T06:06:34.964Z INFO [registrar] registrar/registrar.go:109 States Loaded from registrar: 0 2022-12-28T06:06:34.964Z INFO [crawler] beater/crawler.go:71 Loading Inputs: 1 2022-12-28T06:06:34.964Z INFO [crawler] beater/crawler.go:117 starting input, keys present on the config: [filebeat.inputs.0.fields.evn filebeat.inputs.0.fields.language filebeat.inputs.0.fields.log-type filebeat.inputs.0.fields.server filebeat.inputs.0.fields_under_root filebeat.inputs.0.paths.0 filebeat.inputs.0.processors.0.add_kubernetes_metadata.host filebeat.inputs.0.processors.0.add_kubernetes_metadata.matchers.0.logs_path.logs_path filebeat.inputs.0.processors.1.drop_fields.fields.0 filebeat.inputs.0.processors.1.drop_fields.fields.1 filebeat.inputs.0.processors.1.drop_fields.fields.2 filebeat.inputs.0.processors.1.drop_fields.ignore_missing filebeat.inputs.0.tail_files filebeat.inputs.0.type] 2022-12-28T06:06:34.964Z WARN [cfgwarn] log/input.go:89 DEPRECATED: Log input. Use Filestream input instead. 2022-12-28T06:06:34.965Z INFO [input] log/input.go:171 Configured paths: [/var/log/containers/*_apaas_xjsd*.log] {"input_id": "d63f155a-8eec-4768-9026-3344c12d5a52"} 2022-12-28T06:06:34.965Z INFO [crawler] beater/crawler.go:148 Starting input (ID: 870644319368240099) 2022-12-28T06:06:34.965Z INFO [crawler] beater/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1 2022-12-28T06:06:34.975Z INFO add_kubernetes_metadata/kubernetes.go:72 add_kubernetes_metadata: kubernetes env detected, with version: v1.20.8 2022-12-28T06:06:34.975Z INFO [kubernetes] kubernetes/util.go:122 kubernetes: Using node work1 provided in the config {"libbeat.processor": "add_kubernetes_metadata"} 2022-12-28T06:06:37.953Z INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:101 add_cloud_metadata: hosting provider type not detected. 2022-12-28T06:07:04.979Z INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000}},"id":"/"},"cpuacct":{"id":"/","total":{"ns":262761106}},"memory":{"id":"/","mem":{"limit":{"bytes":209715200},"usage":{"bytes":43851776}}}},"cpu":{"system":{"ticks":50,"time":{"ms":59}},"total":{"ticks":230,"time":{"ms":250},"value":230},"user":{"ticks":180,"time":{"ms":191}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral_id":"c51f2099-815c-44b2-85cc-74b28954da22","uptime":{"ms":33068},"version":"7.17.8"},"memstats":{"gc_next":22072360,"memory_alloc":15622000,"memory_sys":43861000,"memory_total":65474720,"rss":88260608},"runtime":{"goroutines":74}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0},"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0},"queue":{"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":16},"load":{"1":1.23,"15":1.36,"5":1.35,"norm":{"1":0.0769,"15":0.085,"5":0.0844}}}}}} ``` filebeat pod cannot open log ---
**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\ **Post date:** [December 29, 2022, 9:00am UTC](https://discuss.elastic.co/t/filebeat-in-k8s-pod-cannot-read-log-file-no-such-file-or-directory/322074/2 "2022-12-29T09:00:04Z")
Hey! Are those file maybe symlinks to other files? If so maybe you can [Container input | Filebeat Reference [8.5] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-container.html#_symlinks) to indicate to Filebeat to follow them and open the original files. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 26, 2023, 11:00am UTC](https://discuss.elastic.co/t/filebeat-in-k8s-pod-cannot-read-log-file-no-such-file-or-directory/322074/3 "2023-01-26T11:00:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.