Filebeat can not collect logs data from mounted disk

Elastic Stack Beats
,
1

Hi all,

I used filebeat for collecting my logs and my filebeat version is 7.16.3.

When I collect my logs from /var/lib/docker/containers/${data.docker.container.id}/*.log, logs were collected. If I used /mnt/.. (my mounted disk), it can not collect our log datas. My docker set up is in mounted disk.

How can I solve this problem? Any idea?

Thanks

2

What do the Filebeat logs show?

3

Filebeat logs as following :

2022-04-29T07:40:36.782+0300 INFO instance/beat.go:686 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs] Hostfs Path: [/]

2022-04-29T07:40:36.790+0300 INFO instance/beat.go:694 Beat ID: d32e5111-fd2a-48a4-b465-b7a8011a914d

2022-04-29T07:40:36.847+0300 INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed

2022-04-29T07:40:36.847+0300 INFO [beat] instance/beat.go:1040 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "d32e5111-fd2a-48a4-b465-b7a8011a914d"}}}

2022-04-29T07:40:36.847+0300 INFO [beat] instance/beat.go:1049 Build info {"system_info": {"build": {"commit": "d420ccdaf201e32a524632b5da729522e50257ae", "libbeat": "7.16.3", "time": "2022-01-07T00:36:57.000Z", "version": "7.16.3"}}}

2022-04-29T07:40:36.847+0300 INFO [beat] instance/beat.go:1052 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":96,"version":"go1.17.5"}}}

2022-04-29T07:40:36.853+0300 INFO [beat] instance/beat.go:1056 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2022-03-14T15:26:20+03:00","containerized":true,"name":"50c442b913bd","ip":["127.0.0.1/8","10.0.12.220/24","172.19.0.9/16"],"kernel_version":"3.10.0-1160.el7.x86_64","mac":["02:42:0a:00:0c:dc","02:42:ac:13:00:09"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"+03","timezone_offset_sec":10800,"id":"e43c664ae344d1677949b27d14e216f3"}}}

2022-04-29T07:40:36.854+0300 INFO [beat] instance/beat.go:1085 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 6, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2022-04-29T07:40:36.200+0300"}}}

2022-04-29T07:40:36.854+0300 INFO instance/beat.go:328 Setup Beat: filebeat; Version: 7.16.3

2022-04-29T07:40:36.855+0300 INFO [publisher] pipeline/module.go:113 Beat name: 50c442b913bd

2022-04-29T07:40:36.858+0300 WARN beater/filebeat.go:202 Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.

2022-04-29T07:40:36.858+0300 INFO [monitoring] log/log.go:142 Starting metrics logging every 30s

2022-04-29T07:40:36.858+0300 INFO instance/beat.go:492 filebeat start running.

2022-04-29T07:40:36.865+0300 INFO memlog/store.go:119 Loading data file of '/usr/share/filebeat/data/registry/filebeat' succeeded. Active transaction id=0

2022-04-29T07:40:36.865+0300 INFO memlog/store.go:124 Finished loading transaction log file for '/usr/share/filebeat/data/registry/filebeat'. Active transaction id=0

2022-04-29T07:40:36.865+0300 WARN beater/filebeat.go:411 Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.

2022-04-29T07:40:36.865+0300 INFO [registrar] registrar/registrar.go:109 States Loaded from registrar: 0

2022-04-29T07:40:36.866+0300 INFO [crawler] beater/crawler.go:71 Loading Inputs: 1

2022-04-29T07:40:36.923+0300 WARN [cfgwarn] log/input.go:89 DEPRECATED: Log input. Use Filestream input instead. Will be removed in version:

2022-04-29T07:40:36.925+0300 INFO [input] log/input.go:171 Configured paths: [/mnt//docker/containers//*.log] {"input_id": "a72325fd-5ca1-4974-bf41-3ab9495eebe2"}

2022-04-29T07:40:36.925+0300 INFO [crawler] beater/crawler.go:141 Starting input (ID: 11593768310582725890)

2022-04-29T07:40:36.925+0300 INFO [crawler] beater/crawler.go:108 Loading and starting Inputs completed. Enabled inputs: 1

2022-04-29T07:41:06.876+0300 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000},"quota":{"us":100000}},"id":"/","stats":{"periods":32,"throttled":{"ns":637757510,"periods":4}}},"cpuacct":{"id":"/","total":{"ns":616818299}},"memory":{"id":"/","mem":{"limit":{"bytes":8589934592},"usage":{"bytes":65220608}}}},"cpu":{"system":{"ticks":150,"time":{"ms":159}},"total":{"ticks":520,"time":{"ms":533},"value":520},"user":{"ticks":370,"time":{"ms":374}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral_id":"6aea41bc-44e9-45c2-baa1-ba2dd793ba89","uptime":{"ms":30304},"version":"7.16.3"},"memstats":{"gc_next":21913536,"memory_alloc":11036032,"memory_sys":49890312,"memory_total":58833040,"rss":115126272},"runtime":{"goroutines":127}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0},"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0},"queue":{"max_events":4096}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":96},"load":{"1":0.45,"15":0.54,"5":0.51,"norm":{"1":0.0047,"15":0.0056,"5":0.0053}}}}}}

2022-04-29T07:41:36.878+0300 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"stats":{"periods":11}},"cpuacct":{"total":{"ns":17161736}},"memory":{"mem":{"usage":{"bytes":-122880}}}},"cpu":{"system":{"ticks":170,"time":{"ms":18}},"total":{"ticks":560,"time":{"ms":35},"value":560},"user":{"ticks":390,"time":{"ms":17}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral_id":"6aea41bc-44e9-45c2-baa1-ba2dd793ba89","uptime":{"ms":60315},"version":"7.16.3"},"memstats":{"gc_next":21913536,"memory_alloc":13139640,"memory_total":60936648,"rss":115339264},"runtime":{"goroutines":127}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.49,"15":0.54,"5":0.51,"norm":{"1":0.0051,"15":0.0056,"5":0.0053}}}}}}

4

Hi,

can you share the filebeat config.

5

In future please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

But yes, seeing your config would be handy.

6

My filebeat.yml as following :

#==========================  Modules configuration =============================

filebeat.inputs:
  - type: container
    enabled: true
    paths:
      - "/mnt/*/docker/containers/*/*.log"
    multiline.pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2}'
    multiline.negate: true
    multiline.match: after
    exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
    processors:
      - add_docker_metadata:
          match_source_index: 4 #subfolder for extract container id from path
      - add_host_metadata:
          cache.ttl: 5m
      - add_fields:
          target: project
          fields:
            name: test
            id: 'deneme'

#=========================== Filebeat inputs ==============================

output.logstash:
  hosts: ["logstash:5044"]
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.