UNABLE TO START FILEBEAT ON ELASTIC V7.9 LAB

Elastic Certified Observability Engineer practice lab when i start the filebeat with the command ./filebeat i get the below logs the filebeat is not starting.

[elastic@mysql filebeat]$ ./filebeat -e
2023-02-23T07:31:49.912Z        INFO    instance/beat.go:640    Home path: [/home/elastic/filebeat] Config path: [/home/elastic/filebeat] Data path: [/home/elastic/filebeat/data] Logs path: [/home/elastic/filebeat/logs]
2023-02-23T07:31:49.912Z        INFO    instance/beat.go:648    Beat ID: d75cecf0-a0d9-4d98-a904-176ef3cf51c9
2023-02-23T07:31:49.919Z        INFO    [seccomp]       seccomp/seccomp.go:124  Syscall filter successfully installed
2023-02-23T07:31:49.919Z        INFO    [beat]  instance/beat.go:976    Beat info       {"system_info": {"beat": {"path": {"config": "/home/elastic/filebeat", "data": "/home/elastic/filebeat/data", "home": "/home/elastic/filebeat", "logs": "/home/elastic/filebeat/logs"}, "type": "filebeat", "uuid": "d75cecf0-a0d9-4d98-a904-176ef3cf51c9"}}}
2023-02-23T07:31:49.919Z        INFO    [beat]  instance/beat.go:985    Build info      {"system_info": {"build": {"commit": "ad823eca4cc74439d1a44351c596c12ab51054f5", "libbeat": "7.9.1", "time": "2020-09-01T19:58:51.000Z", "version": "7.9.1"}}}
2023-02-23T07:31:49.919Z        INFO    [beat]  instance/beat.go:988    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.14.7"}}}
2023-02-23T07:31:49.920Z        INFO    [beat]  instance/beat.go:992    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2023-02-23T07:25:40Z","containerized":true,"name":"mysql","ip":["127.0.0.1/8","172.18.0.13/24"],"kernel_version":"4.4.0-1117-aws","mac":["02:42:ac:12:00:0d"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":8,"patch":2003,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0}}}
2023-02-23T07:31:49.921Z        INFO    [beat]  instance/beat.go:1021   Process info    {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"permitted":null,"effective":null,"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/home/elastic/filebeat", "exe": "/home/elastic/filebeat/filebeat", "name": "filebeat", "pid": 224, "ppid": 123, "seccomp": {"mode":"filter"}, "start_time": "2023-02-23T07:31:48.940Z"}}}
2023-02-23T07:31:49.921Z        INFO    instance/beat.go:299    Setup Beat: filebeat; Version: 7.9.1
2023-02-23T07:31:49.921Z        INFO    [index-management]      idxmgmt/std.go:184      Set output.elasticsearch.index to 'filebeat-7.9.1' as ILM is enabled.
2023-02-23T07:31:49.922Z        INFO    eslegclient/connection.go:99    elasticsearch url: https://elasticsearch:9200
2023-02-23T07:31:49.922Z        INFO    [publisher]     pipeline/module.go:113  Beat name: mysql
2023-02-23T07:31:49.926Z        INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2023-02-23T07:31:49.927Z        INFO    instance/beat.go:450    filebeat start running.
2023-02-23T07:31:49.928Z        INFO    memlog/store.go:119     Loading data file of '/home/elastic/filebeat/data/registry/filebeat' succeeded. Active transaction id=0
2023-02-23T07:31:49.928Z        INFO    memlog/store.go:124     Finished loading transaction log file for '/home/elastic/filebeat/data/registry/filebeat'. Active transaction id=34
2023-02-23T07:31:49.929Z        INFO    [registrar]     registrar/registrar.go:109      States Loaded from registrar: 1
2023-02-23T07:31:49.929Z        INFO    [crawler]       beater/crawler.go:71    Loading Inputs: 1
2023-02-23T07:31:49.929Z        INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:93     add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"214514861431"},"availability_zone":"eu-west-1b","image":{"id":"ami-0dd224a9da0499f7c"},"instance":{"id":"i-0534d1e549ac677d7"},"machine":{"type":"t3.xlarge"},"provider":"aws","region":"eu-west-1"}
2023-02-23T07:31:49.930Z        INFO    log/input.go:157        Configured paths: [/var/log/*.log]
2023-02-23T07:31:49.930Z        INFO    [crawler]       beater/crawler.go:141   Starting input (ID: 11204088409762598069)
2023-02-23T07:31:49.931Z        INFO    log/harvester.go:297    Harvester started for file: /var/log/mysqld.log
2023-02-23T07:31:49.932Z        INFO    log/input.go:157        Configured paths: [/var/log/mysql/mysql-slow.log* /var/lib/mysql/mysql-slow.log]
2023-02-23T07:31:49.935Z        ERROR   fileset/factory.go:103  Error creating input: Can only start an input when all related states are finished: {Id: native::1027528-66305, Finished: false, Fileinfo: &{mysqld.log 19525 420 {332000000 63812733959 0x607c540} {66305 1027528 1 33188 27 27 0 0 19525 4096 48 {1602017717 0} {1677137159 332000000} {1677137159 332000000} [0 0 0]}}, Source: /var/log/mysqld.log, Offset: 11715, Timestamp: 2023-02-23 07:31:49.931609031 +0000 UTC m=+0.230395355, TTL: -1ns, Type: log, Meta: map[], FileStateOS: 1027528-66305}
2023-02-23T07:31:49.934Z        ERROR   log/input.go:519        Harvester could not be started on new file: /var/log/yum.log, Err: error setting up harvester: Harvester setup failed. Unexpected file opening error: Failed opening /var/log/yum.log: open /var/log/yum.log: permission denied
2023-02-23T07:31:49.936Z        INFO    beater/crawler.go:148   Stopping Crawler
2023-02-23T07:31:49.937Z        INFO    beater/crawler.go:158   Stopping 1 inputs
2023-02-23T07:31:49.937Z        INFO    [crawler]       beater/crawler.go:163   Stopping input: 11204088409762598069
2023-02-23T07:31:49.937Z        INFO    input/input.go:136      input ticker stopped
2023-02-23T07:31:49.937Z        INFO    log/harvester.go:320    Reader was closed: /var/log/mysqld.log. Closing.
2023-02-23T07:31:49.937Z        INFO    beater/crawler.go:178   Crawler stopped
2023-02-23T07:31:49.937Z        INFO    [registrar]     registrar/registrar.go:132      Stopping Registrar
2023-02-23T07:31:49.937Z        INFO    [registrar]     registrar/registrar.go:166      Ending Registrar
2023-02-23T07:31:49.938Z        INFO    [registrar]     registrar/registrar.go:137      Registrar stopped
2023-02-23T07:31:49.946Z        INFO    [monitoring]    log/log.go:153  Total non-zero metrics  {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":52}},"total":{"ticks":130,"time":{"ms":136},"value":0},"user":{"ticks":80,"time":{"ms":84}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":8},"info":{"ephemeral_id":"1b7bb1b0-a5cb-49c9-9dd7-d1451fa68e7e","uptime":{"ms":236}},"memstats":{"gc_next":16887968,"memory_alloc":10123144,"memory_total":37240176,"rss":69107712},"runtime":{"goroutines":12}},"filebeat":{"events":{"active":85,"added":87,"done":2},"harvester":{"closed":1,"open_files":0,"running":0,"started":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":84,"filtered":3,"published":84,"total":87}}},"registrar":{"states":{"current":1,"update":2},"writes":{"success":2,"total":2}},"system":{"cpu":{"cores":4},"load":{"1":0.78,"15":0.86,"5":1.42,"norm":{"1":0.195,"15":0.215,"5":0.355}}}}}}
2023-02-23T07:31:49.946Z        INFO    [monitoring]    log/log.go:154  Uptime: 236.81664ms
2023-02-23T07:31:49.946Z        INFO    [monitoring]    log/log.go:131  Stopping metrics logging.
2023-02-23T07:31:49.946Z        INFO    instance/beat.go:456    filebeat stopped.
2023-02-23T07:31:49.946Z        ERROR   instance/beat.go:951    Exiting: Failed to start crawler: creating module reloader failed: Can only start an input when all related states are finished: {Id: native::1027528-66305, Finished: false, Fileinfo: &{mysqld.log 19525 420 {332000000 63812733959 0x607c540} {66305 1027528 1 33188 27 27 0 0 19525 4096 48 {1602017717 0} {1677137159 332000000} {1677137159 332000000} [0 0 0]}}, Source: /var/log/mysqld.log, Offset: 11715, Timestamp: 2023-02-23 07:31:49.931609031 +0000 UTC m=+0.230395355, TTL: -1ns, Type: log, Meta: map[], FileStateOS: 1027528-66305}
Exiting: Failed to start crawler: creating module reloader failed: Can only start an input when all related states are finished: {Id: native::1027528-66305, Finished: false, Fileinfo: &{mysqld.log 19525 420 {332000000 63812733959 0x607c540} {66305 1027528 1 33188 27 27 0 0 19525 4096 48 {1602017717 0} {1677137159 332000000} {1677137159 332000000} [0 0 0]}}, Source: /var/log/mysqld.log, Offset: 11715, Timestamp: 2023-02-23 07:31:49.931609031 +0000 UTC m=+0.230395355, TTL: -1ns, Type: log, Meta: map[], FileStateOS: 1027528-66305}

Please assist.

Welcome to our community! :smiley:

Please take a look at Dec 10th, 2022: [EN] Asking top notch technical questions to get you help quicker! and update your post with some more information, this will allow us to help you.

Thanks @warkolm post updated.

Hi @chikugerson ,

it looks like you might be running more than one instance of Filebeat. Please run ps aux | grep filebeat to make sure no other instance of Filebeat is running before you start it again.

Also, it looks like you are trying to ingest /var/log/yum.log, but you don't have permission to do that. Please check your configuration files and make sure you only ingest the files requested in the lab instructions.

Best,
Andre