# Help me in configure filebeat index **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 20, 2022, 5:55am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253 "2022-05-20T05:55:56Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 20, 2022, 5:55am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/1 "2022-05-20T05:55:56Z")
i use this command, it will not shown my filebeat index > curl [http://localhost:9200/\_cat/indices?v](http://localhost:9200/_cat/indices?v) And i use below command to know the status > filebeat -e -c /etc/filebeat/filebeat.yml > **output:** ```auto {"log.level":"info","@timestamp":"2022-05-20T06:46:04.062Z","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T06:46:04.063Z","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: a985c9e2-fd39-42dd-9b1d-6c45a4629a2b","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T06:46:04.069Z","log.origin":{"file.name":"instance/beat.go","file.line":424},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-05-20T06:46:04.069Z","log.origin":{"file.name":"instance/beat.go","file.line":1038},"message":"Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).","service.name":"filebeat","ecs.version":"1.6.0"} Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data). ``` Please anyone help me to fix this. Thanks in advance. ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 20, 2022, 10:49am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/2 "2022-05-20T10:49:48Z")
Hi @Rajesh119 when you tried to run the below command, the filebeat service was still running , this is why you have this error. > [@Rajesh119](#): > > filebeat -e -c /etc/filebeat/filebeat.yml > [@Rajesh119](#): > > ```auto > {"log.level":"error","@timestamp":"2022-05-20T06:46:04.069Z","log.origin":{"file.name":"instance/beat.go","file.line":1038},"message":"Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).","service.name":"filebeat","ecs.version":"1.6.0"} > Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data). > > ``` ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 20, 2022, 10:57am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/3 "2022-05-20T10:57:26Z")
@ibra_013 Thank you It shows like this > sudo service filebeat status **Output:** ```auto filebeat-god (pid 3695) is running... ``` ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 20, 2022, 11:01am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/4 "2022-05-20T11:01:59Z")
Hi @Rajesh119 So i you want to test the filebeat output on the console `(filebeat -e )`, you have to make sure that the filebeat service is stopped. ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 20, 2022, 11:07am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/5 "2022-05-20T11:07:32Z")
Hi @ibra_013 once i stop the filebeat service and check status > filebeat -e ```auto {"log.level":"info","@timestamp":"2022-05-20T11:06:30.604Z","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.605Z","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: a985c9e2-fd39-42dd-9b1d-6c45a4629a2b","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.609Z","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.609Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"a985c9e2-fd39-42dd-9b1d-6c45a4629a2b"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.609Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1072},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"045da3a1bb89944373c33332c18ca99ef6192df2","libbeat":"8.2.0","time":"2022-04-19T23:31:06.000Z","version":"8.2.0"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.609Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1075},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":1,"version":"go1.17.8"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.610Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1079},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-05-16T12:47:00Z","containerized":false,"name":"mail.thapos.net","ip":["127.0.0.1/8","::1/128","172.31.19.214/20","fe80::8bf:52ff:febd:8a4/64"],"kernel_version":"4.14.214-118.339.amzn1.x86_64","mac":["0a:bf:52:bd:08:a4"],"os":{"type":"linux","family":"redhat","platform":"amzn","name":"Amazon Linux AMI","version":"2018.03","major":2018,"minor":3,"patch":0},"timezone":"UTC","timezone_offset_sec":0,"id":"ef32a8ed995e86063ec99c115b0b9d78"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.610Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null},"cwd":"/root","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":4253,"ppid":2146,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-05-20T11:06:30.330Z"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.610Z","log.origin":{"file.name":"instance/beat.go","file.line":325},"message":"Setup Beat: filebeat; Version: 8.2.0","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.611Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: http://34.205.74.243:9200","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.611Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: mail.thapos.net","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.611Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.612Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":142},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.612Z","log.origin":{"file.name":"instance/beat.go","file.line":505},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.612Z","log.origin":{"file.name":"memlog/store.go","file.line":134},"message":"Finished loading transaction log file for '/var/lib/filebeat/filebeat'. Active transaction id=8","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.612Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.612Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.612Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.612Z","log.logger":"cfgwarn","log.origin":{"file.name":"log/input.go","file.line":89},"message":"DEPRECATED: Log input. Use Filestream input instead.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.613Z","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":171},"message":"Configured paths: [/home/thapos/tomcat/tomcat8/logs/catalina.out]","service.name":"filebeat","input_id":"f0e80e3a-2020-407d-a26b-8f669d0d9eca","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.613Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 8813593274292630345)","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.614Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: tomcat (log)","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.615Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.616Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":164},"message":"Config reloader started","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.616Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: tomcat (log)","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.643Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.644Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.644Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.644Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.644Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-20T11:06:30.644Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.644Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: http://34.205.74.243:9200","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.651Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":105},"message":"add_cloud_metadata: hosting provider type detected as aws, metadata={\"cloud\":{\"account\":{\"id\":\"632467875476\"},\"availability_zone\":\"us-east-1c\",\"image\":{\"id\":\"ami-14c5486b\"},\"instance\":{\"id\":\"i-0e4a37307f16515d3\"},\"machine\":{\"type\":\"t2.small\"},\"provider\":\"aws\",\"region\":\"us-east-1\",\"service\":{\"name\":\"EC2\"}}}","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.654Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.2.0","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.658Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":224},"message":"Loading of config files completed.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.658Z","log.origin":{"file.name":"udp/input.go","file.line":98},"message":"Starting UDP input","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:06:30.658Z","log.logger":"UDP","log.origin":{"file.name":"dgram/server.go","file.line":99},"message":"Started listening for UDP connection","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:00.614Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":20}},"total":{"ticks":170,"time":{"ms":170},"value":0},"user":{"ticks":150,"time":{"ms":150}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":12},"info":{"ephemeral_id":"9166e7ea-1600-434b-bdd2-00976b2f1c12","uptime":{"ms":30084},"version":"8.2.0"},"memstats":{"gc_next":26460864,"memory_alloc":13813096,"memory_sys":41501704,"memory_total":60948384,"rss":133533696},"runtime":{"goroutines":34}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1,"starts":1},"reloads":1,"scans":1},"output":{"events":{"active":0},"type":"elasticsearch"},"pipeline":{"clients":2,"events":{"active":0,"filtered":1,"total":1},"queue":{"max_events":4096}}},"registrar":{"states":{"current":1,"update":1},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} ^C{"log.level":"info","@timestamp":"2022-05-20T11:07:01.423Z","log.origin":{"file.name":"beater/filebeat.go","file.line":425},"message":"Stopping filebeat","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.424Z","log.origin":{"file.name":"beater/crawler.go","file.line":155},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.424Z","log.origin":{"file.name":"beater/crawler.go","file.line":165},"message":"Stopping 1 inputs","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.428Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":227},"message":"Dynamic config reloader stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.428Z","log.logger":"reload","log.origin":{"file.name":"cfgfile/list.go","file.line":129},"message":"Stopping 1 runners ...","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.428Z","log.origin":{"file.name":"input/input.go","file.line":134},"message":"input ticker stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.428Z","log.origin":{"file.name":"udp/input.go","file.line":113},"message":"Stopping UDP input","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.logger":"udp","log.origin":{"file.name":"dgram/handler.go","file.line":73},"message":"Connection has been closed","service.name":"filebeat","address":"localhost:9523","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":170},"message":"Stopping input: 8813593274292630345","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.origin":{"file.name":"input/input.go","file.line":134},"message":"input ticker stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.origin":{"file.name":"beater/crawler.go","file.line":185},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":132},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":166},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.429Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":137},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.442Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":192},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":20}},"total":{"ticks":180,"time":{"ms":180},"value":0},"user":{"ticks":160,"time":{"ms":160}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral_id":"9166e7ea-1600-434b-bdd2-00976b2f1c12","uptime":{"ms":30912},"version":"8.2.0"},"memstats":{"gc_next":26460864,"memory_alloc":14114672,"memory_sys":41501704,"memory_total":61249960,"rss":133533696},"runtime":{"goroutines":15}},"filebeat":{"events":{"active":0,"added":1,"done":1},"harvester":{"closed":0,"open_files":0,"running":0,"skipped":0,"started":0},"input":{"log":{"files":{"renamed":0,"truncated":0}},"netflow":{"flows":0,"packets":{"dropped":0,"received":0}}}},"libbeat":{"config":{"module":{"running":1,"starts":1,"stops":0},"reloads":1,"scans":1},"output":{"events":{"acked":0,"active":0,"batches":0,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":0},"read":{"bytes":0,"errors":0},"type":"elasticsearch","write":{"bytes":0,"errors":0}},"pipeline":{"clients":0,"events":{"active":0,"dropped":0,"failed":0,"filtered":1,"published":0,"retry":0,"total":1},"queue":{"acked":0,"max_events":4096}}},"registrar":{"states":{"cleanup":0,"current":1,"update":1},"writes":{"fail":0,"success":1,"total":1}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.442Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":193},"message":"Uptime: 30.913237556s","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.442Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":160},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-20T11:07:01.442Z","log.origin":{"file.name":"instance/beat.go","file.line":510},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"} ``` ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 20, 2022, 11:10am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/6 "2022-05-20T11:10:47Z")
Hi @Rajesh119 We need some context, what are you trying to achieve?, please share the filebeat.yml file. ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 20, 2022, 11:16am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/7 "2022-05-20T11:16:33Z")
Hi @ibra_013 This is my .yml file > vim /etc/filebeat/filebeat.yml ```auto ###################### Filebeat Configuration Example ######################### # This file is an example configuration file highlighting only the most common # options. The filebeat.reference.yml file from the same directory contains all the # supported options with more comments. You can use it as a reference. # # You can find the full configuration reference here: # https://www.elastic.co/guide/en/beats/filebeat/index.html # For more available modules and options, please see the filebeat.reference.yml sample # configuration file. # ============================== Filebeat inputs =============================== filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. # filestream is an input for collecting log messages from files. - type: log # Unique ID among all inputs, an ID is required. #id: my-filestream-id # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - /home/thapos/tomcat/tomcat8/logs/catalina.out #- c:\programdata\elasticsearch\logs\* # Exclude lines. A list of regular expressions to match. It drops the lines that are # matching any regular expression from the list. #exclude_lines: ['^DBG'] # Include lines. A list of regular expressions to match. It exports the lines that are # matching any regular expression from the list. #include_lines: ['^ERR', '^WARN'] # Exclude files. A list of regular expressions to match. Filebeat drops the files that # are matching any regular expression from the list. By default, no files are dropped. #prospector.scanner.exclude_files: ['.gz$'] # Optional additional fields. These fields can be freely picked # to add additional information to the crawled log files for filtering #fields: # level: debug # review: 1 # ============================== Filebeat modules ============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/*.yml # Set to true to enable config reloading reload.enabled: false # Period on which files under path should be checked for changes #reload.period: 10s # ======================= Elasticsearch template setting ======================= setup.template.settings: index.number_of_shards: 1 #index.codec: best_compression #_source.enabled: false # ================================== General =================================== # The name of the shipper that publishes the network data. It can be used to group # all the transactions sent by a single shipper in the web interface. #name: filebeat.registry.path: ${path.data}/. # The tags of the shipper are included in their own field with each # transaction published. #tags: ["service-X", "web-tier"] setup.ilm.overwrite: true # Optional fields that you can specify to add additional information to the # output. #fields: # env: staging # ================================= Dashboards ================================= # These settings control loading the sample dashboards to the Kibana index. Loading # the dashboards is disabled by default and can be enabled either by setting the # options here or by using the `setup` command. #setup.dashboards.enabled: false # The URL from where to download the dashboards archive. By default this URL # has a value which is computed based on the Beat name and version. For released # versions, this URL points to the dashboard archive on the artifacts.elastic.co # website. #setup.dashboards.url: # =================================== Kibana =================================== # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API. # This requires a Kibana endpoint configuration. setup.kibana: host: "34.205.74.243:5601" # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 #host: "localhost:5601" # Kibana Space ID # ID of the Kibana Space into which the dashboards should be loaded. By default, # the Default Space will be used. #space.id: # =============================== Elastic Cloud ================================ # These settings simplify using Filebeat with the Elastic Cloud (https://cloud.elastic.co/). # The cloud.id setting overwrites the `output.elasticsearch.hosts` and # `setup.kibana.host` options. # You can find the `cloud.id` in the Elastic Cloud web UI. #cloud.id: # The cloud.auth setting overwrites the `output.elasticsearch.username` and # `output.elasticsearch.password` settings. The format is `:`. #cloud.auth: # ================================== Outputs =================================== # Configure what output to use when sending the data collected by the beat. # ---------------------------- Elasticsearch Output ---------------------------- output.elasticsearch: # Array of hosts to connect to. hosts: ["34.205.74.243:9200"] #index: "elastic" # Protocol - either `http` (default) or `https`. #protocol: "https" # Authentication credentials - either API key or username/password. #api_key: "id:api_key" #username: "elastic" #password: "changeme" # ------------------------------ Logstash Output ------------------------------- #output.logstash: # The Logstash hosts #hosts: ["localhost:5044"] # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] # Certificate for SSL client authentication #ssl.certificate: "/etc/pki/client/cert.pem" # Client Certificate Key #ssl.key: "/etc/pki/client/cert.key" # ================================= Processors ================================= processors: - add_host_metadata: when.not.contains.tags: forwarded - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ # ================================== Logging =================================== # Sets log level. The default log level is info. # Available log levels are: error, warning, info, debug #logging.level: debug # At debug level, you can selectively enable logging only for some components. # To enable all selectors use ["*"]. Examples of other selectors are "beat", # "publisher", "service". #logging.selectors: ["*"] # ============================= X-Pack Monitoring ============================== # Filebeat can export internal metrics to a central Elasticsearch monitoring # cluster. This requires xpack monitoring to be enabled in Elasticsearch. The # reporting is disabled by default. # Set to true to enable the monitoring reporter. #monitoring.enabled: false # Sets the UUID of the Elasticsearch cluster under which monitoring data for this # Filebeat instance will appear in the Stack Monitoring UI. If output.elasticsearch # is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch. #monitoring.cluster_uuid: # Uncomment to send the metrics to Elasticsearch. Most settings from the # Elasticsearch output are accepted here as well. # Note that the settings should point to your Elasticsearch *monitoring* cluster. # Any setting that is not set is automatically inherited from the Elasticsearch # output configuration, so if you have the Elasticsearch output configured such # that it is pointing to your Elasticsearch monitoring cluster, you can simply # uncomment the following line. #monitoring.elasticsearch: # ============================== Instrumentation =============================== # Instrumentation support for the filebeat. #instrumentation: # Set to true to enable instrumentation of filebeat. #enabled: false # Environment in which filebeat is running on (eg: staging, production, etc.) #environment: "" # APM Server hosts to report instrumentation results to. #hosts: # - http://localhost:8200 # API Key for the APM Server(s). # If api_key is set then secret_token will be ignored. #api_key: # Secret token for the APM Server(s). #secret_token: # ================================= Migration ================================== # This allows to enable 6.7 migration aliases #migration.6_to_7.enabled: true ``` ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 20, 2022, 11:24am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/8 "2022-05-20T11:24:44Z")
Hi @Rajesh119 can you share the output of the below commands: ```auto filebeat modules list | head cat /etc/filebeat/modules.d/tomcat.yml ``` ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 20, 2022, 11:41am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/9 "2022-05-20T11:41:33Z")
Hi @ibra_013 **output:** > filebeat modules list | head ```auto Enabled: tomcat Disabled: activemq apache auditd aws awsfargate azure ``` > cat /etc/filebeat/modules.d/tomcat.yml ```auto # Module: tomcat # Docs: https://www.elastic.co/guide/en/beats/filebeat/8.2/filebeat-module-tomcat.html - module: tomcat log: enabled: true # Set which input to use between udp (default), tcp or file. # var.input: udp # var.syslog_host: localhost # var.syslog_port: 9501 # Set paths for the log files when file input is used. # var.paths: # - /var/log/tomcat/*.log # Toggle output of non-ECS fields (default true). # var.rsa_fields: true # Set custom timezone offset. # "local" (default) for system timezone. # "+02:00" for GMT+02:00 # var.tz_offset: local ``` ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 23, 2022, 7:39am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/10 "2022-05-23T07:39:31Z")
Hi @ibra_013 Please Help ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 23, 2022, 9:36am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/11 "2022-05-23T09:36:58Z")
Hi @Rajesh119 Add these logging parameters to your filebeat and restart the filebeat service to see the logs. ```auto logging.level: info logging.to_files: true logging.files: path: /var/log/filebeat name: filebeat keepfiles: 7 permissions: 0640 ``` and set this to false > [@Rajesh119](#): > > `setup.ilm.overwrite: true` ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 23, 2022, 11:14am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/12 "2022-05-23T11:14:15Z")
Hi @ibra_013 First of all Thank you for your response. > filebeat -e -c /etc/filebeat/filebeat.yml **Output:** ```auto {"log.level":"info","@timestamp":"2022-05-23T11:12:06.718Z","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:12:06.719Z","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: a985c9e2-fd39-42dd-9b1d-6c45a4629a2b","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:12:06.725Z","log.origin":{"file.name":"instance/beat.go","file.line":424},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-05-23T11:12:06.725Z","log.origin":{"file.name":"instance/beat.go","file.line":1038},"message":"Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).","service.name":"filebeat","ecs.version":"1.6.0"} Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data). ``` ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 23, 2022, 11:18am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/13 "2022-05-23T11:18:46Z")
Hi @Rajesh119 in order to ran this command you have to stop filebeat service > [@Rajesh119](#): > > filebeat -e -c /etc/filebeat/filebeat.yml ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 23, 2022, 11:21am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/14 "2022-05-23T11:21:27Z")
Hi @ibra_013 I stopped the filebeat and run the command > filebeat -e -c /etc/filebeat/filebeat.yml **Output:** ```auto {"log.level":"info","@timestamp":"2022-05-23T11:26:11.385Z","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.385Z","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: a985c9e2-fd39-42dd-9b1d-6c45a4629a2b","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.390Z","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.390Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"a985c9e2-fd39-42dd-9b1d-6c45a4629a2b"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.390Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1072},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"045da3a1bb89944373c33332c18ca99ef6192df2","libbeat":"8.2.0","time":"2022-04-19T23:31:06.000Z","version":"8.2.0"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.391Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1075},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":1,"version":"go1.17.8"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.391Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1079},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-05-16T12:47:00Z","containerized":false,"name":"mail.thapos.net","ip":["127.0.0.1/8","::1/128","172.31.19.214/20","fe80::8bf:52ff:febd:8a4/64"],"kernel_version":"4.14.214-118.339.amzn1.x86_64","mac":["0a:bf:52:bd:08:a4"],"os":{"type":"linux","family":"redhat","platform":"amzn","name":"Amazon Linux AMI","version":"2018.03","major":2018,"minor":3,"patch":0},"timezone":"UTC","timezone_offset_sec":0,"id":"ef32a8ed995e86063ec99c115b0b9d78"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.392Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null},"cwd":"/root","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":8646,"ppid":8177,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-05-23T11:26:11.120Z"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.392Z","log.origin":{"file.name":"instance/beat.go","file.line":325},"message":"Setup Beat: filebeat; Version: 8.2.0","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.393Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: http://34.205.74.243:9200","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.393Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: mail.thapos.net","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.393Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.394Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":142},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.394Z","log.origin":{"file.name":"instance/beat.go","file.line":505},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.394Z","log.origin":{"file.name":"memlog/store.go","file.line":134},"message":"Finished loading transaction log file for '/var/lib/filebeat/filebeat'. Active transaction id=16","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.395Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.395Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.395Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.paths.0 filebeat.inputs.0.type]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.395Z","log.logger":"cfgwarn","log.origin":{"file.name":"log/input.go","file.line":89},"message":"DEPRECATED: Log input. Use Filestream input instead.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.395Z","log.logger":"input","log.origin":{"file.name":"log/input.go","file.line":171},"message":"Configured paths: [/home/thapos/tomcat/tomcat8/logs/catalina.out]","service.name":"filebeat","input_id":"c6698066-efc9-4dae-a774-e88b4af6ece2","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.395Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 8813593274292630345)","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.396Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: tomcat (log)","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.397Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.398Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":164},"message":"Config reloader started","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.399Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: tomcat (log)","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"warn","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"cfgwarn","log.origin":{"file.name":"registered_domain/registered_domain.go","file.line":61},"message":"BETA: The registered_domain processor is beta.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.428Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: http://34.205.74.243:9200","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.435Z","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/add_cloud_metadata.go","file.line":105},"message":"add_cloud_metadata: hosting provider type detected as aws, metadata={\"cloud\":{\"account\":{\"id\":\"632467875476\"},\"availability_zone\":\"us-east-1c\",\"image\":{\"id\":\"ami-14c5486b\"},\"instance\":{\"id\":\"i-0e4a37307f16515d3\"},\"machine\":{\"type\":\"t2.small\"},\"provider\":\"aws\",\"region\":\"us-east-1\",\"service\":{\"name\":\"EC2\"}}}","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.437Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":287},"message":"Attempting to connect to Elasticsearch version 8.2.0","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.441Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":224},"message":"Loading of config files completed.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.441Z","log.origin":{"file.name":"udp/input.go","file.line":98},"message":"Starting UDP input","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:11.441Z","log.logger":"UDP","log.origin":{"file.name":"dgram/server.go","file.line":99},"message":"Started listening for UDP connection","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T11:26:41.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":30}},"total":{"ticks":170,"time":{"ms":170},"value":0},"user":{"ticks":140,"time":{"ms":140}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":12},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":30084},"version":"8.2.0"},"memstats":{"gc_next":26556240,"memory_alloc":13811192,"memory_sys":37307400,"memory_total":60991840,"rss":132804608},"runtime":{"goroutines":34}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1,"starts":1},"reloads":1,"scans":1},"output":{"events":{"active":0},"type":"elasticsearch"},"pipeline":{"clients":2,"events":{"active":0,"filtered":1,"total":1},"queue":{"max_events":4096}}},"registrar":{"states":{"current":1,"update":1},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:27:11.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":30},"total":{"ticks":170,"value":0},"user":{"ticks":140}},"handles":{"limit":{"hard":4096,"soft":1024},"open":12},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":60084},"version":"8.2.0"},"memstats":{"gc_next":26556240,"memory_alloc":14048640,"memory_total":61229288,"rss":132804608},"runtime":{"goroutines":34}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:27:41.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":30},"total":{"ticks":170,"value":0},"user":{"ticks":140}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":90084},"version":"8.2.0"},"memstats":{"gc_next":26556240,"memory_alloc":14296016,"memory_total":61476664,"rss":132804608},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:28:11.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":30},"total":{"ticks":180,"time":{"ms":10},"value":0},"user":{"ticks":150,"time":{"ms":10}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":120084},"version":"8.2.0"},"memstats":{"gc_next":26556240,"memory_alloc":14522304,"memory_total":61702952,"rss":132804608},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:28:41.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":10}},"total":{"ticks":210,"time":{"ms":30},"value":0},"user":{"ticks":170,"time":{"ms":20}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":150084},"version":"8.2.0"},"memstats":{"gc_next":23205408,"memory_alloc":11635952,"memory_total":61869104,"rss":123580416},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:29:11.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":40},"total":{"ticks":220,"time":{"ms":10},"value":0},"user":{"ticks":180,"time":{"ms":10}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":180084},"version":"8.2.0"},"memstats":{"gc_next":23205408,"memory_alloc":11980504,"memory_total":62213656,"rss":123580416},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:29:41.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":40},"total":{"ticks":220,"value":0},"user":{"ticks":180}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":210083},"version":"8.2.0"},"memstats":{"gc_next":23205408,"memory_alloc":12093976,"memory_total":62327128,"rss":123580416},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.07,"15":0,"5":0.02,"norm":{"1":0.07,"15":0,"5":0.02}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:30:11.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":10}},"total":{"ticks":230,"time":{"ms":10},"value":0},"user":{"ticks":180}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":240083},"version":"8.2.0"},"memstats":{"gc_next":23205408,"memory_alloc":12305776,"memory_total":62538928,"rss":123580416},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.04,"15":0,"5":0.01,"norm":{"1":0.04,"15":0,"5":0.01}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T11:30:41.395Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":50},"total":{"ticks":250,"time":{"ms":20},"value":0},"user":{"ticks":200,"time":{"ms":20}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":11},"info":{"ephemeral_id":"24291d39-e901-4296-bb74-1a8ebf4a2310","uptime":{"ms":270084},"version":"8.2.0"},"memstats":{"gc_next":23112592,"memory_alloc":11619488,"memory_total":62813720,"rss":123580416},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0}},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.02,"15":0,"5":0.01,"norm":{"1":0.02,"15":0,"5":0.01}}}},"ecs.version":"1.6.0"}} ``` ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 23, 2022, 12:38pm UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/15 "2022-05-23T12:38:01Z")
hi @Rajesh119 on the tomcat module configure the path and comment the part of filebeat inputs > [@Rajesh119](#): > > ```auto > filebeat.inputs: > > # Each - is an input. Most options can be set at the input level, so > # you can use different inputs for various configurations. > # Below are the input specific configurations. > > # filestream is an input for collecting log messages from files. > - type: log > > # Unique ID among all inputs, an ID is required. > #id: my-filestream-id > > # Change to true to enable this input configuration. > enabled: true > > # Paths that should be crawled and fetched. Glob based paths. > paths: > - /home/thapos/tomcat/tomcat8/logs/catalina.out > > ``` ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 23, 2022, 1:02pm UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/16 "2022-05-23T13:02:54Z")
Hi @ibra_013 After comment the filebeat inputs it's shows like this: > filebeat -e -c /etc/filebeat/filebeat.yml ```auto {"log.level":"info","@timestamp":"2022-05-23T12:58:11.675Z","log.origin":{"file.name":"instance/beat.go","file.line":685},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.676Z","log.origin":{"file.name":"instance/beat.go","file.line":693},"message":"Beat ID: a985c9e2-fd39-42dd-9b1d-6c45a4629a2b","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.679Z","log.logger":"seccomp","log.origin":{"file.name":"seccomp/seccomp.go","file.line":124},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.680Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1063},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"a985c9e2-fd39-42dd-9b1d-6c45a4629a2b"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.680Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1072},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"045da3a1bb89944373c33332c18ca99ef6192df2","libbeat":"8.2.0","time":"2022-04-19T23:31:06.000Z","version":"8.2.0"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.680Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1075},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":1,"version":"go1.17.8"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.680Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1079},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-05-16T12:47:00Z","containerized":false,"name":"mail.thapos.net","ip":["127.0.0.1/8","::1/128","172.31.19.214/20","fe80::8bf:52ff:febd:8a4/64"],"kernel_version":"4.14.214-118.339.amzn1.x86_64","mac":["0a:bf:52:bd:08:a4"],"os":{"type":"linux","family":"redhat","platform":"amzn","name":"Amazon Linux AMI","version":"2018.03","major":2018,"minor":3,"patch":0},"timezone":"UTC","timezone_offset_sec":0,"id":"ef32a8ed995e86063ec99c115b0b9d78"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.681Z","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null},"cwd":"/home/thapos/tomcat/tomcat8/logs","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":10266,"ppid":8177,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2022-05-23T12:58:11.400Z"},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.681Z","log.origin":{"file.name":"instance/beat.go","file.line":325},"message":"Setup Beat: filebeat; Version: 8.2.0","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.682Z","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":105},"message":"elasticsearch url: http://34.205.74.243:9200","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.682Z","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: mail.thapos.net","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.682Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.683Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":142},"message":"Starting metrics logging every 30s","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.683Z","log.origin":{"file.name":"instance/beat.go","file.line":505},"message":"filebeat start running.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.683Z","log.origin":{"file.name":"memlog/store.go","file.line":134},"message":"Finished loading transaction log file for '/var/lib/filebeat/filebeat'. Active transaction id=20","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.684Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":109},"message":"States Loaded from registrar: 1","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.684Z","log.logger":"crawler","log.origin":{"file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 0","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-05-23T12:58:11.684Z","log.origin":{"file.name":"cfgfile/reload.go","file.line":273},"message":"Error loading config from file '/etc/filebeat/modules.d/tomcat.yml', error invalid config: yaml: line 13: did not find expected key","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.684Z","log.origin":{"file.name":"beater/crawler.go","file.line":155},"message":"Stopping Crawler","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.684Z","log.origin":{"file.name":"beater/crawler.go","file.line":165},"message":"Stopping 0 inputs","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.684Z","log.origin":{"file.name":"beater/crawler.go","file.line":185},"message":"Crawler stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.684Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":132},"message":"Stopping Registrar","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.685Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":166},"message":"Ending Registrar","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.685Z","log.logger":"registrar","log.origin":{"file.name":"registrar/registrar.go","file.line":137},"message":"Registrar stopped","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.688Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":192},"message":"Total metrics","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":20}},"total":{"ticks":130,"time":{"ms":130},"value":0},"user":{"ticks":110,"time":{"ms":110}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":16},"info":{"ephemeral_id":"16e301f9-8c16-425d-9554-1dc069e85ae5","uptime":{"ms":91},"version":"8.2.0"},"memstats":{"gc_next":22180720,"memory_alloc":13348432,"memory_sys":32850952,"memory_total":52250360,"rss":127102976},"runtime":{"goroutines":36}},"filebeat":{"events":{"active":0,"added":0,"done":0},"harvester":{"closed":0,"open_files":0,"running":0,"skipped":0,"started":0},"input":{"log":{"files":{"renamed":0,"truncated":0}},"netflow":{"flows":0,"packets":{"dropped":0,"received":0}}}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":0,"scans":0},"output":{"events":{"acked":0,"active":0,"batches":0,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":0},"read":{"bytes":0,"errors":0},"type":"elasticsearch","write":{"bytes":0,"errors":0}},"pipeline":{"clients":0,"events":{"active":0,"dropped":0,"failed":0,"filtered":0,"published":0,"retry":0,"total":0},"queue":{"acked":0,"max_events":4096}}},"registrar":{"states":{"cleanup":0,"current":0,"update":0},"writes":{"fail":0,"success":0,"total":0}},"system":{"cpu":{"cores":1},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}},"ecs.version":"1.6.0"}} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.689Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":193},"message":"Uptime: 91.804282ms","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.689Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":160},"message":"Stopping metrics logging.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"info","@timestamp":"2022-05-23T12:58:11.689Z","log.origin":{"file.name":"instance/beat.go","file.line":510},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"} {"log.level":"error","@timestamp":"2022-05-23T12:58:11.689Z","log.origin":{"file.name":"instance/beat.go","file.line":1038},"message":"Exiting: Failed to start crawler: creating module reloader failed: loading configs: 1 error: invalid config: yaml: line 13: did not find expected key","service.name":"filebeat","ecs.version":"1.6.0"} Exiting: Failed to start crawler: creating module reloader failed: loading configs: 1 error: invalid config: yaml: line 13: did not find expected key ``` > cat /etc/filebeat/modules.d/tomcat.yml ```auto # Module: tomcat # Docs: https://www.elastic.co/guide/en/beats/filebeat/8.2/filebeat-module-tomcat.html - module: tomcat log: enabled: true # Set which input to use between udp (default), tcp or file. # var.input: udp # var.syslog_host: localhost # var.syslog_port: 9501 # Set paths for the log files when file input is used. var.paths: - /home/thapos/tomcat/tomcat8/logs/catalina.out # Toggle output of non-ECS fields (default true). # var.rsa_fields: true # Set custom timezone offset. # "local" (default) for system timezone. # "+02:00" for GMT+02:00 # var.tz_offset: local ``` ---
**Author:** ![ibra\_013](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibra_013/32/104827_2.png) [@ibra\_013](https://discuss.elastic.co/u/ibra_013)\ **Post date:** [May 23, 2022, 2:08pm UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/17 "2022-05-23T14:08:58Z")
hi @Rajesh119 you have an issue on the filebeat yml file on line 13 ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 23, 2022, 2:13pm UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/18 "2022-05-23T14:13:11Z")
Hi @ibra_013 This is my filebeat.yml file. i didn't get why the error coming. that 13th line is filebeat inputs. ```auto ###################### Filebeat Configuration Example ######################### # This file is an example configuration file highlighting only the most common # options. The filebeat.reference.yml file from the same directory contains all the # supported options with more comments. You can use it as a reference. # # You can find the full configuration reference here: # https://www.elastic.co/guide/en/beats/filebeat/index.html # For more available modules and options, please see the filebeat.reference.yml sample # configuration file. # ============================== Filebeat inputs =============================== #filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. # filestream is an input for collecting log messages from files. #- type: log # Unique ID among all inputs, an ID is required. #id: my-filestream-id # Change to true to enable this input configuration. #enabled: true # Paths that should be crawled and fetched. Glob based paths. #paths: #- /home/thapos/tomcat/tomcat8/logs/catalina.out #- c:\programdata\elasticsearch\logs\* # Exclude lines. A list of regular expressions to match. It drops the lines that are # matching any regular expression from the list. #exclude_lines: ['^DBG'] # Include lines. A list of regular expressions to match. It exports the lines that are # matching any regular expression from the list. #include_lines: ['^ERR', '^WARN'] # Exclude files. A list of regular expressions to match. Filebeat drops the files that # are matching any regular expression from the list. By default, no files are dropped. #prospector.scanner.exclude_files: ['.gz$'] # Optional additional fields. These fields can be freely picked # to add additional information to the crawled log files for filtering #fields: # level: debug # review: 1 # ============================== Filebeat modules ============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/*.yml # Set to true to enable config reloading reload.enabled: false # Period on which files under path should be checked for changes #reload.period: 10s # ======================= Elasticsearch template setting ======================= setup.template.settings: index.number_of_shards: 1 #index.codec: best_compression #_source.enabled: false # ================================== General =================================== # The name of the shipper that publishes the network data. It can be used to group # all the transactions sent by a single shipper in the web interface. #name: filebeat.registry.path: ${path.data}/. # The tags of the shipper are included in their own field with each # transaction published. #tags: ["service-X", "web-tier"] #setup.ilm.overwrite: true # Optional fields that you can specify to add additional information to the # output. #fields: # env: staging # ================================= Dashboards ================================= # These settings control loading the sample dashboards to the Kibana index. Loading # the dashboards is disabled by default and can be enabled either by setting the # options here or by using the `setup` command. #setup.dashboards.enabled: false # The URL from where to download the dashboards archive. By default this URL # has a value which is computed based on the Beat name and version. For released # versions, this URL points to the dashboard archive on the artifacts.elastic.co # website. #setup.dashboards.url: # =================================== Kibana =================================== # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API. # This requires a Kibana endpoint configuration. setup.kibana: host: "34.205.74.243:5601" # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 #host: "localhost:5601" # Kibana Space ID # ID of the Kibana Space into which the dashboards should be loaded. By default, # the Default Space will be used. #space.id: # =============================== Elastic Cloud ================================ # These settings simplify using Filebeat with the Elastic Cloud (https://cloud.elastic.co/). # The cloud.id setting overwrites the `output.elasticsearch.hosts` and # `setup.kibana.host` options. # You can find the `cloud.id` in the Elastic Cloud web UI. #cloud.id: # The cloud.auth setting overwrites the `output.elasticsearch.username` and # `output.elasticsearch.password` settings. The format is `:`. #cloud.auth: # ================================== Outputs =================================== # Configure what output to use when sending the data collected by the beat. # ---------------------------- Elasticsearch Output ---------------------------- output.elasticsearch: # Array of hosts to connect to. hosts: ["34.205.74.243:9200"] #index: "elastic" # Protocol - either `http` (default) or `https`. #protocol: "https" # Authentication credentials - either API key or username/password. #api_key: "id:api_key" #username: "elastic" #password: "changeme" # ------------------------------ Logstash Output ------------------------------- #output.logstash: # The Logstash hosts #hosts: ["localhost:5044"] # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] # Certificate for SSL client authentication #ssl.certificate: "/etc/pki/client/cert.pem" # Client Certificate Key #ssl.key: "/etc/pki/client/cert.key" # ================================= Processors ================================= processors: - add_host_metadata: when.not.contains.tags: forwarded - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ # ================================== Logging =================================== # Sets log level. The default log level is info. # Available log levels are: error, warning, info, debug #logging.level: debug logging.level: info logging.to_files: true logging.files: path: /var/log/filebeat name: filebeat keepfiles: 7 permissions: 0640 # At debug level, you can selectively enable logging only for some components. # To enable all selectors use ["*"]. Examples of other selectors are "beat", # "publisher", "service". #logging.selectors: ["*"] # ============================= X-Pack Monitoring ============================== # Filebeat can export internal metrics to a central Elasticsearch monitoring # cluster. This requires xpack monitoring to be enabled in Elasticsearch. The # reporting is disabled by default. # Set to true to enable the monitoring reporter. #monitoring.enabled: false # Sets the UUID of the Elasticsearch cluster under which monitoring data for this # Filebeat instance will appear in the Stack Monitoring UI. If output.elasticsearch # is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch. #monitoring.cluster_uuid: # Uncomment to send the metrics to Elasticsearch. Most settings from the # Elasticsearch output are accepted here as well. # Note that the settings should point to your Elasticsearch *monitoring* cluster. # Any setting that is not set is automatically inherited from the Elasticsearch # output configuration, so if you have the Elasticsearch output configured such # that it is pointing to your Elasticsearch monitoring cluster, you can simply # uncomment the following line. #monitoring.elasticsearch: # ============================== Instrumentation =============================== # Instrumentation support for the filebeat. #instrumentation: # Set to true to enable instrumentation of filebeat. #enabled: false # Environment in which filebeat is running on (eg: staging, production, etc.) #environment: "" # APM Server hosts to report instrumentation results to. #hosts: # - http://localhost:8200 # API Key for the APM Server(s). # If api_key is set then secret_token will be ignored. #api_key: # Secret token for the APM Server(s). #secret_token: # ================================= Migration ================================== # This allows to enable 6.7 migration aliases #migration.6_to_7.enabled: true ``` ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 24, 2022, 3:43am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/19 "2022-05-24T03:43:38Z")
Hi @ibra_013 Please solve this issue i faced. ---
**Author:** ![Rajesh119](https://avatars.discourse-cdn.com/v4/letter/r/77aa72/32.png) [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Post date:** [May 24, 2022, 5:21am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253/20 "2022-05-24T05:21:26Z")
Please anyone help me regarding this issue. Iam new to this elk tool. Iam stucked here from last few days. [Next page](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253.md?page=2)