# Multi-File configuration **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 3, 2018, 8:26am UTC](https://discuss.elastic.co/t/multi-file-configuration/142868 "2018-08-03T08:26:59Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 3, 2018, 8:27am UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/1 "2018-08-03T08:27:00Z")
My filebeat.yml configuration is as below : ``` output.logstash: hosts: ["localhost:5044"] filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: false setup.template.settings: index.number_of_shards: 0 filebeat.prospectors: - input_type: log enabled: true paths: - data\Logs_20180720\AB.log fields: {log_type: ab} - input_type: log enabled: true paths: - data\Logs_20180720\BC.log fields: {log_type: bc} ``` My complete logstash configuration is looking as below : ``` input { beats { port=>5044 } } filter { if [message] =~ "\tat" { grok { match => ["message", "^(\tat)"] add_tag => ["stacktrace"] } } if ([fields][log_type] == "ab") { mutate { replace => { "[type]" => "ab" } } } else if ([fields][log_type] == "bc") { mutate { replace => { "[type]" => "bc" } } } #Grokking Spring Boot's default log format grok { match => [ "message",'^%{TIMESTAMP_ISO8601:timestamp} (\[%{DATA:thread}\] )?%{LOGLEVEL:level}%{SPACE}%{JAVACLASS:class}\.%{DATA:method} - %{GREEDYDATA:logMessage}$' ] } } output { if "_grokparsefailure" in [tags] { stdout { codec => rubydebug {metadata => true }} } if "log" in [tags]{ if "ERROR" in [level]{ elasticsearch { hosts => ["localhost:9200"] } } else if "WARN" in [level]{ elasticsearch { hosts => ["localhost:9200"] } } else if "INFO" in [level]{ elasticsearch { hosts => ["localhost:9200"] } } else if "FATAL" in [level]{ elasticsearch { hosts => ["localhost:9200"] } } } elasticsearch { hosts => ["localhost:9200"] index => "raghu-%{type}-%{+YYYY.MM}" } } ``` There is no error log on console. But nothing is happening the logs are generating but filebeat is silent. kindly suggest. Below the log on console of filebeat: ``` 2018-08-03T15:11:15.183+0530 INFO instance/beat.go:492 Home path: [C:\tools\filebeat] Config path: [C:\tools\filebeat] Data path: [C:\tools\filebeat\data] Logs path: [C:\tools\filebeat\logs] 2018-08-03T15:11:15.184+0530 INFO instance/beat.go:499 Beat UUID: 2e8df794-8fd7-4fd1-a380-67d2ede115c7 2018-08-03T15:11:15.185+0530 INFO [beat] instance/beat.go:716 Beat info {"system_info": {"beat": {"path": {"config": "C:\\tools\\filebeat", "data": "C:\\tools\\filebeat\\data", "home": "C:\\tools\\filebeat", "logs": "C:\\tools\\filebeat\\logs"}, "type": "filebeat", "uuid": "2e8df794-8fd7-4fd1-a380-67d2ede115c7"}}} 2018-08-03T15:11:15.185+0530 INFO [beat] instance/beat.go:725 Build info {"system_info": {"build": {"commit": "ed42bb85e72ae58cc09748dc1825159713e0ffd4", "libbeat": "6.3.1", "time": "2018-06-29T21:09:04.000Z", "version": "6.3.1"}}} 2018-08-03T15:11:15.185+0530 INFO [beat] instance/beat.go:728 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.9.4"}}} 2018-08-03T15:11:15.218+0530 INFO instance/beat.go:225 Setup Beat: filebeat; Version: 6.3.1 2018-08-03T15:11:15.219+0530 INFO pipeline/module.go:81 Beat name: RILITS-HWLTP132 2018-08-03T15:11:15.219+0530 WARN [cfgwarn] beater/filebeat.go:61 DEPRECATED: prospectors are deprecated, Use `inputs` instead. Will be removed in version: 7.0.0 2018-08-03T15:11:15.223+0530 INFO instance/beat.go:315 filebeat start running. 2018-08-03T15:11:15.224+0530 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s 2018-08-03T15:11:15.225+0530 INFO registrar/registrar.go:116 Loading registrar data from C:\tools\filebeat\data\registry 2018-08-03T15:11:15.226+0530 INFO registrar/registrar.go:127 States Loaded from registrar: 3 2018-08-03T15:11:15.227+0530 WARN beater/filebeat.go:354 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning. 2018-08-03T15:11:15.227+0530 INFO crawler/crawler.go:48 Loading Inputs: 2 2018-08-03T15:11:15.227+0530 WARN [cfgwarn] input/config.go:25 DEPRECATED: input_type input config is deprecated. Use type instead. Will be removed in version: 6.0.0 2018-08-03T15:11:15.229+0530 INFO log/input.go:113 Configured paths: [C:\tools\filebeat\data\Logs_20180720\eis_log_file.log] 2018-08-03T15:11:15.231+0530 INFO input/input.go:88 Starting input of type: log; ID: 10642447689570774022 2018-08-03T15:11:15.231+0530 WARN [cfgwarn] input/config.go:25 DEPRECATED: input_type input config is deprecated. Use type instead. Will be removed in version: 6.0.0 2018-08-03T15:11:15.232+0530 INFO log/input.go:113 Configured paths: [C:\tools\filebeat\data\Logs_20180720\SA.log] 2018-08-03T15:11:15.232+0530 INFO input/input.go:88 Starting input of type: log; ID: 15709701256631990576 2018-08-03T15:11:15.233+0530 INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 22018-08-03T15:11:15.233+0530 INFO cfgfile/reload.go:122 Config reloader started 2018-08-03T15:11:15.234+0530 INFO cfgfile/reload.go:214 Loading of config files completed. ``` ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 3, 2018, 8:40am UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/2 "2018-08-03T08:40:59Z")
Could you please format your config using ``? Also, please attach debug logs. ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 3, 2018, 9:25am UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/4 "2018-08-03T09:25:12Z")
Could you also add the `output` part of your config? Also, please attach the output of `./filebeat -e -d "*"`. ---
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 3, 2018, 9:43am UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/5 "2018-08-03T09:43:25Z")
added please check. ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 3, 2018, 12:47pm UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/6 "2018-08-03T12:47:34Z")
So it looks like you have already sent events from these inputs. Filebeat does not reread messages it has encountered previously. If you delete your `data/registry` file, all files will be read again and events will be forwarded. Please, don't delete the file if it's important that there is duplication of logs in the output. ---
**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Post date:** [August 6, 2018, 10:06am UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/7 "2018-08-06T10:06:56Z")
I deleted the registry file, I see only one file is getting logged. Since i wanted to test I gave an output config to a file. How can I give 2 output files in my case where ab.log should log to filebeat-ab.log and the other to filebeat-bc.log. thanks. ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 13, 2018, 4:42pm UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/8 "2018-08-13T16:42:37Z")
You can only write events to a single file. There is no way to separate the input files. So you could try testing it once with the first input file and than the second one. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 10, 2018, 4:42pm UTC](https://discuss.elastic.co/t/multi-file-configuration/142868/9 "2018-09-10T16:42:38Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.