# Multi file send from filebeat to logstash **URL:** \ **Category:** Beats\ **Created:** [December 26, 2016, 6:04pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018 "2016-12-26T18:04:47Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 26, 2016, 6:04pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/1 "2016-12-26T18:04:47Z")
i configured a filebeat.yml to get multi file log and create type fields for each file like this: filebeat.prospectors: - paths: - /var/log/access.log fields: {log\_type: access} - paths: - /var/log/errors.log fields: {log\_type: errors} then in logstash.cfg i want to cfg in fileter like: if ([fields][log\_type]=="errors){ do something }else if ([fields][log\_type]=="access"){ do something } but error occur so i can't start filebeat, logstash worked normal ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 26, 2016, 6:09pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/2 "2016-12-26T18:09:21Z")
> [@Tr\_ng\_Trang](#): > > but error occur so i can't start filebeat What's the error? ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 26, 2016, 6:17pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/3 "2016-12-26T18:17:07Z")
Dec 26 10:14:34 ubuntu systemd[1]: Stopped filebeat. Dec 26 10:14:34 ubuntu systemd[1]: Started filebeat. Dec 26 10:14:34 ubuntu filebeat[6300]: Exiting: error loading config file: yaml: Dec 26 10:14:34 ubuntu systemd[1]: filebeat.service: Main process exited, code=e Dec 26 10:14:34 ubuntu systemd[1]: filebeat.service: Unit entered failed state. Dec 26 10:14:34 ubuntu systemd[1]: filebeat.service: Failed with result 'exit-co Dec 26 10:14:35 ubuntu systemd[1]: filebeat.service: Service hold-off time over, Dec 26 10:14:35 ubuntu systemd[1]: Stopped filebeat. Dec 26 10:14:35 ubuntu systemd[1]: filebeat.service: Start request repeated too Dec 26 10:14:35 ubuntu systemd[1]: Failed to start filebeat. this is error messages ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 26, 2016, 6:17pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/4 "2016-12-26T18:17:35Z")
i configured my filebeat: ```auto #=========================== Filebeat prospectors ============================= filebeat.prospectors: # Each - is a prospector. Most options can be set at the prospector level, so # you can use different prospectors for various configurations. # Below are the prospector specific configurations. # Paths that should be crawled and fetched. Glob based paths. paths: - "/var/log/access.log" fields: {log_type: access} paths: -"/var/log/errors.log" fields: {log_type: errors} ``` ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 26, 2016, 6:19pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/5 "2016-12-26T18:19:09Z")
Run `./filebeat.sh -configtest -e` to test the configuration (if you installed via rpm/deb). What does that output? ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 26, 2016, 6:21pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/6 "2016-12-26T18:21:11Z")
Try this: ```auto filebeat.prospectors: - paths: - "/var/log/access.log" fields: {log_type: access} - paths: - "/var/log/errors.log" fields: {log_type: errors} ``` ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 26, 2016, 6:22pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/7 "2016-12-26T18:22:54Z")
thanks, its working guys, is you just add - on paths to fixed my configured? ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 26, 2016, 6:24pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/8 "2016-12-26T18:24:48Z")
In the configuration, `prospectors` is a YAML list so each individual prospector needs a `-` to begin a new list entry. ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 26, 2016, 6:29pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/9 "2016-12-26T18:29:31Z")
thanks you, i will try to find more about yaml. can i asked you 1 question, when i configure 2 file send by 1 filebeat, i check kibana, but just 1 file log is parse by logstash and show in kibana, do i need to configure other index or use same index is logstash ? sry if the question is so recondite ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 26, 2016, 6:33pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/10 "2016-12-26T18:33:47Z")
i check log of filebeat and notice it just read file apear in first line is: accesslog, and file error log don't see any action with it. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 26, 2016, 6:40pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/11 "2016-12-26T18:40:29Z")
Writing all of the data to the same index should not be a problem. I noticed I had a space missing from the second `paths` given in my example and I corrected it. Please update yours. While you are testing you might need to delete the [registry file](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-global-options.html#_registry_file) to get Filebeat to resend data from those files. You can increase logging [level](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html#_level) to see more details on Filebeat is doing while you debug the problem. ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 27, 2016, 1:13am UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/12 "2016-12-27T01:13:01Z")
best thanks guys. merry christmas :)) ---
**Author:** ![Tr\_ng\_Trang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tr_ng_trang/32/13053_2.png) [@Tr\_ng\_Trang](https://discuss.elastic.co/u/Tr_ng_Trang)\ **Post date:** [December 29, 2016, 7:45am UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/13 "2016-12-29T07:45:46Z")
hey guys, i configure successfull file configure logstash and filebeat on ubuntu but when i copy file configure on centos, filebeat not send log of error.log. Just send log file of access.log. This is debug log filebeat: 2016-12-29T14:25:24+07:00 DBG Start next scan 2016-12-29T14:25:24+07:00 DBG scan path /opt/apache2/logs/error\_log 2016-12-29T14:25:24+07:00 DBG Check file for harvesting: /opt/apache2/logs/error\_log 2016-12-29T14:25:24+07:00 DBG Same file as before found. Fetch the state. 2016-12-29T14:25:24+07:00 DBG Update existing file for harvesting: /opt/apache2/logs/error\_log 2016-12-29T14:25:24+07:00 DBG Not harvesting, file didn't change: /opt/apache2/logs/error\_log 2016-12-29T14:25:24+07:00 DBG End of file reached: /opt/apache2/logs/error\_log; Backoff now. 2016-12-29T14:25:24+07:00 DBG End of file reached: /opt/apache2/logs/access\_log; Backoff now. although i have updated error.log, but filebeat seem think no changed in file error. i have searched more, but can't find extacly the answer for this problem. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 16, 2017, 6:05pm UTC](https://discuss.elastic.co/t/multi-file-send-from-filebeat-to-logstash/70018/14 "2017-01-16T18:05:01Z")
This topic was automatically closed after 21 days. New replies are no longer allowed.