# Multiple config file logstash pipelines

**URL:** <https://discuss.elastic.co/t/multiple-config-file-logstash-pipelines/129091>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 23, 2018, 11:45am UTC](https://discuss.elastic.co/t/multiple-config-file-logstash-pipelines/129091 "2018-04-23T11:45:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kingmario](https://avatars.discourse-cdn.com/v4/letter/k/6bbea6/32.png) [@kingmario](https://discuss.elastic.co/u/kingmario)\
**Post date:** [April 23, 2018, 11:45am UTC](https://discuss.elastic.co/t/multiple-config-file-logstash-pipelines/129091/1 "2018-04-23T11:45:23Z")

</div>

Hey,

I am setting up ELK with filebeats. I created some pipelines (apache, ssh..) in logstash. In my webserver, I setting up filebeat.

I have multiple config file that I moved in a created directory in "/etc/filebeat/conf.d/\*" :

ssh.yml :

```
filebeat.prospectors:                                                  
                                                                       
- type: log                                                                  
  paths:                                                                           
    - "/var/log/auth.log"
  fields:
    log_type: ssh
    pipeline: ssh
  enabled: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
    hosts: ["192.168.10.1:5443"]
    ssl.certificate_authorities: ["/etc/ssl/logstash-forwarder.crt"]

```

apache.yml :

```
filebeat.prospectors:                                                  
                                                                       
- type: log                                                                  
  paths:                                                                           
    - "/home/user/http/logs/access.log"
  fields:
    log_type: http
    pipeline: apache
  enabled: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
    hosts: ["192.168.10.1:5444"]
    ssl.certificate_authorities: ["/etc/ssl/logstash-forwarder.crt"]

```

How I can say to filebeat : "Go in conf.d directory and launch the config files" ?

I tested this config in filebeat.yml :

```
filebeat.config.prospectors:
  enabled: true
  path: /etc/filebeat/conf.d/*.yml

```

but in the logs, he said to me that "Please define on under the output section".

Thank you !

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [April 24, 2018, 9:06am UTC](https://discuss.elastic.co/t/multiple-config-file-logstash-pipelines/129091/2 "2018-04-24T09:06:38Z")

</div>

You can try one of this options:

- Place your configuration files in `/etc/filebeat/modules.d/`
- Modify module loader to also load files from `/etc/filebeat/conf.d/`:

```auto
filebeat.config.modules:
  path: ${path.config}/*.d/*.yml

```

---

<div class="post-metadata">

**Author:** ![kingmario](https://avatars.discourse-cdn.com/v4/letter/k/6bbea6/32.png) [@kingmario](https://discuss.elastic.co/u/kingmario)\
**Post date:** [April 24, 2018, 11:50am UTC](https://discuss.elastic.co/t/multiple-config-file-logstash-pipelines/129091/3 "2018-04-24T11:50:12Z")

</div>

Thank you,

but it doesn't work, because in the filebeat configuration, I don't precise an output (they are precised in the conf.d modules).

Unfortunately I understood that filebeat can't send the output to differents instances (on \*:5443, \*:5444...). I think that I need to launch multiples instances of filebeat or need to use a redis or kafka server to send all the logs to my logstash pipelines.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2018, 11:50am UTC](https://discuss.elastic.co/t/multiple-config-file-logstash-pipelines/129091/4 "2018-05-22T11:50:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.