# Trying to use pipelines with filebeat output.elasticsearch

**URL:** https://discuss.elastic.co/t/trying-to-use-pipelines-with-filebeat-output-elasticsearch/156361
**Category:** Beats
**Tags:** filebeat
**Created:** [November 12, 2018, 10:19pm UTC](https://discuss.elastic.co/t/trying-to-use-pipelines-with-filebeat-output-elasticsearch/156361 "2018-11-12T22:19:28Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![srkrishna](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@srkrishna](https://discuss.elastic.co/u/srkrishna)
#### Post date: [November 12, 2018, 10:19pm UTC](https://discuss.elastic.co/t/trying-to-use-pipelines-with-filebeat-output-elasticsearch/156361/1 "2018-11-12T22:19:28Z")

</div>

Hello,

I configured multiple .yml config file for different types of logs.  
Now i need to add conditions to send outputs 1. to elasticsearch and 2.logstash

this is based on a condition with fields.logtype value. After doing a lot of google and the elastic search forums figured out we can use pipelines to acheive this.

the issue here is when I add "pipelines" token in the configuration, it doesn't accept while starting the filebeat instance. below is my configuration file.

#========================= Filebeat global options ============================

filebeat.registry\_file\_permissions: 0600

# By default Ingest pipelines are not updated if a pipeline with the same ID

# already exists. If this option is enabled Filebeat overwrites pipelines

# everytime a new Elasticsearch connection is established.

filebeat.overwrite\_pipelines: true

# How long filebeat waits on shutdown for the publisher to finish.

# Default is 0, not waiting.

#filebeat.shutdown\_timeout: 0

# Enable filebeat inputs config

filebeat.config.inputs:  
enabled: true  
path: config/\*.yml

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------  
output.elasticsearch:

# Array of hosts to connect to.

hosts: ["localhost:9200"]  
pipelines:  
- pipeline: "ErrorLogs"  
when.equals:  
fields.log\_type: "ErrorLog"  
- pipeline: "IPLogs"  
when.equals:  
fields.log\_type: "IPLog"

# Optional protocol and basic auth credentials.

#protocol: "https"  
#username: "elastic"  
#password: "changeme"

#----------------------------- Logstash output --------------------------------  
output.logstash:

# The Logstash hosts

hosts: ["localhost:5044"]  
pipelines:  
- pipeline: "DebugLogs"  
when.equals:  
fields.log\_type: "Debug"

#================================ Logging =====================================

# Sets log level. The default log level is info.

# Available log levels are: error, warning, info, debug

logging.level: debug

# At debug level, you can selectively enable logging only for some components.

# To enable all selectors use ["\*"]. Examples of other selectors are "beat",

# "publish", "service".

#logging.selectors: ["\*"]

#============================== Xpack Monitoring ===============================

# filebeat can export internal metrics to a central Elasticsearch monitoring

# cluster. This requires xpack monitoring to be enabled in Elasticsearch. The

# reporting is disabled by default.

# Set to true to enable the monitoring reporter.

#xpack.monitoring.enabled: false

# Uncomment to send the metrics to Elasticsearch. Most settings from the

# Elasticsearch output are accepted here as well. Any setting that is not set is

# automatically inherited from the Elasticsearch output configuration, so if you

# have the Elasticsearch output configured, you can simply uncomment the

# following line.

#xpack.monitoring.elasticsearch:

Please advise.

Regards,  
Siva

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [November 16, 2018, 9:16am UTC](https://discuss.elastic.co/t/trying-to-use-pipelines-with-filebeat-output-elasticsearch/156361/2 "2018-11-16T09:16:07Z")

</div>

Could you please format your configuration using `</>`?

---

<div class="post-metadata">

### Author: ![srkrishna](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@srkrishna](https://discuss.elastic.co/u/srkrishna)
#### Post date: [November 17, 2018, 3:53am UTC](https://discuss.elastic.co/t/trying-to-use-pipelines-with-filebeat-output-elasticsearch/156361/3 "2018-11-17T03:53:12Z")

</div>

Hi Noémi Ványi,

I tried to manage it by sending all the events to logstash with seperate filebeat yml files.  
I guess it not possible to push the events to multiple output sources from filebeats.

Thank you.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 15, 2018, 4:05am UTC](https://discuss.elastic.co/t/trying-to-use-pipelines-with-filebeat-output-elasticsearch/156361/4 "2018-12-15T04:05:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
