# Cluster and scheduled pipelines

**URL:** https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897
**Category:** Logstash
**Created:** [July 27, 2018, 7:43am UTC](https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897 "2018-07-27T07:43:04Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Jasper\_Nygaard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasper_nygaard/32/4238_2.png) [@Jasper\_Nygaard](https://discuss.elastic.co/u/Jasper_Nygaard)
#### Post date: [July 27, 2018, 7:43am UTC](https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897/1 "2018-07-27T07:43:04Z")

</div>

Setting up a Logstash cluster to run multiple pipelines. It will be running a number of scheduled JDBC pipelines, besides Beats. But I do not want to have the JDBC pipelines running in parallel, resulting in multiple records. One solution is to only deploy the scheduled pipelines to a single Logstash node, however complicating our deployment pipeline a bit. Is there any way to scope a pipeline to run on a single node, by a host name, tag or some other meta data?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 27, 2018, 4:16pm UTC](https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897/2 "2018-07-27T16:16:01Z")

</div>

There's no great way of doing this. You should be able to wrap all configuration files in the pipeline in conditionals like

```
if "${HOSTNAME}" == "somehost.example.com" { ... }

```

assuming HOSTNAME is an environment variable that you define somewhere. So all your Logstash instances would technically run the same pipelines but on all hosts except one it wouldn't actually do anything.

---

<div class="post-metadata">

### Author: ![Jasper\_Nygaard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasper_nygaard/32/4238_2.png) [@Jasper\_Nygaard](https://discuss.elastic.co/u/Jasper_Nygaard)
#### Post date: [July 30, 2018, 8:10am UTC](https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897/3 "2018-07-30T08:10:20Z")

</div>

Hi Magnus,

Thanks for the feedback. Do I have to wrap the configuration in both input and output or can use conditions in e.g. pipelines.yml?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 30, 2018, 8:26pm UTC](https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897/4 "2018-07-30T20:26:48Z")

</div>

Conditions aren't supported in pipelines.yml.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 27, 2018, 8:32pm UTC](https://discuss.elastic.co/t/cluster-and-scheduled-pipelines/141897/5 "2018-08-27T20:32:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
