# Pipelines.yml still ignored

**URL:** <https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697>\
**Category:** Logstash\
**Created:** [January 30, 2018, 7:44pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697 "2018-01-30T19:44:28Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jamest23](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@jamest23](https://discuss.elastic.co/u/jamest23)\
**Post date:** [January 30, 2018, 7:44pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697/1 "2018-01-30T19:44:28Z")

</div>

Greetings,  
I have populated the pipelines.yml file but it is not getting picked up.  
The goal being to separate the 2 pipelines i have running.  
When i put them both in the /etc/logstash/conf.d they collide and logstash blends the fields from each pipeline, so i want to separate them.  
But when i start up logstash as a service, as the log file shows, it just loops endlessly trying to load the apache and netflow modules. Any tips or pointers appreciated.  
thx,james

Background Info

Section1 = Versions  
Section2 = Contents of /etc/logstash/pipelines.yml  
Section3 = Contents of /etc/logstash/logstash.yml  
Section4 = Contents of /etc/systemd/system/logstash.service  
Section5 = Contents /var/log/logstash/logstash-plain.log ( loops endlessly)

section 1  
logstash.noarch 1:6.1.2-1 @elasticsearch-6.x  
elasticsearch.noarch 6.1.2-1 @elasticsearch-6.x  
kibana.x86\_64 6.1.2-1 @elasticsearch-6.x

section 2  
pipelines.yml

/etc/logstash# cat pipelines.yml  
[pipeline.id](http://pipeline.id): flowlogs  
path.config: "/etc/logstash/pipelines.d/flowlogs/flowlogs.conf"  
[pipeline.id](http://pipeline.id): guestoslogs  
path.config: "/etc/logstash/pipelines.d/guestos/guestoslogs.conf"

section 3  
logstash.yml  
Path config commented out  
62 # Where to fetch the pipeline configuration for the main pipeline  
63 #  
64 #path.config: /etc/logstash/conf.d/\*.conf  
65 #

section 4  
/etc/systemd/system/logstash.service  
[root@zsmsazure logstash]# cat /etc/systemd/system/logstash.service  
[Unit]  
Description=logstash

[Service]  
Type=simple  
User=logstash  
Group=logstash

EnvironmentFile=-/etc/default/logstash  
EnvironmentFile=-/etc/sysconfig/logstash  
#ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"  
ExecStart=/usr/share/logstash/bin/logstash  
Restart=always  
WorkingDirectory=/  
Nice=19  
LimitNOFILE=16384

[Install]  
WantedBy=multi-user.target  
[root@zsmsazure logstash]#

section 5  
[2018-01-30T19:32:35,622][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"/usr/share/logstash/modules/fb\_apache/configuration"}  
[2018-01-30T19:32:35,625][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"/usr/share/logstash/modules/netflow/configuration"}  
[2018-01-30T19:32:52,112][INFO][logstash.modules.scaffold] Initializing module

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [January 31, 2018, 5:08pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697/2 "2018-01-31T17:08:08Z")

</div>

It looks like the format of your pipelines.yml is wrong. Indentation is important with yaml. Here is an example that works...

```auto
- pipeline.id: process-ipfix
  queue.type: memory
  path.config: "/etc/logstash/pipelines/process/ipfix"

```

Notice the `-` before `pipeline.id` and two spaces before the lines beneath.

I have had a simple indentation error cause me problems as well. Hopefully this helps.

---

<div class="post-metadata">

**Author:** ![jamest23](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@jamest23](https://discuss.elastic.co/u/jamest23)\
**Post date:** [February 1, 2018, 1:03pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697/3 "2018-02-01T13:03:22Z")

</div>

Thanks! will try this today.

---

<div class="post-metadata">

**Author:** ![jamest23](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@jamest23](https://discuss.elastic.co/u/jamest23)\
**Post date:** [February 1, 2018, 4:46pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697/4 "2018-02-01T16:46:09Z")

</div>

Thanks Robert,  
Much Appreciated.  
That did the trick! My formatting was bad.  
Now all 3 of my pipelines are running and there are no collisions.

cat pipelines.yml

- [pipeline.id](http://pipeline.id): eventhubs  
queue.type: memory  
path.config: "/etc/logstash/pipelines.d/eventhubs/geteventhub.conf"
- [pipeline.id](http://pipeline.id): flowlogs  
queue.type: memory  
path.config: "/etc/logstash/pipelines.d/flowlogs/flowlogs.conf"
- [pipeline.id](http://pipeline.id): guestoslogs  
queue.type: memory  
path.config: "/etc/logstash/pipelines.d/oslogs/guestoslogs.conf"

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 1, 2018, 4:46pm UTC](https://discuss.elastic.co/t/pipelines-yml-still-ignored/117697/5 "2018-03-01T16:46:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
