# Odd behavior when logstash finished pipeline

**URL:** https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612
**Category:** Logstash
**Created:** [January 8, 2018, 8:34pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612 "2018-01-08T20:34:53Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)
#### Post date: [January 8, 2018, 8:34pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612/1 "2018-01-08T20:34:53Z")

</div>

I have two pipelines defined in `pipelines.yml`: one for my main pipeline which imports data from jdbc input, and another for the dead\_letter\_queue input.

When I run Logstash when it gets to the end of the database, it seems to stop on the last database query and doesn't import anything else. I thought Logstash was supposed to end the pipeline and restart after it was done or something. Is that not the normal behavior?

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [January 11, 2018, 12:00pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612/2 "2018-01-11T12:00:42Z")

</div>

Logstash wants to run forever processing streams of events.  
Some inputs set up a server like process that waits (forever) for clients to connect.  
Some inputs read from local or networked (S3) folders in a loop looking for new files to read or more content in found files (tail).  
Some inputs query message brokers in a loop fetching data from topics.  
Some inputs use cron like schedules to periodically read from a source.  
Only one input `generator` will run for a certain time and then shutdown Logstash.

Due to continual streaming, the concept of reaching the end of a pipeline only applies when the `generator` input is used.

---

<div class="post-metadata">

### Author: ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)
#### Post date: [January 11, 2018, 2:10pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612/3 "2018-01-11T14:10:28Z")

</div>

So for JDBC input is it best practice to use the `schedule` parameter and not just let it run?

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [January 11, 2018, 2:32pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612/4 "2018-01-11T14:32:04Z")

</div>

In most cases, yes.

However if you want to use the elasticsearch filter to do enhancement on another pipeline and you choose to load the records from a static reference DB table into ES then you would only run this "loader" pipeline once.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 8, 2018, 2:32pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612/5 "2018-02-08T14:32:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
