# Can Logstash jdbc inputs be processed in certain order and how to control that 1 input process has ended?

**URL:** <https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856>\
**Category:** Logstash\
**Created:** [December 22, 2016, 11:23pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856 "2016-12-22T23:23:22Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![iti](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@iti](https://discuss.elastic.co/u/iti)\
**Post date:** [December 22, 2016, 11:23pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/1 "2016-12-22T23:23:22Z")

</div>

I have the following config:  
input {  
jdbc { ...  
staement =\> "select \* from tab\_1st"  
type =\> "1st"}  
jdbc { ...  
staement =\> "select \* from tab\_2nd"  
type =\> "2nd"}  
jdbc { ...  
staement =\> "select \* from tab\_3rd"  
type =\> "3rd"}  
}  
filter { ...}  
output {  
#update certain table regarding processing status  
if "1st" == [type] {  
jdbc {...  
statement =\> ["update log set status='P' where log\_id=(select es\_etl\_log\_id from es\_etl\_log where status='N')"]  
}  
if "3rd" == [type] {  
jdbc {...  
statement =\> ["update log set status='C' where log\_id=(select es\_etl\_log\_id from es\_etl\_log where status='P')"]  
}  
}

I am using logstash 5.1, ES 5.1  
logstash-input-jdbc (4.1.3)  
logstash-output-jdbc (5.1.0)

I seems to me the logstash does not process the input in th 1st, 2nd, 3rd order. It could be 3rd....

BTW, I have pipeline.workers set to 1.

Can anyone advise?

Thanks

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 28, 2016, 9:04am UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/2 "2016-12-28T09:04:44Z")

</div>

Try splitting them into files like;

- input1.conf
- input2.conf
- input3.conf

And then see if that helps.

---

<div class="post-metadata">

**Author:** ![iti](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@iti](https://discuss.elastic.co/u/iti)\
**Post date:** [December 28, 2016, 9:26pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/3 "2016-12-28T21:26:31Z")

</div>

Thanks for the reply.  
Splitting the config files does not help. Logstash will combine them when it starts. So, it ends up the same as one file.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 28, 2016, 9:27pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/4 "2016-12-28T21:27:48Z")

</div>

Yes, but they are merged alphanumerically, so if you number them like that then they will be in order.

---

<div class="post-metadata">

**Author:** ![iti](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@iti](https://discuss.elastic.co/u/iti)\
**Post date:** [December 28, 2016, 9:28pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/5 "2016-12-28T21:28:39Z")

</div>

My one file already in correct order. Combing does not change anything. Same order.  
The problem is not the config order. It is how logstash process the input. So far, look like it works this way:  
Get input 1,  
process filter1, get input2  
output1 proc filter2 get input3  
output2 proc filter3  
output3  
See the problem?  
I want them to be processed sequentially, not parallely. Since I specified 1 thread (1 worker)  
I like:  
Get input1  
Proc filter 1  
Output1  
Get input 2  
Proc filter 2  
Output 2  
Get input3  
....

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 28, 2016, 9:41pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/6 "2016-12-28T21:41:18Z")

</div>

Oh, then you can't do that.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 25, 2017, 9:41pm UTC](https://discuss.elastic.co/t/can-logstash-jdbc-inputs-be-processed-in-certain-order-and-how-to-control-that-1-input-process-has-ended/69856/7 "2017-01-25T21:41:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
