Logstash Multi Pipeline breaking and logstash all pipelines completely stopping when specific Pipelines DB is down or DB details are changed


(Suman Behara) #1

Hi logstash team,

  I have configured logstash Mutlipipeline for jdbc inputs and Pipeline is working fine and perfectly 
 from couple of days. each pipeline contains multiple jdbc inputs.
 later after some days some how one specific pipeline(Ste_DB) db is down for some time or db details are 
 changed. logstash scheduler run on scheduled time and it's failed to connect that particular db i think 
 and it's failed to connect db for specific pipeline(Ste_DB).

  We are getting below error 

[2018-07-19T03:31:00,175][ERROR][logstash.inputs.jdbc ] Unable to connect to database. Tried 1 times {:error_message=>"Java::JavaSql::SQLRecoverableException: IO Error: The Network Adapter could not establish the connection"}
[2018-07-19T03:31:00,176][ERROR][logstash.inputs.jdbc ] Unable to connect to database. Tried 1 times {:error_message=>"Java::JavaSql::SQLRecoverableException: IO Error: The Network Adapter could not establish the connection"}
[2018-07-19T03:31:00,562][FATAL][logstash.runner ] An unexpected error occurred!

and i observed that all the pipelines are stopped and logstash process also stopped for all the 
pipelines. 

Q) My question here is If one specific pipelines db is down or db details are changed, 
then it should break that particular pipeline only not for all the pipelines and not to stop logstash 
process . Can you please help me out for 
there is any solution for this or logstash behaves to stop all the pipelines worker thread if one pipeline 
db breaks?

Here is my pipeline.yml configuration file

- pipeline.id: CC_DB
   path.config: "/etc/logstash/dbconfigs/CC_DB/CC_DB.conf"

- pipeline.id: OS_DB
  path.config: "/etc/logstash/dbconfigs/OS_DB/OS_DB.conf"

- pipeline.id: Ste_DB
  path.config: "/etc/logstash/dbconfigs/Ste_DB/Ste_DB.conf"

- pipeline.id: IM_DB
  path.config: "/etc/logstash/dbconfigs/IM_DB/IM_DB.conf"

- pipeline.id: Sit_DB
  path.config: "/etc/logstash/dbconfigs/Sit_DB/Sit_DB.conf"

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.