# Logstash - do not starts when jdbc filter fails connection **URL:** \ **Category:** Logstash\ **Created:** [March 17, 2021, 3:55pm UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545 "2021-03-17T15:55:59Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![cesar.hernandez.a3se](https://avatars.discourse-cdn.com/v4/letter/c/c77e96/32.png) [@cesar.hernandez.a3se](https://discuss.elastic.co/u/cesar.hernandez.a3se)\ **Post date:** [March 17, 2021, 3:55pm UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/1 "2021-03-17T15:55:59Z")
Hi I've created a logstash input filter: ```auto filter { jdbc_static { loaders => [ { id => "remote-ipsservers" query => "select address, display_name from host_list'" local_table => "host_list" } ] local_db_objects => [ { name => "host_list" index_columns => ["address"] columns => [ ["address", "varchar(255)"], ["display_name", "varchar(255)"] ] } ] local_lookups => [ { id => "local-ipsservers" query => "select display_name as description from host_list WHERE address = :ip" parameters => {ip => "[host][ip]"} target => "probes" } ] # reload database every our loader_schedule => "00 * * * *" jdbc_user => "logstash" jdbc_password => "12345678" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_driver_library => "/extras/mysql-connector-java-5.1.46.jar" jdbc_connection_string => "jdbc:mysql://10.33.33.10:3306/inventory?useSSL=false" } # If found, add name if [probes][0][description] { mutate { # using add_field here to add & rename values to the event root add_field => { nombre_sonda => "%{[probes][0][description]}" } } } # If not, write a default name else { mutate { add_field => { nombre_sonda => "unknown" } } } # we use probes just temporarily and we don't need it here anymore mutate { remove_field => ["probes"] } } ``` I'm receiving netflow packets, and then I do a lookup of the host.ip parameter in netflow to a remote mysql database to query for the name for that ip. The filter works perfect, but if I start logstash and the mysql server is down, or fails authentication, logstash just shut down with the following logs: ```auto [ERROR][logstash.agent] Failed to execute action {:id=>:flows, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil} [INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} [INFO][logstash.runner] Logstash shut down. ``` Is there any logstash parameter to disable shutdown when the jdbc driver fails connection? As I said, that's not a syntax error, it fails only when the remote mysql is unavailable Thanks Cheers ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [March 19, 2021, 1:52am UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/2 "2021-03-19T01:52:34Z")
Hi Cesar, looking to the documentation: > **[Jdbc\_static filter plugin | Logstash Reference \[7.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_static.html#plugins-filters-jdbc_static-loaders)** It looks to me like the local\_lookups wrong. Would you mind to change: ` parameters => { ip => "[host][ip]"}` to ` parameters => { "ip" => "%{[host][ip]}"}` What about the outcome. does it work for you? Let me know how it goes. ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [March 19, 2021, 1:59pm UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/3 "2021-03-19T13:59:44Z")
If forgot to mention you might add a conditional around the JDBC filter to add the notion of, if the host.ip field doesn't exist, don't do the lookup action: ```auto if [host][ip] { .... } ``` this means if field host.ip exists or is not null, do the jdbc lookup action. ---
**Author:** ![cesar.hernandez.a3se](https://avatars.discourse-cdn.com/v4/letter/c/c77e96/32.png) [@cesar.hernandez.a3se](https://discuss.elastic.co/u/cesar.hernandez.a3se)\ **Post date:** [March 22, 2021, 7:25am UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/4 "2021-03-22T07:25:14Z")
Thanks Andre for the suggestions. But that's not the problem I reported: when the mysql server is not up, the logstash startup fails and it just shutdowns Cheers ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [March 23, 2021, 3:09am UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/5 "2021-03-23T03:09:44Z")
Yeah, sorry I was a bit too focused on the errors mentioned. So if you startup logstash and at that time MySQL is not available, I don't think there is much we could do, as JDBC relies on sequel component, > **[jeremyevans/sequel](https://github.com/jeremyevans/sequel)** > > Sequel: The Database Toolkit for Ruby. Contribute to jeremyevans/sequel development by creating an account on GitHub. which itself relies on the Java methods to deal with JDBC connections. [https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html](https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html) So it might be better to rely on HA availability of the MySQL database like Master/Slave or Galera Server. If the MySQL connection itself gets unavailable during logstash is running, this should be keeping running. [https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-usagenotes-troubleshooting.html](https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-usagenotes-troubleshooting.html) Some parameters adding to the jdbc:MySQL URL should help at the end: `&autoReconnect=true&failOverReadOnly=false&maxReconnects=10` and at least I am getting before this `nil` error you mentioned the original exception which I a little bit formatted for you in next comment ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [March 23, 2021, 3:17am UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/6 "2021-03-23T03:17:09Z")
[https://ctxt.io/2/AACgRQmHFw](https://ctxt.io/2/AACgRQmHFw) ---
**Author:** ![Andre\_Letterer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andre_letterer/32/42248_2.png) [@Andre\_Letterer](https://discuss.elastic.co/u/Andre_Letterer)\ **Post date:** [March 23, 2021, 8:59pm UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/7 "2021-03-23T20:59:04Z")
Ok, in the meanwhile I did some additional digging. What you normally do if pipelines cannot load is to enable automatic reloading of your pipeline, which can help your logstash on: > **[Reloading the Config File | Logstash Reference \[7.12\] | Elastic](https://www.elastic.co/guide/en/logstash/current/reloading-config.html)** > **[logstash.yml | Logstash Reference \[7.12\] | Elastic](https://www.elastic.co/guide/en/logstash/current/logstash-settings-file.html)** `config.reload.automatic: true` Additionally you can still rectify the situation by splitting this pipeline to a pipeline to pipeline configuration, but you still need to have the reload enabled. However this particular one if you would isolate the JDBC part and the output into a separate pipeline will help to have the input port of logstash online, even if the JDBC MySQL port would not be available. Once it is available again, it will kick in by the config reload setting: > **[Pipeline-to-Pipeline Communication | Logstash Reference \[7.12\] | Elastic](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html)** If you still have questions, please let me know. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 20, 2021, 8:59pm UTC](https://discuss.elastic.co/t/logstash-do-not-starts-when-jdbc-filter-fails-connection/267545/8 "2021-04-20T20:59:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.