The Network Adapter could not establish the connection

Elastic Stack Logstash
1

I'm new here and also to logstash.
I'm having trouble trying to connect.
I have several branches spread across Brazil, I get data from each one through a VPN connection, and I use a .sh scheduled in CRON to call the scripts, but when the access to one of the branches freezes I can't finish that call and go to the next.
I want you to try 3 times and after the third try it's finished.

Conf:

input { jdbc { jdbc_driver_library => "/etc/logstash/ojdbc8.jar" 
	      jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver" 
              jdbc_connection_string => "jdbc:oracle:thin:@//x.x.x.x:1521/XE" 
	      jdbc_user => "user" 
	      jdbc_password => "password"
	connection_retry_attempts => 3
	connection_retry_attempts_wait_time => 300
    statement_filepath => "/etc/logstash/sql/franquia/xtransacao_franquia_toff.sql"
  }
}
output { elasticsearch { hosts => ["http://localhost:9200"] index => "index"
    user => "elastic" password => "password"
  }
}

Result01:

[2022-03-16T00:04:20,694][ERROR][logstash.inputs.jdbc     ][main][e96d8a80a28becfb9fc7e8f415363a3a2cd75d922e53fadbc0f6c034deb089bc] 
java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection
	at oracle.jdbc.driver.T4CConnection.logon(oracle/jdbc/driver/T4CConnection.java:805) ~[ojdbc8.jar:18.15.0.0.0]
	at oracle.jdbc.driver.PhysicalConnection.connect(oracle/jdbc/driver/PhysicalConnection.java:782) ~[ojdbc8.jar:18.15.0.0.0]
	at oracle.jdbc.driver.T4CDriverExtension.getConnection(oracle/jdbc/driver/T4CDriverExtension.java:39) ~[ojdbc8.jar:18.15.0.0.0]
	at oracle.jdbc.driver.OracleDriver.connect(oracle/jdbc/driver/OracleDriver.java:704) ~[ojdbc8.jar:18.15.0.0.0]
...
Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection
...
Caused by: java.io.IOException: Connection timed out, socket connect lapse 31582 ms.
...
Caused by: java.net.ConnectException: Connection timed out

Enters an infinite loop causing the queue to stop.

2

I already have several extractions running, my problem is just ending the script when I can't access the destination:
If the first one doesn't finish it doesn't go to the second.
Here's an example .sh:

#!/bin/bash

DATAHORA1=$(datahora)
curl -X DELETE -u elastic:xxxxxx "localhost:9200/cs_rz_pending" > /etc/logstash/log_execucao/cs_rz_pending/cur_DEL_cs_rz_pending_"$DATAHORA1".log
/usr/share/logstash/bin/logstash -f /etc/logstash/config/sap/cs_rz_pending.conf --path.data /home/lbelk/temp1 --path.settings /etc/logstash > /etc/logstash/log_execucao/cs_rz_pending/lgtsh_cs_rz_pending_"$DATAHORA1".log

DATAHORA2=$(datahora)
curl -X DELETE -u elastic:xxxxxx "localhost:9200/cs_dif_101j1" > /etc/logstash/log_execucao/cs_dif_101j1/cur_DEL_cs_dif_101j1_"$DATAHORA2".log
/usr/share/logstash/bin/logstash -f /etc/logstash/config/sap/cs_dif_101j1.conf --path.data /home/lbelk/temp1 --path.settings /etc/logstash > /etc/logstash/log_execucao/cs_dif_101j1/lgtsh_cs_dif_101j1_"$DATAHORA2".log

DATAHORA3=$(datahora)
curl -X DELETE -u elastic:xxxxxx "localhost:9200/cs_dif_z101" > /etc/logstash/log_execucao/cs_dif_z101/cur_DEL_cs_dif_z101_"$DATAHORA3".log
/usr/share/logstash/bin/logstash -f /etc/logstash/config/sap/cs_dif_z101.conf --path.data /home/lbelk/temp1 --path.settings /etc/logstash > /etc/logstash/log_execucao/cs_dif_z101/lgtsh_cs_dif_z101_"$DATAHORA3".log
3

SQLRecoverableException is a sub-class of java.sql.SQLException, so I would expect the code to be going through this path. Are you seeing the "Unable to connect to database" messages?

4

Yes, in the log there is also this message:

[2022-03-16T00:04:20,706][ERROR][logstash.inputs.jdbc ][main][e96d8a80a28becfb9fc7e8f415363a3a2cd75d922e53fadbc0f6c034deb089bc] **Unable to connect to database**. Tried 1 times {:message=>"Java::JavaSql::SQLRecoverableException: IO Error: The Network Adapter could not establish the connection", :exception=>Sequel::DatabaseConnectionError, :cause=>java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection, :backtrace=>["oracle.jdbc.driver.T4CConnection.logon(oracle/jdbc/driver/T4CConnection.java:805)", "oracle.jdbc.driver.PhysicalConnection.connect(oracle/jdbc/driver/PhysicalConnection.java:782)", "oracle.jdbc.driver.T4CDriverExtension.getConnection(oracle/jdbc/driver/T4CDriverExtension.java:39)", "oracle.jdbc.driver.OracleDriver.connect(oracle/jdbc/driver/OracleDriver.java:704)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)", "jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)", ...

5

OK, so the retry logic is working as expected. You would like logstash to exit if it fails to start the pipeline. Instead I suspect it will endlessly try to restart the pipeline. There is no way to turn that off.

Instead of using a jdbc input it might be possible to fetch the data to a file using an SQL tool, then cat the file into logstash with a stdin input. That will shutdown the instance after it reaches end-of-file.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.