Logstash Jdbc plugin - How to get reason for remote database connectivity issues populated in a variable?


(Amit Saxena) #1

Hi all,

I am using ELK stack 6.2.4 . For one of our requirements, I am using Logstash Jdbc plugin to fetch the records from the database and then send it to Elasticsearch.

While I am not facing any major issues with the plugin, however I have noticed that if Logstash is unable to connect to any remote database instance ( like username - password incorrect or database service not running etc ), it only logs the information in the logfile under "logs" directory but does not populate the error message / reason in any variable etc so that it can be used for reference purposes. As an example, I would like to have a variable like %{error} automatically populated if for any reason, Logstash is unable to connect to remote database instance.

Please let me know if this requirement makes sense and how to achieve it in Logstash.

For reference, I am pasting my sample Logstash configuration below.

input {
    jdbc {
        jdbc_driver_library => "/Data/elk/ojdbc8-full/ojdbc8.jar"
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        jdbc_connection_string => "jdbc:oracle:thin:@//12.34.56.78:2222/DB1"
        jdbc_default_timezone => "Etc/UTC"
        jdbc_user => "#####"
        jdbc_password => "#####"
        statement_filepath => "fetch_db_records.sql"
        schedule => "*/5 * * * *"
    }
}

output {
    elasticsearch {
        hosts => ["10.10.10.10:9200", "20.20.20.20:9200", "30.30.30.30:9200"]
        index => "index1"
        user => "#####"
        password => "#####"
    }
}

Regards,
Amit Saxena