In our setup, there are ~ 30 JDBC configs *.conf (one per client) within that it will have one input section with 4 different jdbc section (one per functionality). Every functionality JDBC sections are scheduled to run every 10 seconds.
DB password is stored and retrieved from logstash keystore to make JDBC connection. As per organization mandate passwords needs to be rotated without any human intervention.
So going forward passwords needs to be retrieved from password vault like “CyberArk” by making a https call and cache the password locally as “CyberArk” can’t handle load to take a hit every 10 secs.
How to make a https call ?
How to cache the retrieved password locally?
Using step 2 password make JDBC call and retrieve value.
That is a completely unreasonable requirement unless CyberArk supports triggers to update the configuration when the password changes. Either they have to support asking for the password for every access, or they have to support letting you know when to invalidate the cached value. Otherwise you will keep using an invalid password when the password changes, which would hopefully result in the account being disabled.
That said, if someone held a gun to my head and told me to implement this then I would configure two heartbeat inputs. One to drive retrieval of the password in a ruby filter (and caching it in an instance or class variable, depending on whether you use the same ruby filter to retrieve the password). The other would run every 10 second to drive ~30 jdbc_streaming filters (not jdbc inputs). Depending on how quick those jdbc queries are you might need one pipeline or thirty to keep things moving.
I think it can be done but it is a terrible match for logstash's functionality.
This seems to be CybeArk Vault, in this case it takes cares of the secrets, getting the secret and updating it in the places it is being used is done by something else and depends on the application/product.
Some products natively support getting secrets from Key Vaults like CyberArk Vault, Hashicorp Vault, Azure Key Vault, AWS Secret Manager etc.
And of course this is not the case for logstash, so a routine like the one you described will need to be built.
Hi Badger,
Any specific reasons to use JDBC streaming? Its all changing data so I don't need to it to cache it, just want to understand the rational behind it. Pardon my ignorance.
Is there a way for JDBC to check if password is wrong and then retrigger the https call and take the latest password and do the login attempt next time?
Can you please give a high level Code I will try to implement it, As I'm new to ruby I'm not sure how to cache it.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.