Logstash 7.17 keystore - get value by variable key

Elastic Stack Logstash
1

Hello, long story short: i must process logs that are encryped with EC keys. The customer requires the keys' passwords to be stored in Logstash keystore where the key is the p12's serial and the value is the password.
The question is, how can i refer to the key in the logstash keystore by a variable value that comes from the processed log. Since the p12's serial can vary from log to log i can't store them in environment variables.
(The logs consist of a header part that contains the - variable - serial for the encryption key and an enrypted body part)
Thank you very much for any help!!

2

The logstash configuration language does not support doing a field substitution before doing an environment substitution. It is one or the other. That said you can repurpose that code in a ruby filter to achieve what you want.

    ruby {
        init => '
            SECRET_STORE = ::LogStash::Util::LazySingleton.new { SecretStoreExt.getIfExists(LogStash::SETTINGS.get_setting("keystore.file").value, LogStash::SETTINGS.get_setting("keystore.classname").value) }
        '
        code => '
            key = event.get("a")
            secret_store = SECRET_STORE.instance
            pass = secret_store.retrieveSecret(SecretStoreExt.getStoreId(key))
            event.set("pass", pass.to_s)
        '
    }

This is not a documented or supported interface, but it works in the current version of logstash. Elastic are free to rewrite the internals of substitution in ways that break that.

Obviously you will need to add error handling, but if I start with

input { generator { count => 1 lines => [ '{ "a": "key.foo" }', '{ "a": "key.bar" }' ] codec => json } }

then that produces

{
"@timestamp" => 2023-09-06T13:53:49.924638742Z,
      "pass" => "changeMe",
         "a" => "key.foo",
  "@version" => "1"
}
{
"@timestamp" => 2023-09-06T13:53:49.925617761Z,
      "pass" => "weakPassword",
         "a" => "key.bar",
  "@version" => "1"
}
3

Hi Badger, thank you very much!! I'm going to try it and will be back soon.

4

Hello Badger, the solution works fine, thank you very much. One more question: how to instantiate the secret store object if it is password protected? Is it just another argument of the constructor?

5

The documentation says to use an environment variable (LOGSTASH_KEYSTORE_PASS). And I get that doing that sucks, but I verified that it works with the above ruby filter.

In the secret store Java code there is a .GetSecureConfig method that takes a string for the password. The ruby filter calls .getIfExists, and that calls the code that extracts the password from the environment and then calls that method. That's down in SecretStoreExt.java. It should be possible to pull all that code into the ruby filter and pass it a password instead of using the environment, but I am not inclined to write that myself. Not sure doing that would help much, you will still end up with a plaintext password in a file somewhere.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.