Hi,
I am using redis behind stunnel to gain TLS encryption for redis. Filebeat is running fine, ships it's data sucessfully to redis database.
Now I want to establish secured connection from logstash to redis.
I updated logstash to version 7.2.0 where the docs are offering a flag for ssl:
https://www.elastic.co/guide/en/logstash/current/plugins-inputs-redis.html#plugins-inputs-redis-ssl
Sounds like it is what I need.
So my input looks like this:
input
{
redis
{
data_type => "list"
db => "${REDIS_DB}"
host => "${REDIS_HOST}"
port => "${REDIS_PORT}"
ssl => "${REDIS_SSL}"
key => "timer"
}
}
When I check the environment variables on container level, they are looking correctly to me:
[root@poc-logstash-5ddf9b77db-756d6 logstash]# echo $REDIS_SSL
true
Here are the logfiles. Looks like ssl flag is unknown for logstash, but it is documented...
kubectl logs poc-logstash-5ddf9b77db-228n7
2019/07/12 09:01:22 Setting 'xpack.monitoring.elasticsearch.hosts' from environment.
2019/07/12 09:01:22 Setting 'xpack.monitoring.enabled' from environment.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2019-07-12T09:01:42,207][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2019-07-12T09:01:42,228][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2019-07-12T09:01:42,763][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.2.0"}
[2019-07-12T09:01:42,795][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"d5a7bc34-522e-4afb-bfa9-170640a783ba", :path=>"/usr/share/logstash/data/uuid"}
[2019-07-12T09:01:44,429][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://poc-es-master:9200/]}}
[2019-07-12T09:01:44,705][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://poc-es-master:9200/"}
[2019-07-12T09:01:44,758][INFO ][logstash.licensechecker.licensereader] ES Output version determined {:es_version=>7}
[2019-07-12T09:01:44,761][WARN ][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2019-07-12T09:01:44,893][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
[2019-07-12T09:01:44,894][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
[aschampe@kubernetes01 elasticsearch]$ kubectl logs poc-logstash-5ddf9b77db-228n7
2019/07/12 09:01:22 Setting 'xpack.monitoring.elasticsearch.hosts' from environment.
2019/07/12 09:01:22 Setting 'xpack.monitoring.enabled' from environment.
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2019-07-12T09:01:42,207][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2019-07-12T09:01:42,228][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2019-07-12T09:01:42,763][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.2.0"}
[2019-07-12T09:01:42,795][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"d5a7bc34-522e-4afb-bfa9-170640a783ba", :path=>"/usr/share/logstash/data/uuid"}
[2019-07-12T09:01:44,429][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://poc-es-master:9200/]}}
[2019-07-12T09:01:44,705][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://poc-es-master:9200/"}
[2019-07-12T09:01:44,758][INFO ][logstash.licensechecker.licensereader] ES Output version determined {:es_version=>7}
[2019-07-12T09:01:44,761][WARN ][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2019-07-12T09:01:44,893][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
[2019-07-12T09:01:44,894][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
[2019-07-12T09:02:12,925][ERROR][logstash.inputs.redis ] Unknown setting 'ssl' for redis
[2019-07-12T09:02:12,925][ERROR][logstash.inputs.redis ] Unknown setting 'ssl' for redis
Is it a bug or did I made a mistake?
What other options do I have to access redis via TLS? Do I really need another stunnel instance as accesspoint for logstash?
Thanks, Andreas