How do we connect the memecached filter for logstash to AWS Elasticache with tls? We are using the memcached filter for logstash to augment our logs using AWS Elasticache and the filter looks something like this:
memcached {
hosts => ["memcache.us1.cache.amazonaws.com:11212"]
get => {
"host" => "host_metadata"
}
}
We are getting the following error when trying to connect to Elasticache Memcached:
[
{
"level": "ERROR",
"loggerName": "logstash.filters.memcached",
"timeMillis": 1723492753438,
"thread": "[main]-pipeline-manager",
"logEvent":
{
"message": "No server available",
"hosts":
[
"memcache.us1.cache.amazonaws.com:11212"
],
"options":
{
"expires_in": 0
}
}
},
{
"level": "ERROR",
"loggerName": "logstash.javapipeline",
"timeMillis": 1723492753447,
"thread": "[main]-pipeline-manager",
"logEvent":
{
"message": "Pipeline error",
"pipeline_id": "main",
"exception":
{
"cause":
{
"stackTrace":
[
{
"class": "usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.dalli_minus_3_dot_2_dot_5.lib.dalli.ring",
"method": "server_for_key",
"file": "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/dalli-3.2.5/lib/dalli/ring.rb",
"line": 48
},
{
"class": "usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.dalli_minus_3_dot_2_dot_5.lib.dalli.client",
"method": "alive!",
"file": "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/dalli-3.2.5/lib/dalli/client.rb",
"line": 343
},
{
"class": "usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_filter_minus_memcached_minus_1_dot_2_dot_0.lib.logstash.filters.memcached",
"method": "new_connection",
"file": "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-memcached-1.2.0/lib/logstash/filters/memcached.rb",
"line": 178
},
{
"class": "org.jruby.RubyKernel",
"method": "tap",
"file": "org/jruby/RubyKernel.java",
"line": 1940
},
{
"class": "usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_filter_minus_memcached_minus_1_dot_2_dot_0.lib.logstash.filters.memcached",
"method": "new_connection",
"file": "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-memcached-1.2.0/lib/logstash/filters/memcached.rb",
"line": 178
},
{
"class": "usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_filter_minus_memcached_minus_1_dot_2_dot_0.lib.logstash.filters.memcached",
"method": "register",
"file": "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-memcached-1.2.0/lib/logstash/filters/memcached.rb",
"line": 84
},
{
"class": "org.logstash.config.ir.compiler.AbstractFilterDelegatorExt",
"method": "register",
"file": "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java",
"line": 75
},
{
"class": "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline",
"method": "register_plugins",
"file": "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb",
"line": 237
},
{
"class": "org.jruby.RubyArray",
"method": "each",
"file": "org/jruby/RubyArray.java",
"line": 1865
},
{
"class": "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline",
"method": "register_plugins",
"file": "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb",
"line": 236
},
{
"class": "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline",
"method": "maybe_setup_out_plugins",
"file": "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb",
"line": 608
},
{
"class": "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline",
"method": "start_workers",
"file": "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb",
"line": 249
},
{
"class": "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline",
"method": "run",
"file": "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb",
"line": 194
},
{
"class": "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline",
"method": "start",
"file": "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb",
"line": 146
}
],
"exception":
{
"metaClass":
{
"metaClass":
{
"metaClass":
{
"exception": "org.jruby.exceptions.RuntimeError: (RuntimeError) failed to connect to memcached",
"backtrace":
[
"/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-memcached-1.2.0/lib/logstash/filters/memcached.rb:87:in `register'",
"org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'",
"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:237:in `block in register_plugins'",
"org/jruby/RubyArray.java:1865:in `each'",
"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:236:in `register_plugins'",
"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:608:in `maybe_setup_out_plugins'",
"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:249:in `start_workers'",
"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'",
"/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"
],
"pipeline.sources":
[
"/etc/logstash/conf.d/000_input.conf",
"/etc/logstash/conf.d/015_naming.conf",
"/etc/logstash/conf.d/020_cleanup.conf",
"/etc/logstash/conf.d/025_mandatory_fields.conf",
"/etc/logstash/conf.d/030_vmware_metadata.conf",
"/etc/logstash/conf.d/999_output.conf"
],
"thread": "#<Thread:0x43391a7 run>"
}
}
}
}
}
}
}
}
]
This works just fine when pointing to a local memcached server in our datacenter, but not when pointing to Elasticache. One difference between Elasticache and our local server is Elasticache uses TLS. Looking at the source code for logstash-filter-memcached it doesn't look like tls is supported:
This is how tls should be configured:
This is how logstash-filter-memcached calls Dallit the Memcached client
Is there away to get this working with Elasticache?