Memcached performance

As I have not encountered any post related to the memcached so far.

I have a very powerful machine, 512 GB of ram, RAID0 of SSD, 80 CPU cores of Xeon v4.... so the hardware is not the limit, potentially my misconfiguration.

My memcached settings are:

logfile /var/log/memcached.log
# -v
-m 512
-p 11211
-u memcache
-t 32
# -c 1024
# -k
# -M
# -r

# service memcached status
● memcached.service - LSB: Start memcached daemon
   Loaded: loaded (/etc/init.d/memcached; generated)
   Active: active (running) since Wed 2019-05-22 14:26:55 CEST; 1h 14min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 15169 ExecStart=/etc/init.d/memcached start (code=exited, status=0/SUCCESS)
    Tasks: 35 (limit: 4915)
   CGroup: /system.slice/memcached.service
           └─15176 /usr/bin/memcached -vv -m 512 -p 11211 -u memcache -l -t 32

Logstash partial filter (precumputed md5 hash is passed from the grok to the memcached get query) config:

memcached {
        hosts => [ "localhost:11211" ]
        get => {
                "%{md5_hash}" => WHITELISTED_HASH

From memcache-top:

From logstash monitoring:

Memcached is running on the same machine, localhost where MTU is maxed out:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet scope host lo
valid_lft forever preferred_lft forever

Logstash seems to have only 4 connections over the localhost to the local memcached instance and it seems to be the bottleneck.

Anyone had the same problem/performance limitation etc?
What is your performance when running the memcached on the local machine?
I guess I have hit the Logstash processing limit and I need to place Kawka or other message broker in front of Logstash galaxy in order to get higher numbers.

I wonder if the lookups could be improved by switching to the jdbc plugin?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.