Hello.
As I have not encountered any post related to the memcached so far.
I have a very powerful machine, 512 GB of ram, RAID0 of SSD, 80 CPU cores of Xeon v4.... so the hardware is not the limit, potentially my misconfiguration.
My memcached settings are:
-d
logfile /var/log/memcached.log
# -v
-vv
-m 512
-p 11211
-u memcache
-l 127.0.0.1
-t 32
# -c 1024
# -k
# -M
# -r
# service memcached status
● memcached.service - LSB: Start memcached daemon
Loaded: loaded (/etc/init.d/memcached; generated)
Active: active (running) since Wed 2019-05-22 14:26:55 CEST; 1h 14min ago
Docs: man:systemd-sysv-generator(8)
Process: 15169 ExecStart=/etc/init.d/memcached start (code=exited, status=0/SUCCESS)
Tasks: 35 (limit: 4915)
CGroup: /system.slice/memcached.service
└─15176 /usr/bin/memcached -vv -m 512 -p 11211 -u memcache -l 127.0.0.1 -t 32
Logstash partial filter (precumputed md5 hash is passed from the grok to the memcached get query) config:
memcached { hosts => [ "localhost:11211" ] get => { "%{md5_hash}" => WHITELISTED_HASH } }
From memcache-top:
From logstash monitoring:
Memcached is running on the same machine, localhost where MTU is maxed out:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
Logstash seems to have only 4 connections over the localhost to the local memcached instance and it seems to be the bottleneck.
Anyone had the same problem/performance limitation etc?
What is your performance when running the memcached on the local machine?
I guess I have hit the Logstash processing limit and I need to place Kawka or other message broker in front of Logstash galaxy in order to get higher numbers.
I wonder if the lookups could be improved by switching to the jdbc plugin?