Memcached error saturate the syslog

Elastic Stack Beats
1

Hi eveyone,

I need some help, i'm experimenting problems with packetbeat, i've the following error:

Jun 11 03:16:43 nombrehost /usr/bin/packetbeat[29569]: log.go:114: Stacktrace: /go/src/github.com/elastic/beats/libbeat/logp/log.go:114 (0x4c5a96)#012/usr/local/go/src/runtime/asm_amd64.s:437 (0x47c88e)#012/usr/local/go/src/runtime/panic.go:423 (0x44b119)#012/usr/local/go/src/runtime/panic.go:42 (0x4497d9)#012/usr/local/go/src/runtime/sigpanic_unix.go:24 (0x46024a)#012/go/src/github.com/elastic/beats/packetbeat/protos/memcache/plugin_tcp.go:335 (0x4f9bad)#012/go/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:112 (0x52cc9d)#012/go/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:148 (0x52d4ac)#012/go/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:183 (0x7137b7)#012/go/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:101 (0x712cf1)#012/go/src/github.com/elastic/beats/packetbeat/sniffer/sniffer.go:356 (0x5417f9)#012/go/src/github.com/elastic/beats/packetbeat/beat/packetbeat.go:232 (0x481d4b)#012/usr/local/go/src/runtime/asm_amd64.s:1696 (0x47ebd1)
Jun 11 03:19:24 ubumem000 /usr/bin/packetbeat[31426]: log.go:113: GapInStream(memcache) exception. Recovering, but please report this: runtime error: invalid memory address or nil pointer dereference.
Jun 11 03:19:24 nombrehost /usr/bin/packetbeat[31426]: log.go:114: Stacktrace: /go/src/github.com/elastic/beats/libbeat/logp/log.go:114 (0x4c5a96)#012/usr/local/go/src/runtime/asm_amd64.s:437 (0x47c88e)#012/usr/local/go/src/runtime/panic.go:423 (0x44b119)#012/usr/local/go/src/runtime/panic.go:42 (0x4497d9)#012/usr/local/go/src/runtime/sigpanic_unix.go:24 (0x46024a)#012/go/src/github.com/elastic/beats/packetbeat/protos/memcache/plugin_tcp.go:335 (0x4f9bad)#012/go/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:112 (0x52cc9d)#012/go/src/github.com/elastic/beats/packetbeat/protos/tcp/tcp.go:148 (0x52d4ac)#012/go/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:183 (0x7137b7)#012/go/src/github.com/elastic/beats/packetbeat/decoder/decoder.go:101 (0x712cf1)#012/go/src/github.com/elastic/beats/packetbeat/sniffer/sniffer.go:356 (0x5417f9)#012/go/src/github.com/elastic/beats/packetbeat/beat/packetbeat.go:232 (0x481d4b)#012/usr/local/go/src/runtime/asm_amd64.s:1696 (0x47ebd1)

2

Error is triggered due to packet loss. The memcached plugin tries to recover from packet loss (not drop complete message), if it's in middle of some bigger message.

Reading the memcache code I can see this exception happen if 2 consecutive packet losses are noticed by TCP module in case memcache module decided to drop state on packetloss.

Related github issue: https://github.com/elastic/beats/issues/1162

Fix should be easy (well, you never know ;)).

1 Like
3

Ouuu! thanks so much for your help :smiley:

4

This topic was automatically closed after 21 days. New replies are no longer allowed.