# Failed to Fetching the Redis Info and keyspace logs to Kibana **URL:** \ **Category:** Beats\ **Tags:** filebeat, metricbeat\ **Created:** [January 2, 2024, 8:10am UTC](https://discuss.elastic.co/t/failed-to-fetching-the-redis-info-and-keyspace-logs-to-kibana/350201 "2024-01-02T08:10:10Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Pranjal\_Sett](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pranjal_sett/32/98834_2.png) [@Pranjal\_Sett](https://discuss.elastic.co/u/Pranjal_Sett)\ **Post date:** [January 2, 2024, 8:10am UTC](https://discuss.elastic.co/t/failed-to-fetching-the-redis-info-and-keyspace-logs-to-kibana/350201/1 "2024-01-02T08:10:10Z")
So my task was to install the metricbeat and enable the redis module to pick the INFO and Keyspace values. By providing that I was facing lot of issues mostly on TCP related. I have tried multiple ways to mitigate this but failed to resolve the issue. The error I was facing like below: [Masking the IP] "message":"Error retrieving INFO stats: read tcp :35536-\>:6379: read: connection reset by peer","service.name":"metricbeat","ecs.version":"1.6.0"} 56},"message":"Error fetching data for metricset redis.keyspace: Failed to fetch redis info for keyspaces: read tcp :35536-\>:6379: read: connection reset by peer","se\> "message":"Error retrieving INFO stats: read tcp :35536-\>:6379: read: connection reset by peer","service.name":"metricbeat","ecs.version":"1.6.0"} 56},"message":"Error fetching data for metricset redis.info: failed to fetch redis info: read tcp :35536-\>:6379: read: connection reset by peer","service.name": Now I have tried the another way to fetch the redis info details. 1. Installed the filebat. 2. wrote a shell script to fetch the redis details and store it to one output file and scheduled to crontab 3. In filebeat I have given the path to fetch the output file. So the idea is fetch the output file and send it to logstahs for parsing the logs. Now I have written the grok pattern and not able to parse properly. Here is the format of the logs and filebat multiline pattern conf and logstash conf. Filebeat: multiline.pattern: '\[1\]' multiline.negate: false multiline.match: after Lostash: input { beats { port =\> "8" } } filter { if "redis\_server\_sit\_1" in [tags] { grok { match =\> { "message" =\> "# Server\s_redis\_version:%{DATA:redis\_version}\s_redis\_git\_sha1:%{NUMBER:redis\_git\_sha1}\s_redis\_git\_dirty:%{NUMBER:redis\_git\_dirty}\s_redis\_build\_id:%{WORD:redis\_build\_id}\s_redis\_mode:%{WORD:redis\_mode}\s_os:%{DATA:os}\s_arch\_bits:%{NUMBER:arch\_bits}\s_monotonic\_clock:%{DATA:monotonic\_clock}\s_multiplexing\_api:%{WORD:multiplexing\_api}\s_atomicvar\_api:%{DATA:atomicvar\_api}\s_gcc\_version:%{DATA:gcc\_version}\s_process\_id:%{NUMBER:process\_id}\s_process\_supervised:%{WORD:process\_supervised}\s_run\_id:%{DATA:run\_id}\s_tcp\_port:%{NUMBER:tcp\_port}\s_server\_time\_usec:%{NUMBER:server\_time\_usec}\s_uptime\_in\_seconds:%{NUMBER:uptime\_in\_seconds}\s_uptime\_in\_days:%{NUMBER:uptime\_in\_days}\s_hz:%{NUMBER:hz}\s_configured\_hz:%{NUMBER:configured\_hz}\s_lru\_clock:%{NUMBER:lru\_clock}\s_executable:%{DATA:executable}\s_config\_file:%{DATA:config\_file}\s_io\_threads\_active:%{NUMBER:io\_threads\_active}\s\*# Clients\s_connected\_clients:%{NUMBER:connected\_clients}\s_cluster\_connections:%{NUMBER:cluster\_connections}\s_maxclients:%{NUMBER:maxclients}\s_client\_recent\_max\_input\_buffer:%{NUMBER:client\_recent\_max\_input\_buffer}\s_client\_recent\_max\_output\_buffer:%{NUMBER:client\_recent\_max\_output\_buffer}\s_blocked\_clients:%{NUMBER:blocked\_clients}\s_tracking\_clients:%{NUMBER:tracking\_clients}\s_clients\_in\_timeout\_table:%{NUMBER:clients\_in\_timeout\_table}\s\*# Memory\s_used\_memory:%{NUMBER:used\_memory}\s_used\_memory\_human:%{DATA:used\_memory\_human}\s_used\_memory\_rss:%{NUMBER:used\_memory\_rss}\s_used\_memory\_rss\_human:%{DATA:used\_memory\_rss\_human}\s_used\_memory\_peak:%{DATA:used\_memory\_peak}\s_used\_memory\_peak\_human:%{DATA:used\_memory\_peak\_human}\s_used\_memory\_peak\_perc:%{DATA:used\_memory\_peak\_perc}\s_used\_memory\_overhead:%{NUMBER:used\_memory\_overhead}\s_used\_memory\_startup:%{NUMBER:used\_memory\_startup}\s_used\_memory\_dataset:%{NUMBER:used\_memory\_dataset}\s_used\_memory\_dataset\_perc:%{DATA:used\_memory\_dataset\_perc}\s_allocator\_allocated:%{NUMBER:allocator\_allocated}\s_allocator\_active:%{NUMBER:allocator\_active}\s_allocator\_resident:%{NUMBER:allocator\_resident}\s_total\_system\_memory:%{NUMBER:total\_system\_memory}\s_total\_system\_memory\_human:%{DATA:total\_system\_memory\_human}\s_used\_memory\_lua:%{NUMBER:used\_memory\_lua}\s_used\_memory\_lua\_human:%{DATA:used\_memory\_lua\_human}\s_used\_memory\_scripts:%{NUMBER:used\_memory\_scripts}\s_used\_memory\_scripts\_human:%{WORD:used\_memory\_scripts\_human}\s_number\_of\_cached\_scripts:%{NUMBER:number\_of\_cached\_scripts}\s_maxmemory:%{NUMBER:maxmemory}\s_maxmemory\_human:%{DATA:maxmemory\_human}\s_maxmemory\_policy:%{WORD:maxmemory\_policy}\s_allocator\_frag\_ratio:%{NUMBER:allocator\_frag\_ratio}\s_allocator\_frag\_bytes:%{NUMBER:allocator\_frag\_bytes}\s_allocator\_rss\_ratio:%{NUMBER:allocator\_rss\_ratio}\s_allocator\_rss\_bytes:%{NUMBER:allocator\_rss\_bytes}\s_rss\_overhead\_ratio:%{NUMBER:rss\_overhead\_ratio}\s_rss\_overhead\_bytes:%{NUMBER:rss\_overhead\_bytes}\s_mem\_fragmentation\_ratio:%{NUMBER:mem\_fragmentation\_ratio}\s_mem\_fragmentation\_bytes:%{NUMBER:mem\_fragmentation\_bytes}\s_mem\_not\_counted\_for\_evict:%{NUMBER:mem\_not\_counted\_for\_evict}\s_mem\_replication\_backlog:%{NUMBER:mem\_replication\_backlog}\s_mem\_clients\_slaves:%{NUMBER:mem\_clients\_slaves}\s_mem\_clients\_normal:%{NUMBER:mem\_clients\_normal}\s_mem\_aof\_buffer:%{NUMBER:mem\_aof\_buffer}\s_mem\_allocator:%{DATA:mem\_allocator}\s_active\_defrag\_running:%{NUMBER:active\_defrag\_running}\s_lazyfree\_pending\_objects:%{NUMBER:lazyfree\_pending\_objects}\s_lazyfreed\_objects:%{NUMBER:lazyfreed\_objects}\s_# Persistence\s_loading:%{NUMBER:loading}\s_current\_cow\_size:%{NUMBER:current\_cow\_size}\s_current\_cow\_size\_age:%{NUMBER:current\_cow\_size\_age}\s_current\_fork\_perc:%{NUMBER:current\_fork\_perc}\s_current\_save\_keys\_processed:%{NUMBER:current\_save\_keys\_processed}\s_current\_save\_keys\_total:%{NUMBER:current\_save\_keys\_total}\s_rdb\_changes\_since\_last\_save:%{NUMBER:rdb\_changes\_since\_last\_save}\s_rdb\_bgsave\_in\_progress:%{NUMBER:rdb\_bgsave\_in\_progress}\s_rdb\_last\_save\_time:%{NUMBER:rdb\_last\_save\_time}\s_rdb\_last\_bgsave\_status:%{WORD:rdb\_last\_bgsave\_status}\s_rdb\_last\_bgsave\_time\_sec:%{NUMBER:rdb\_last\_bgsave\_time\_sec}\s_rdb\_current\_bgsave\_time\_sec:%{NUMBER:rdb\_current\_bgsave\_time\_sec}\s_rdb\_last\_cow\_size:%{NUMBER:rdb\_last\_cow\_size}\s_aof\_enabled:%{NUMBER:aof\_enabled}\s_aof\_rewrite\_in\_progress:%{NUMBER:aof\_rewrite\_in\_progress}\s_aof\_rewrite\_scheduled:%{NUMBER:aof\_rewrite\_scheduled}\s_aof\_last\_rewrite\_time\_sec:%{NUMBER:aof\_last\_rewrite\_time\_sec}\s_aof\_current\_rewrite\_time\_sec:%{NUMBER:aof\_current\_rewrite\_time\_sec}\s_aof\_last\_bgrewrite\_status:%{WORD:aof\_last\_bgrewrite\_status}\s_aof\_last\_write\_status:%{WORD:aof\_last\_write\_status}\s_aof\_last\_cow\_size:%{NUMBER:aof\_last\_cow\_size}\s_module\_fork\_in\_progress:%{NUMBER:module\_fork\_in\_progress}\s_module\_fork\_last\_cow\_size:%{NUMBER:module\_fork\_last\_cow\_size}\s_# Stats\s_total\_connections\_received:%{NUMBER:total\_connections\_received}\s_total\_commands\_processed:%{NUMBER:total\_commands\_processed}\s_instantaneous\_ops\_per\_sec:%{NUMBER:instantaneous\_ops\_per\_sec}\s_total\_net\_input\_bytes:%{NUMBER:total\_net\_input\_bytes}\s_total\_net\_output\_bytes:%{NUMBER:total\_net\_output\_bytes}\s_instantaneous\_input\_kbps:%{NUMBER:instantaneous\_input\_kbps}\s_instantaneous\_output\_kbps:%{NUMBER:instantaneous\_output\_kbps}\s_rejected\_connections:%{NUMBER:rejected\_connections}\s_sync\_full:%{NUMBER:sync\_full}\s_sync\_partial\_ok:%{NUMBER:sync\_partial\_ok}\s_sync\_partial\_err:%{NUMBER:sync\_partial\_err}\s_expired\_keys:%{NUMBER:expired\_keys}\s_expired\_stale\_perc:%{NUMBER:expired\_stale\_perc}\s_expired\_time\_cap\_reached\_count:%{NUMBER:expired\_time\_cap\_reached\_count}\s_expire\_cycle\_cpu\_milliseconds:%{NUMBER:expire\_cycle\_cpu\_milliseconds}\s_evicted\_keys:%{NUMBER:evicted\_keys}\s_keyspace\_hits:%{NUMBER:keyspace\_hits}\s_keyspace\_misses:%{NUMBER:keyspace\_misses}\s_pubsub\_channels:%{NUMBER:pubsub\_channels}\s_pubsub\_patterns:%{NUMBER:pubsub\_patterns}\s_latest\_fork\_usec:%{NUMBER:latest\_fork\_usec}\s_total\_forks:%{NUMBER:total\_forks}\s_migrate\_cached\_sockets:%{NUMBER:migrate\_cached\_sockets}\s_slave\_expires\_tracked\_keys:%{NUMBER:slave\_expires\_tracked\_keys}\s_active\_defrag\_hits:%{NUMBER:active\_defrag\_hits}\s_active\_defrag\_misses:%{NUMBER:active\_defrag\_misses}\s_active\_defrag\_key\_hits:%{NUMBER:active\_defrag\_key\_hits}\s_active\_defrag\_key\_misses:%{NUMBER:active\_defrag\_key\_misses}\s_tracking\_total\_keys:%{NUMBER:tracking\_total\_keys}\s_tracking\_total\_items:%{NUMBER:tracking\_total\_items}\s_tracking\_total\_prefixes:%{NUMBER:tracking\_total\_prefixes}\s_unexpected\_error\_replies:%{NUMBER:unexpected\_error\_replies}\s_total\_error\_replies:%{NUMBER:total\_error\_replies}\s_dump\_payload\_sanitizations:%{NUMBER:dump\_payload\_sanitizations}\s_total\_reads\_processed:%{NUMBER:total\_reads\_processed}\s_total\_writes\_processed:%{NUMBER:total\_writes\_processed}\s_io\_threaded\_reads\_processed:%{NUMBER:io\_threaded\_reads\_processed}\s_io\_threaded\_writes\_processed:%{NUMBER:io\_threaded\_writes\_processed}\s\*# Replication\s_role:%{WORD:role}\s_master\_host:%{IP:master\_host}\s_master\_port:%{NUMBER:master\_port}\s_master\_link\_status:%{WORD:master\_link\_status}\s_master\_last\_io\_seconds\_ago:%{NUMBER:master\_last\_io\_seconds\_ago}\s_master\_sync\_in\_progress:%{NUMBER:master\_sync\_in\_progress}\s_slave\_read\_repl\_offset:%{NUMBER:slave\_read\_repl\_offset}\s_slave\_repl\_offset:%{NUMBER:slave\_repl\_offset}\s_slave\_priority:%{NUMBER:slave\_priority}\s_slave\_read\_only:%{NUMBER:slave\_read\_only}\s_replica\_announced:%{NUMBER:replica\_announced}\s_connected\_slaves:%{NUMBER:connected\_slaves}\s_master\_failover\_state:%{DATA:master\_failover\_state}\s_master\_replid:%{WORD:master\_replid}\s_master\_replid2:%{WORD:master\_replid2}\s_master\_repl\_offset:%{NUMBER:master\_repl\_offset}\s_second\_repl\_offset:%{NUMBER:second\_repl\_offset}\s_repl\_backlog\_active:%{NUMBER:repl\_backlog\_active}\s_repl\_backlog\_size:%{NUMBER:repl\_backlog\_size}\s_repl\_backlog\_first\_byte\_offset:%{NUMBER:repl\_backlog\_first\_byte\_offset}\s_repl\_backlog\_histlen:%{NUMBER:repl\_backlog\_histlen}\s_# CPU\s_used\_cpu\_sys:%{NUMBER:used\_cpu\_sys}\s_used\_cpu\_user:%{NUMBER:used\_cpu\_user}\s_used\_cpu\_sys\_children:%{NUMBER:used\_cpu\_sys\_children}\s_used\_cpu\_user\_children:%{NUMBER:used\_cpu\_user\_children}\s_used\_cpu\_sys\_main\_thread:%{NUMBER:used\_cpu\_sys\_main\_thread}\s_used\_cpu\_user\_main\_thread:%{NUMBER:used\_cpu\_user\_main\_thread}\s\*# Modules\s\*# Errorstats\s_errorstat\_ERR:count=%{NUMBER:errorstat\_ERR\_count}\s_# Cluster\s_cluster\_enabled:%{NUMBER:cluster\_enabled}\s_# Keyspace" } } mutate { add\_tag =\> "redis\_logs" } } } output { if "redis\_logs" in [tags] { elasticsearch { hosts =\> ["host1:9200","host2:9200","host3:9200"] ssl =\> true user =\> "user" password =\> "pass" truststore =\> "/etc/ela12.ts" truststore\_password =\> "pass" ssl\_certificate\_verification =\> true ilm\_rollover\_alias =\> "redis-logs" ilm\_pattern =\> "{now/d}-000001" ilm\_policy =\> "redis-logs" ``` } ``` } } the above one is configuration. but got the grok parsing error but in grok debugger it is coming perfectly fine. the log pattern is: # Server redis\_version:6.2.7 redis\_git\_sha1:00000000 redis\_git\_dirty:0 redis\_build\_id:5d88ce217879027a redis\_mode:standalone os:Linux 4.18.0-477.15.1.el8\_8.x86\_64 x86\_64 arch\_bits:64 monotonic\_clock:POSIX clock\_gettime multiplexing\_api:epoll atomicvar\_api:c11-builtin gcc\_version:8.5.0 process\_id:3297159 process\_supervised:systemd run\_id:47c587c5f86a9d37b780c4de6bbcf4656a8d4cab tcp\_port:6379 server\_time\_usec:1703757767880238 uptime\_in\_seconds:1466763 uptime\_in\_days:16 hz:10 configured\_hz:10 lru\_clock:9258951 executable:/usr/bin/redis-server config\_file:/etc/redis/redis.conf io\_threads\_active:0 # Clients connected\_clients:16 cluster\_connections:0 maxclients:10000 client\_recent\_max\_input\_buffer:32 client\_recent\_max\_output\_buffer:0 blocked\_clients:0 tracking\_clients:0 clients\_in\_timeout\_table:0 # Memory used\_memory:2325192 used\_memory\_human:2.22M used\_memory\_rss:17420288 used\_memory\_rss\_human:16.61M used\_memory\_peak:2734208 used\_memory\_peak\_human:2.61M used\_memory\_peak\_perc:85.04% used\_memory\_overhead:2169768 used\_memory\_startup:813120 used\_memory\_dataset:155424 used\_memory\_dataset\_perc:10.28% allocator\_allocated:2389120 allocator\_active:2850816 allocator\_resident:5304320 total\_system\_memory:6192857088 total\_system\_memory\_human:5.77G used\_memory\_lua:35840 used\_memory\_lua\_human:35.00K used\_memory\_scripts:528 used\_memory\_scripts\_human:528B number\_of\_cached\_scripts:2 maxmemory:1073741824 maxmemory\_human:1.00G maxmemory\_policy:noeviction allocator\_frag\_ratio:1.19 allocator\_frag\_bytes:461696 allocator\_rss\_ratio:1.86 allocator\_rss\_bytes:2453504 rss\_overhead\_ratio:3.28 rss\_overhead\_bytes:12115968 mem\_fragmentation\_ratio:7.57 mem\_fragmentation\_bytes:15117888 mem\_not\_counted\_for\_evict:0 mem\_replication\_backlog:1048576 mem\_clients\_slaves:0 mem\_clients\_normal:307544 mem\_aof\_buffer:0 mem\_allocator:jemalloc-5.1.0 active\_defrag\_running:0 lazyfree\_pending\_objects:0 lazyfreed\_objects:0 # Persistence loading:0 current\_cow\_size:0 current\_cow\_size\_age:0 current\_fork\_perc:0.00 current\_save\_keys\_processed:0 current\_save\_keys\_total:0 rdb\_changes\_since\_last\_save:38 rdb\_bgsave\_in\_progress:0 rdb\_last\_save\_time:1703756924 rdb\_last\_bgsave\_status:ok rdb\_last\_bgsave\_time\_sec:0 rdb\_current\_bgsave\_time\_sec:-1 rdb\_last\_cow\_size:536576 aof\_enabled:0 aof\_rewrite\_in\_progress:0 aof\_rewrite\_scheduled:0 aof\_last\_rewrite\_time\_sec:-1 aof\_current\_rewrite\_time\_sec:-1 aof\_last\_bgrewrite\_status:ok aof\_last\_write\_status:ok aof\_last\_cow\_size:0 module\_fork\_in\_progress:0 module\_fork\_last\_cow\_size:0 # Stats total\_connections\_received:122 total\_commands\_processed:9309604 instantaneous\_ops\_per\_sec:6 total\_net\_input\_bytes:704769007 total\_net\_output\_bytes:3921457572 instantaneous\_input\_kbps:0.55 instantaneous\_output\_kbps:4.32 rejected\_connections:0 sync\_full:0 sync\_partial\_ok:0 sync\_partial\_err:0 expired\_keys:0 expired\_stale\_perc:0.00 expired\_time\_cap\_reached\_count:0 expire\_cycle\_cpu\_milliseconds:0 evicted\_keys:0 keyspace\_hits:0 keyspace\_misses:1 pubsub\_channels:2 pubsub\_patterns:0 latest\_fork\_usec:683 total\_forks:553 migrate\_cached\_sockets:0 slave\_expires\_tracked\_keys:0 active\_defrag\_hits:0 active\_defrag\_misses:0 active\_defrag\_key\_hits:0 active\_defrag\_key\_misses:0 tracking\_total\_keys:0 tracking\_total\_items:0 tracking\_total\_prefixes:0 unexpected\_error\_replies:0 total\_error\_replies:199 dump\_payload\_sanitizations:0 total\_reads\_processed:9080383 total\_writes\_processed:21023215 io\_threaded\_reads\_processed:0 io\_threaded\_writes\_processed:0 # Replication role:slave master\_host:IP master\_port:6379 master\_link\_status:up master\_last\_io\_seconds\_ago:0 master\_sync\_in\_progress:0 slave\_read\_repl\_offset:2676731963 slave\_repl\_offset:2676731963 slave\_priority:100 slave\_read\_only:1 replica\_announced:1 connected\_slaves:0 master\_failover\_state:no-failover master\_replid:75c2674d500152ef1e92f72065940c8c9292e2f5 master\_replid2:0000000000000000000000000000000000000000 master\_repl\_offset:2676731963 second\_repl\_offset:-1 repl\_backlog\_active:1 repl\_backlog\_size:1048576 repl\_backlog\_first\_byte\_offset:2675683388 repl\_backlog\_histlen:1048576 # CPU used\_cpu\_sys:1057.229563 used\_cpu\_user:1217.442233 used\_cpu\_sys\_children:0.921917 used\_cpu\_user\_children:0.246200 used\_cpu\_sys\_main\_thread:1049.807073 used\_cpu\_user\_main\_thread:1211.922890 # Modules # Errorstats errorstat\_ERR:count=199 # Cluster cluster\_enabled:0 # Keyspace Now what is happening by doing so... In grok debugger the above one is parsing correctly. But the problem is happening every line is treated as a new event. Filebat does not able to send the whole logs at a time to logstash for parsing. It is sending like # Server as a different event redis\_version:6.2.7 as a different event redis\_git\_sha1:00000000 as a different event redis\_git\_dirty:0 as a different event redis\_build\_id:5d88ce217879027a as a different event redis\_mode:standalone as a different event And in kibana I can see every line in a every document. like # Server is a one document like redis\_version:6.2.7 is a one document like redis\_git\_sha1:00000000 is a one document This is how 174 documents are generating instead having one document * * * 1. # ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 30, 2024, 10:11am UTC](https://discuss.elastic.co/t/failed-to-fetching-the-redis-info-and-keyspace-logs-to-kibana/350201/2 "2024-01-30T10:11:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.