# Sending logs from filebeat to redis and then logstash **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 22, 2018, 5:47am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718 "2018-05-22T05:47:42Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![remo](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@remo](https://discuss.elastic.co/u/remo)\ **Post date:** [May 22, 2018, 5:47am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/1 "2018-05-22T05:47:43Z")
I am trying to send my logs from filebeat into redis and then from redis into logstash and elasticsearch. My log file is in json format. What configuration is to be made in my filebeat.yml so that the output is redis. I am getting the error as ( Failed to RPUSH to redis list with ERR wrong number of arguments for 'rpush' command 2018-05-22T11:15:43.063+0530 ERROR pipeline/output.go:92 Failed to publish events: ERR wrong number of arguments for 'rpush' command). Also what configuartion is to be made in my logstash.yml when my input is redis. ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [May 22, 2018, 6:05am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/2 "2018-05-22T06:05:23Z")
Hi, Could you please brief your use case that why are you implementing this mean what is the requirement and paste your pipeline here so that we can check the issue. Regards, ---
**Author:** ![remo](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@remo](https://discuss.elastic.co/u/remo)\ **Post date:** [May 22, 2018, 9:35am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/3 "2018-05-22T09:35:56Z")
i want to make visualizations on kibana of my logs. the pipeline is as follows : Filebeat - Redis - Logstash - Elasticsearch - Kibana. The reason i have included redis in my pipeline is beacause it acts as a buffer and hence in case of data loss we can approach redis for that. But i am unable to send data from filebeat to redis itself. Filebeat - Logstash - Elasticsearch - Kibana this pipeline is working fine. But when I am using redis, I am unable to load data from filebeat to redis. I have attached the log file which m trying to send in the pipeline. Please provide the solution. 2018-05-18 09:29:24,387 [main] INFO org.mule.module.launcher.application.DefaultMuleApplication - ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - Initializing app 'RealtimeFlow' + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2018-05-18 09:29:24,543 [main] INFO org.mule.lifecycle.AbstractLifecycleManager - Initialising RegistryBroker 2018-05-18 09:29:24,636 [main] INFO org.mule.module.extension.internal.manager.DefaultExtensionManager - Starting discovery of extensions 2018-05-18 09:29:24,745 [main] INFO org.mule.module.extension.internal.manager.DefaultExtensionManager - Discovered 1 extensions 2018-05-18 09:29:24,745 [main] INFO org.mule.module.extension.internal.manager.DefaultExtensionManager - Registering extension validation (version 3.8) 2018-05-18 09:29:24,809 [main] INFO org.mule.config.spring.MuleArtifactContext - Refreshing org.mule.config.spring.MuleArtifactContext@9573b3b: startup date [Fri May 18 09:29:24 IST 2018]; root of context hierarchy 2018-05-18 09:29:26,353 [main] INFO org.mule.config.spring.processors.NoDevkitInjectorProcessor - JSR-330 'javax.inject.Inject' annotation found and supported for autowiring 2018-05-18 09:29:31,942 [main] INFO org.mule.module.management.agent.WrapperManagerAgent - This JVM hasn't been launched by the wrapper, the agent will not run. 2018-05-18 09:29:31,973 [main] INFO org.mule.DefaultMuleContext - * * * - Application: RealtimeFlow \* - OS encoding: , Mule encoding: UTF-8 \* - ``` * ``` - Agents Running: \* - JMX Agent \* - DevKit Extension Information \* - Batch module default engine \* - Wrapper Manager \* * * * 2018-05-18 09:29:41,140 [[RealtimeFlow].HTTP\_Listener\_Configuration.worker.01] INFO com.mulesoft.weave.mule.utils.MuleWeaveFactory$ - MimeType was not resolved '_/_' delegating to Java. 2018-05-18 09:29:41,141 [[RealtimeFlow].HTTP\_Listener\_Configuration.worker.01] INFO org.mule.api.processor.LoggerMessageProcessor - **{"versionId":"1.0","environment":{"organizationId":"ejgje","businessUnitId":"dfsf","projectId":"dvbfsd","environmentId":"DEV","serverId":"10.45.43.223"},"interface":{"interfaceId":"REALTIME\_SAP\_interface","sourceId":"SFDC","targetId":"SAP"},"transactionDetail":{"correlationId":"e425d140-5a4f-11e8-81ec-320920524153","businessId":null,"executionTime":"2018-05-18 09:29:41:140","executionPoint":"Execution staring point of REALTIME\_SAP\_interface ","executionMessage":"RealTime Main Flow has been Started and setting the flow vars for logging purpose","executionStatus":"INPROCESS","payload":{}},"error":{"errorType":null,"errorCode":null,"errorText":null}}** 2018-05-18 09:29:41,157 [[RealtimeFlow].HTTP\_Listener\_Configuration.worker.01] INFO com.mulesoft.weave.mule.utils.MuleWeaveFactory$ - MimeType was not resolved '_/_' delegating to Java. ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [May 22, 2018, 9:57am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/4 "2018-05-22T09:57:57Z")
Hi, Could you please paste your filebeat.yml and redis config file here for further invesigation. Thanks, ---
**Author:** ![remo](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@remo](https://discuss.elastic.co/u/remo)\ **Post date:** [May 22, 2018, 11:07am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/5 "2018-05-22T11:07:25Z")
Hi, Below is the configuration of filebeat.yml filebeat.prospectors: - type: log enabled: true paths: - C:\Users\adfgdhchg\Documents\My Received Files\json.log filebeat.config.modules: path: ${path.config}/modules.d/\*.yml reload.enabled: false filebeat.modules: - module: redis log: enabled: true slowlog: enabled: true #------------------------------- Redis output ---------------------------------- output.redis: hosts: ["localhost:6379"] key: "logs" db: 0 timeout: 5 And in redis I didnt make any configuration. In logstash I have given input as input { redis { hosts =\> "localhost:6379" data\_type =\> "list" key =\> "logs" } } and output as output{ elasticsearch { hosts =\> "localhost:9200" index =\> "finalmule\_logs" user=\> elastic password=\>elastic } stdout { } } Thank you ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [May 22, 2018, 11:39am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/6 "2018-05-22T11:39:52Z")
Hi, Hope you have checked the version compatibility if not please check with below link. [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-redis.html#\_compatibility\_14](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-redis.html#_compatibility_14) And please paste your filebeat logs here while starting the service. Regards, ---
**Author:** ![remo](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@remo](https://discuss.elastic.co/u/remo)\ **Post date:** [May 22, 2018, 1:00pm UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/7 "2018-05-22T13:00:20Z")
Hi, My redis version is 2.4.5 and F E L K version is 6.2.4 I followed the steps on the provided link but still i am getting the same error( Failed to rpush to redis list). Please provide me the solution. ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [May 23, 2018, 4:39am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/8 "2018-05-23T04:39:07Z")
Hi, Please let me know below details. Are you able to connect the redis server from filebeat (please check with telnet) ? could you please check with increase the below timeout parameter. > [@](#): > > timeout: 5 if you still getting the error please paste your debug logs here for more investigation. Regards, ---
**Author:** ![remo](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@remo](https://discuss.elastic.co/u/remo)\ **Post date:** [May 23, 2018, 12:38pm UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/9 "2018-05-23T12:38:59Z")
Hi, I increased the timeout paramater value also still i am getting the same error. the error is : **Failed to rpush to redis list.** **Failed to publish events.** Please help me out with this. Thanks. ---
**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\ **Post date:** [May 24, 2018, 9:34am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/10 "2018-05-24T09:34:56Z")
Hi, Have you checked the below details. Please check if not. > [@](#): > > Are you able to connect the redis server from filebeat (please check with telnet) ? Regards, ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 21, 2018, 9:35am UTC](https://discuss.elastic.co/t/sending-logs-from-filebeat-to-redis-and-then-logstash/132718/11 "2018-06-21T09:35:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.