# Syslog-ng + Logstash + Splunk **URL:** \ **Category:** Logstash\ **Created:** [June 30, 2017, 10:09am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406 "2017-06-30T10:09:55Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![prash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prash/32/19702_2.png) [@prash](https://discuss.elastic.co/u/prash)\ **Post date:** [June 30, 2017, 10:09am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/1 "2017-06-30T10:09:55Z")
Hi , I am writing a Log collector for that I used "syslog-ng + Logstash + Splunk ". Along with these I want to use the Message broker or queue . Then my system will be "syslog-ng + Redis or RabitMQ+Logstash + Splunk. I gone through the web search and Logstash documents to finalize which one I have to use along with the Logstash either Redis or RabitMQ. My requirement is as below. Step 1: Syslog-ng is installed in the multiple servers(1 to n-1) . Step 2: Logstash and Splunk Forwarder is installed in the another server. Step 3: Logstash will read the log messages which is sent by syslog-ng. Step 4: Splunk forwarder will forward the logs to the Splunk server. Note: In between the step 1 and step2 I want to use the message broker or queue . So I am in confusion which one I have to use either Redis or RabitMQ. Q) Which(Redis or RabitMQ) the best feasible with the Logstash? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 2, 2017, 9:04am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/2 "2017-07-02T09:04:39Z")
You can use either. Which one you choose depends on the requirements on your solution. Given that Logstash now supports [persistent queues](https://www.elastic.co/guide/en/logstash/5.4/persistent-queues.html) you may be able to avoid a message queue. ---
**Author:** ![prash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prash/32/19702_2.png) [@prash](https://discuss.elastic.co/u/prash)\ **Post date:** [July 2, 2017, 9:05am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/3 "2017-07-02T09:05:54Z")
Thanks for your help. ---
**Author:** ![prash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prash/32/19702_2.png) [@prash](https://discuss.elastic.co/u/prash)\ **Post date:** [July 2, 2017, 9:14am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/4 "2017-07-02T09:14:26Z")
Thanks for the replay, What I understand is " I can use a Redis because Logstash now supports persistant queues", So now I am planning to use the Redis with the Logstash instead of RabbitMQ. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 2, 2017, 9:16am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/5 "2017-07-02T09:16:36Z")
You may not need to use a separate message queue at all as Logstash now can buffer events internally, which is often why Redis is introduced. If you still want to use a message queue, the choice depends on what you expect it to add to the architecture. Redis is very fast but is primarily an in-memory queue, which means messages can be lost on failure. RabbitMQ is slower but can, as far as I recall, persist data to disk and provide stronger delivery guarantees. Be sure to investigate the differences and configuration options before making a decision. ---
**Author:** ![prash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prash/32/19702_2.png) [@prash](https://discuss.elastic.co/u/prash)\ **Post date:** [July 2, 2017, 9:23am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/6 "2017-07-02T09:23:33Z")
Thanks for the help. ---
**Author:** ![fekete\_robert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fekete_robert/32/23144_2.png) [@fekete\_robert](https://discuss.elastic.co/u/fekete_robert)\ **Post date:** [July 3, 2017, 8:20am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/7 "2017-07-03T08:20:43Z")
Newer syslog-ng versions (3.8+) can also buffer messages ---
**Author:** ![prash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prash/32/19702_2.png) [@prash](https://discuss.elastic.co/u/prash)\ **Post date:** [July 3, 2017, 9:03am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/8 "2017-07-03T09:03:46Z")
Thanks , for the help. I will check the implementation . ---
**Author:** ![prash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prash/32/19702_2.png) [@prash](https://discuss.elastic.co/u/prash)\ **Post date:** [July 3, 2017, 11:18am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/9 "2017-07-03T11:18:12Z")
Hi, I found the below limitation in the Elastic documents([https://www.elastic.co/guide/en/logstash/current/persistent-queues.html#persistent-queues-limitations](https://www.elastic.co/guide/en/logstash/current/persistent-queues.html#persistent-queues-limitations)). **Limitations of Persistent Queues** edit The following are problems not solved by the persistent queue feature: ``` **Input plugins that do not use a request-response protocol cannot be protected from data loss**. For example: tcp, udp, zeromq push+pull, and many other inputs do not have a mechanism to acknowledge receipt to the sender. Plugins such as beats and http, which do have an acknowledgement capability, are well protected by this queue. ``` In our project we are using the below code as input: input{ udp { port =\> "${UDP\_PORT:514}" type =\> syslog } } So in this case, can we use the persistence queue ? My understanding is we should not use the " persistence queue". If it is wrong please correct me. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 31, 2017, 11:18am UTC](https://discuss.elastic.co/t/syslog-ng-logstash-splunk/91406/10 "2017-07-31T11:18:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.