# How do you merge the syslogs from network device into message queue when logstash scales out?

**URL:** <https://discuss.elastic.co/t/how-do-you-merge-the-syslogs-from-network-device-into-message-queue-when-logstash-scales-out/66032>\
**Category:** Logstash\
**Created:** [November 14, 2016, 10:56pm UTC](https://discuss.elastic.co/t/how-do-you-merge-the-syslogs-from-network-device-into-message-queue-when-logstash-scales-out/66032 "2016-11-14T22:56:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [November 14, 2016, 10:56pm UTC](https://discuss.elastic.co/t/how-do-you-merge-the-syslogs-from-network-device-into-message-queue-when-logstash-scales-out/66032/1 "2016-11-14T22:56:54Z")

</div>

I'd like to ask question about the theory of scale out process for logstash.

I am planning to have 2300 evt/sec events into logstash. This number includes both from filebeat and syslog of network device. But number of logs will be planned to increase more.

I understand that your options for performance increase for logstash will be increasing input queue (beats or syslog) or logstash itself. However, these option makes you change the configuration of filebeat side (increase worker and end up needing more cpu cores) and the network device side for changing desitination. I believe that idea of Message Queue (redis) can be used which can simplify the configuration of filebeat and network device

My question is how do you usually send logs from network device which was directly sending to logstash into message queue?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 15, 2016, 6:49am UTC](https://discuss.elastic.co/t/how-do-you-merge-the-syslogs-from-network-device-into-message-queue-when-logstash-scales-out/66032/2 "2016-11-15T06:49:13Z")

</div>

Network devices (I'm assuming you're talking about e.g. switches and routers) typically send events via syslog and don't support message queues. What you can do is have one or more syslog receivers that don't do anything with the events except push them onto a queue, and then you can have one or more Logstash instances that pull from this queue and process the events.

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [November 17, 2016, 10:24am UTC](https://discuss.elastic.co/t/how-do-you-merge-the-syslogs-from-network-device-into-message-queue-when-logstash-scales-out/66032/3 "2016-11-17T10:24:01Z")

</div>

@magnusbaeck

Ok. Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 15, 2016, 10:24am UTC](https://discuss.elastic.co/t/how-do-you-merge-the-syslogs-from-network-device-into-message-queue-when-logstash-scales-out/66032/4 "2016-12-15T10:24:05Z")

</div>

Closed on December 15, 2016, 10:24am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.