# Sizing for udp/tcp (syslog type) **URL:** \ **Category:** Logstash\ **Created:** [March 16, 2016, 8:43am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528 "2016-03-16T08:43:53Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![miko370](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@miko370](https://discuss.elastic.co/u/miko370)\ **Post date:** [March 16, 2016, 8:43am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/1 "2016-03-16T08:43:53Z")
Hi. Does anyone have any sizing knowledge and experience using tcp/udp input plugin for syslog messages? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 16, 2016, 8:46am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/2 "2016-03-16T08:46:27Z")
Sizing knowledge? I think you need to be a bit more specific. ---
**Author:** ![miko370](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@miko370](https://discuss.elastic.co/u/miko370)\ **Post date:** [March 16, 2016, 11:36am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/3 "2016-03-16T11:36:49Z")
ok. How many GB per day we can handle per given hardware? What can impact that? Number of CPU's? Number of RegEx's (for example I am going to have dozens of RegEx's that will identify for each message, the vendor and maybe the product e.g Cisco ASA etc) How do I know if I am starting to have data loss since LS cannot handle a given throughput? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 16, 2016, 11:49am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/4 "2016-03-16T11:49:40Z")
When collecting data from UDP and TCP it often makes sense to have the Logstash instance(s) that do the collection do as little processing as possible in order to optimise throughput and just have it write to a message queue. The introduction of a message queue allows you to buffer data and handle peaks better while at the same time decouple collection from processing. You can therefore have a number of Logstash instances read from the message queue in parallel and do the CPU intensive processing without affecting the collection. This layer can be scaled out without affecting the collection layer. ---
**Author:** ![miko370](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@miko370](https://discuss.elastic.co/u/miko370)\ **Post date:** [March 16, 2016, 11:55am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/5 "2016-03-16T11:55:56Z")
Thanks Chris for your reply. Does anyone know - per given hardware - what is the troughput limit in case of TCP/UDP with a minimal processing? How can I monitor a potential data loss? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 16, 2016, 8:20pm UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/6 "2016-03-16T20:20:42Z")
What is the given hardware? ---
**Author:** ![miko370](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@miko370](https://discuss.elastic.co/u/miko370)\ **Post date:** [March 16, 2016, 9:40pm UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/7 "2016-03-16T21:40:08Z")
Lets start with a basic config: 4 CPU 16 GB memory 1 GbE network card. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 17, 2016, 4:06am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/8 "2016-03-17T04:06:23Z")
Are you talking purely about parsing the data, or storing it too? ---
**Author:** ![miko370](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@miko370](https://discuss.elastic.co/u/miko370)\ **Post date:** [March 17, 2016, 11:42am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/9 "2016-03-17T11:42:27Z")
I am more worried about the input side (Not having data loss cause of a high troughput) and also on the parsing part since I have dozens of regEx for each event. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 17, 2016, 10:14pm UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/10 "2016-03-17T22:14:55Z")
Monitoring data loss is hard, because how do you know it's gone? Given you have a bunch of regexes, you really will need to test this yourself. ---
**Author:** ![miko370](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@miko370](https://discuss.elastic.co/u/miko370)\ **Post date:** [March 23, 2016, 7:47am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/11 "2016-03-23T07:47:28Z")
Hi. I have seen this slide in the "latest in LogStash" but it does not mentioned the time period. We are probably not talking about EPS, Does anyone have other benchmarks for the 2.x version? ![](https://us1.discourse-cdn.com/elastic/original/2X/8/80775bd931d20eb5cc6bb1297d55be8a0eb9f364.png) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:05am UTC](https://discuss.elastic.co/t/sizing-for-udp-tcp-syslog-type/44528/12 "2017-07-06T05:05:46Z")