# Logstash input/ouput limits per/sec **URL:** \ **Category:** Logstash\ **Created:** [December 5, 2017, 2:17am UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239 "2017-12-05T02:17:05Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Post date:** [December 5, 2017, 2:17am UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239/1 "2017-12-05T02:17:05Z")
hi everyone. could someone point out the logstash limit, plz? let me say my scenario first. i use cloudwatch\_logs input \> do 50-line grok match filter \> elasticsearch output base on my script, i would like to know few facts: 1. cloudwatch\_logs input streaming limit per/sec ? 2. elasticsearch output streaming limit per/sec ? 3. if many log events come in same sec, this may cause performance issue in logstash and kibana searching? may cause lost logs? 4. using 50-line grok match filter cause slow the logstash performance? I'm running logstash v5.6.3. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 5, 2017, 6:38am UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239/2 "2017-12-05T06:38:12Z")
> cloudwatch\_logs input streaming limit per/sec ? That depends on several factors, including the performance of the host, filters used, etc. > elasticsearch output streaming limit per/sec ? See above. > if many log events come in same sec, this may cause performance issue in logstash and kibana searching? may cause lost logs? If you enable the persistent queue you shouldn't lose any events. Even without the persistent queue you'll be fine if you only have inputs that handle backpressure well. The file input, for example, will just stop reading from the input file and in that sense has its own queue system while the udp input needs to deal with whatever gets sent to it. > using 50-line grok match filter cause slow the logstash performance? Large grok filters will of course be detrimental to performance but whether that actually matters for you is another story. ---
**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Post date:** [December 7, 2017, 12:55pm UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239/3 "2017-12-07T12:55:32Z")
Thank you magnusbaeck. In case of using large grok filter, what should i do not to be detrimental to performance? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 7, 2017, 1:06pm UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239/4 "2017-12-07T13:06:19Z")
[This blog post](https://www.elastic.co/blog/do-you-grok-grok) provides some very good guidelines on how to best use grok. In addition to this I would add that it helps being aware of other types of filters, so you do not try to use grok to parse content where better and more efficient options exist, e.g. lists of key-value pairs and JSON content. ---
**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Post date:** [December 8, 2017, 4:48am UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239/5 "2017-12-08T04:48:55Z")
> [@Christian\_Dahlqvist](#): > > ists of key-value pairs Thanks for your help. Your post is good to help me to take care of regex pattern and regex engine behavior. kv filter is useful for me as well. 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 5, 2018, 4:49am UTC](https://discuss.elastic.co/t/logstash-input-ouput-limits-per-sec/110239/6 "2018-01-05T04:49:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.