# Logstash Limits

**URL:** https://discuss.elastic.co/t/logstash-limits/331353
**Category:** Logstash
**Created:** [April 30, 2023, 2:10pm UTC](https://discuss.elastic.co/t/logstash-limits/331353 "2023-04-30T14:10:25Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![shushuu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shushuu/32/118100_2.png) [@shushuu](https://discuss.elastic.co/u/shushuu)
#### Post date: [April 30, 2023, 2:10pm UTC](https://discuss.elastic.co/t/logstash-limits/331353/1 "2023-04-30T14:10:25Z")

</div>

Hi,  
We would like to use Logstash to receive log messages from multiple services (`nxlog`) and send them further to Elastic.  
i.e. using this architecture - but with `nxlog` instead of `Beats`:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7e5e4061a8b3462f3c0832232c0679bc479bfc3f.png)

What are the limits of a single Logstash instance if used as described above?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2023, 3:50pm UTC](https://discuss.elastic.co/t/logstash-limits/331353/2 "2023-04-30T15:50:32Z")

</div>

> [@shushuu](#):
>
> What are the limits of a single Logstash instance if used as described above?

It is impossible to say. The throughput you can achieve in a single logstash instance may vary by multiple orders of magnitude (i.e. 1000x or more) depending on the hardware configuration and the amount and type of processing you are doing.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [April 30, 2023, 7:36pm UTC](https://discuss.elastic.co/t/logstash-limits/331353/3 "2023-04-30T19:36:41Z")

</div>

The best option is to test. Avoid grok or any other slower plugin or algorithm.

---

<div class="post-metadata">

### Author: ![X11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/x11/32/98759_2.png) [@X11](https://discuss.elastic.co/u/X11)
#### Post date: [May 13, 2023, 1:06am UTC](https://discuss.elastic.co/t/logstash-limits/331353/4 "2023-05-13T01:06:29Z")

</div>

Echoing what @rios said, test, I would recommend testing on a small/medium/large node because you won’t know what the max throughput your logstash configuration can handle is without thorough testing. Heck you can’t even speculate what the throughput might be without testing.

I’ve seen things cap out at 7K eps, but then I’ve also seen 100K eps, but there are many, many variables to consider, so I would say keep your test cases simple and see what kind of results you get. A very very oversimplified example

### small node 2G - JVM - default settings for batch size and workers etc

1. 1 x upstream input source
2. 2 x upstream input source
3. 4 x upstream input source

### medium node 4G - JVM - default settings for batch size and workers etc

1. 1 x upstream input source
2. 2 x upstream input source
3. 4 x upstream input source

Etc etc… the challenging part will be generating load from nxlog depending on your input, if it’s something simple like tcp/udp, you may not even need to actually use nxlog if you know the structure of your logs. For example when testing something like udp I’ve had a lot of success just using 1+ logstash nodes with a generator input shipping tcp/udp logs (stressor) to a second logstash (the system being tested)

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [May 13, 2023, 12:32pm UTC](https://discuss.elastic.co/t/logstash-limits/331353/5 "2023-05-13T12:32:13Z")

</div>

And you can use [LS statistic](https://www.elastic.co/guide/en/logstash/current/node-stats-api.html) and [Kibana LS monitoring](https://www.elastic.co/guide/en/kibana/current/logstash-page.html) in order to get info per plugin.

`curl -XGET 'localhost:9600/_node/stats/pipelines?pretty'`

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 10, 2023, 12:32pm UTC](https://discuss.elastic.co/t/logstash-limits/331353/6 "2023-06-10T12:32:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
