# Load test log stash through file beat

**URL:** <https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 19, 2016, 7:09pm UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483 "2016-05-19T19:09:08Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kasi](https://avatars.discourse-cdn.com/v4/letter/k/e19b73/32.png) [@kasi](https://discuss.elastic.co/u/kasi)\
**Post date:** [May 19, 2016, 7:09pm UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483/1 "2016-05-19T19:09:08Z")

</div>

I would like to stress test our logstash environment, through filebeat, the idea is to generate log files  
with different sizes intervals and call filebeat programmatically to send logs to log stash, any ideas?  
How people stress test their log stash environment?

Thanks,

Kasi

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 20, 2016, 6:37am UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483/2 "2016-05-20T06:37:59Z")

</div>

If you want to do the above with Filebeat, it is probably best to keep Filebeat running on only move the files that should be read into the directories that have to be crawled at the right time. This seems easier then to "remote" trigger Filebeat.

In general for Logstash load testing, it is probably best to use the Logstash forum.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 20, 2016, 9:43am UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483/3 "2016-05-20T09:43:06Z")

</div>

While you can use filebeat, when do this what you're testing is:

```auto
disk IO (IO cache) -> filebeat -> logstash output -> network layer -> beats input -> logstash pipeline -> filters -> ...

```

You want to test logstash only, or your complete processing pipeline? For testing beats -\> logstash (in case I don't want full filebeat), I use [https://github.com/urso/generatorbeat](https://github.com/urso/generatorbeat).

Logstash itself has a [generator plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-generator.html), plus file and event stdin input plugins that can be used to remove some components.

Testing with filebeat I normally use pre-made files + a shell script deleting the old registry file and restart filebeat to ensure the complete file is send again. This is a pretty static setup. Your approach with creating files/logs dynamically you can achieve by having filebeat running in background and just write new files into the directories watched by filebeat. Alternatively you can use pipes if you configure filebeat to use the stdin prospector.

---

<div class="post-metadata">

**Author:** ![kariboe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kariboe/32/12676_2.png) [@kariboe](https://discuss.elastic.co/u/kariboe)\
**Post date:** [November 29, 2016, 9:01am UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483/4 "2016-11-29T09:01:28Z")

</div>

Hello Steffen,

I want to stress test my entire stack and found this discussion. Think generatorbeat can help me.

Node (with generatorbeat) \> HAProxy -\> 2 LS shippers -\> Redis Queue -\> 2 LS Indexers -\> ES cluster -\> Kibana

I managed to install the generatorbeat but I need some help with the configuration.  
Currently I have 2 files under the /etc folder: beat.yml and fields.yml

Where should i put my logstash output config? Can you point me to any documentation that could help me?

Kind Regards,

Stefaan

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [November 29, 2016, 2:22pm UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483/5 "2016-11-29T14:22:39Z")

</div>

normally we build configurations via `make update`. This combines (a simple `cat`) etc/beat.yml with output settings in libbeat/\_meta/config.yml. =\> update beat.yml

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:50pm UTC](https://discuss.elastic.co/t/load-test-log-stash-through-file-beat/50483/6 "2017-07-05T21:50:12Z")

</div>


