# High volume apache httpd 2.4 access log processing

**URL:** <https://discuss.elastic.co/t/high-volume-apache-httpd-2-4-access-log-processing/113519>\
**Category:** Beats\
**Created:** [December 29, 2017, 2:33am UTC](https://discuss.elastic.co/t/high-volume-apache-httpd-2-4-access-log-processing/113519 "2017-12-29T02:33:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![davidbernard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidbernard/32/22472_2.png) [@davidbernard](https://discuss.elastic.co/u/davidbernard)\
**Post date:** [December 29, 2017, 2:33am UTC](https://discuss.elastic.co/t/high-volume-apache-httpd-2-4-access-log-processing/113519/1 "2017-12-29T02:33:56Z")

</div>

I am wanting to move from "scribe" to ElasticStack for processing apache httpd 2.4 access logs.

I have identified the following approaches:

Option 1

CustomLog with pipe configuration for FileBeat processing stdin.

Option 2

CustomLog with pipe configuration to "rotatelog". FileBeat processing on disk logs

Option 3

Standard logging to file FileBeat processing on disk logs.

Typically we are processing 4,000 transactions per second.

Does anyone have experience with these options and transaction rates they could share?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [January 3, 2018, 11:15pm UTC](https://discuss.elastic.co/t/high-volume-apache-httpd-2-4-access-log-processing/113519/2 "2018-01-03T23:15:55Z")

</div>

I would recommend you to go with option 3 as this makes it possible to use disk also as a buffer in case of peaks, when needing to upgrade FB etc.

4k should definitively work with FB but I would recommend to check out the throughput on your setup as it not only depends on FB but also size of your ES cluster, network etc.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 19, 2018, 2:34am UTC](https://discuss.elastic.co/t/high-volume-apache-httpd-2-4-access-log-processing/113519/3 "2018-01-19T02:34:05Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.