# Can I write grok expression to enrich log files in FileBeat before sending to Logstash / elastic search

**URL:** <https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568>\
**Category:** Beats\
**Created:** [May 20, 2016, 2:05pm UTC](https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568 "2016-05-20T14:05:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rohit\_Shrivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohit_shrivastava/32/9850_2.png) [@Rohit\_Shrivastava](https://discuss.elastic.co/u/Rohit_Shrivastava)\
**Post date:** [May 20, 2016, 2:05pm UTC](https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568/1 "2016-05-20T14:05:04Z")

</div>

Hi

My use case is to ship log files from various applications to elastic search so that I can view them from kibana.

I wanted to know can filebeat be configured for grok expression so that application team can manage their log parsing at their end and central logging system / deployment is unaffected. If it can be then the need to Logstash is questionable.

-Rohit

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 20, 2016, 2:10pm UTC](https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568/2 "2016-05-20T14:10:29Z")

</div>

You can add fields with static values but grok-style extraction of fields isn't supported in Filebeat.

---

<div class="post-metadata">

**Author:** ![Rohit\_Shrivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohit_shrivastava/32/9850_2.png) [@Rohit\_Shrivastava](https://discuss.elastic.co/u/Rohit_Shrivastava)\
**Post date:** [May 20, 2016, 2:32pm UTC](https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568/3 "2016-05-20T14:32:18Z")

</div>

Thanks @magnusbaeck

Can you tell me one more thing, when logstash and filebeat output plugin can be configured to use max\_retries to -1, then there isn't need for queue. can you explain me why queue is needed when load increases more than elasticsearch capacity? I am referring [https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html](https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html)

Thanks  
Rohit

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 21, 2016, 4:04pm UTC](https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568/4 "2016-05-21T16:04:28Z")

</div>

Having a queue isn't strictly needed, but it helps get the data off of the shippers as quickly as possible and is a good way of scaling the Logstash instances that feeds ES (useful if Logstash rather than ES is the bottleneck of the system).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:51pm UTC](https://discuss.elastic.co/t/can-i-write-grok-expression-to-enrich-log-files-in-filebeat-before-sending-to-logstash-elastic-search/50568/5 "2017-07-05T21:51:41Z")

</div>