# Logstash forwarder viable?

**URL:** <https://discuss.elastic.co/t/logstash-forwarder-viable/122406>\
**Category:** Logstash\
**Created:** [March 4, 2018, 2:12pm UTC](https://discuss.elastic.co/t/logstash-forwarder-viable/122406 "2018-03-04T14:12:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![amitavmohanty01](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amitavmohanty01/32/58017_2.png) [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\
**Post date:** [March 4, 2018, 2:12pm UTC](https://discuss.elastic.co/t/logstash-forwarder-viable/122406/1 "2018-03-04T14:12:57Z")

</div>

I have a scenario where I need to push logs from edge nodes to central ES. I have to process multiline log messages and the logs messages need to use dissect/grok plugin behaviour to extract parts of it into fields like date, process etc. Filebeat solves the purpose of pushing and multiline parsing but does not offer grok. The other option is to use LS forwarder. However, I am not sure whether the plan is to maintain it or it will be deprecated in favour of filebeat. Also, there are multiple files that are to be pushed. I am not sure if logstash with multiline codec will work properly with multiple file sources.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 4, 2018, 2:22pm UTC](https://discuss.elastic.co/t/logstash-forwarder-viable/122406/2 "2018-03-04T14:22:47Z")

</div>

[logstash-forwarder](https://github.com/elastic/logstash-forwarder) was deprecated a long time ago and replaced by Filebeat, which has a lot more capabilities and is continuously being developed and maintained. I would recommend you use Filebeat to collect logs and perform any multiline processing before sending the data on to Logastash for further processing and enrichment.

---

<div class="post-metadata">

**Author:** ![amitavmohanty01](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amitavmohanty01/32/58017_2.png) [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\
**Post date:** [March 4, 2018, 2:59pm UTC](https://discuss.elastic.co/t/logstash-forwarder-viable/122406/3 "2018-03-04T14:59:06Z")

</div>

@Christian_Dahlqvist thanks for the prompt response. I did not expect a response on a Sunday.

1. Can I not write a custom processor for filebeats to do the job?
2. If I have to do centralized grokking, is it wise to have logstash and ES on the same server and have multiple ES nodes talking to each other for replication? For example, on 10 machines, I can have LS and ES. Each machine's LS only talks to its ES. All ES can talk to each other to form a cluster and offer redundancy.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 4, 2018, 3:18pm UTC](https://discuss.elastic.co/t/logstash-forwarder-viable/122406/4 "2018-03-04T15:18:21Z")

</div>

> [@amitavmohanty01](#):
>
> Can I not write a custom processor for filebeats to do the job?

Filebeat is designed to use as little resources as possible, which is why CPU intensive processing like e.g. grok is not supported. You can naturally fork Filebeat and add whatever logic you like, but you would end up with a custom solution you would have to maintain going forward.

> [@amitavmohanty01](#):
>
> If I have to do centralized grokking, is it wise to have logstash and ES on the same server and have multiple ES nodes talking to each other for replication?

It is always recommended to deploy Logstash and Elasticsearch on dedicated hosts as it reduces resource contention and makes it easier to troubleshoot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 1, 2018, 3:18pm UTC](https://discuss.elastic.co/t/logstash-forwarder-viable/122406/5 "2018-04-01T15:18:39Z")

</div>

Closed on April 1, 2018, 3:18pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
