# Beginners ELK design doubts

**URL:** <https://discuss.elastic.co/t/beginners-elk-design-doubts/82596>\
**Category:** Logstash\
**Created:** [April 17, 2017, 3:58pm UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596 "2017-04-17T15:58:22Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![arpitwanchoo](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@arpitwanchoo](https://discuss.elastic.co/u/arpitwanchoo)\
**Post date:** [April 17, 2017, 3:58pm UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596/1 "2017-04-17T15:58:22Z")

</div>

Hi  
I am exploring the ELK framework for creating a centralized logging system across all our application(java,ruby,php) running on ec2 ubuntu servers . After going through some readings I have few doubts :

1. How Is log stash indexer scalable horizontally ? Can we simply configure it behind a load balancer and simple keep adding/removing boxes in variance with load ? Is it stateless ?

2. How to decide whether to use kafka queue or redis queue ?

3)Will I need to correct logging formats in all my applications for it to be useful or that can be done by logstash itself ? Can single logstash server handle logs with different formats from different applications ?

1. How is filebeat in comparison to logstash forwarder ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 18, 2017, 3:27am UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596/2 "2017-04-18T03:27:31Z")

</div>

1. Yes to those.
2. What's easier?
3. That's exactly what LS is for!
4. Don't use the latter, it's 100% unsupported.

---

<div class="post-metadata">

**Author:** ![arpitwanchoo](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@arpitwanchoo](https://discuss.elastic.co/u/arpitwanchoo)\
**Post date:** [April 18, 2017, 5:08am UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596/3 "2017-04-18T05:08:34Z")

</div>

Thanks Mark for quick response.

Regarding Kafka vs Redis , what factors should help in deciding ?  
Factors I would consider is :  
a) fault tolerance  
b) cost  
c) ease of maintenance/setup

As per my understanding, a) and b) would be in favor of kafka while c) in favor redis

Would be good to have other opinions as well

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 18, 2017, 8:33am UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596/4 "2017-04-18T08:33:04Z")

</div>

That's a fair summary of the two, based on my experience.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 16, 2017, 8:45am UTC](https://discuss.elastic.co/t/beginners-elk-design-doubts/82596/5 "2017-05-16T08:45:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.