# ELK Stack and the Lambda Architecture

**URL:** <https://discuss.elastic.co/t/elk-stack-and-the-lambda-architecture/41149>\
**Category:** Elastic Community and Ecosystem\
**Created:** [February 7, 2016, 8:46pm UTC](https://discuss.elastic.co/t/elk-stack-and-the-lambda-architecture/41149 "2016-02-07T20:46:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Omer\_Uludag](https://avatars.discourse-cdn.com/v4/letter/o/b9e5f3/32.png) [@Omer\_Uludag](https://discuss.elastic.co/u/Omer_Uludag)\
**Post date:** [February 7, 2016, 8:46pm UTC](https://discuss.elastic.co/t/elk-stack-and-the-lambda-architecture/41149/1 "2016-02-07T20:46:53Z")

</div>

Currently, I am wondering whether the whole ELK Stack can be mapped to the Lambda Architecture or not, without regarding other technologies? My idea was that Elasticsearch can be settled to the Serving Layer, whereas Logstash can be used in the Speed & Batch Layer, but at this point I am not clearly sure, because the Speed layer normally uses technologies like Apache Spark or Storm. And Kibana as the tool for the serving layer.

---

<div class="post-metadata">

**Author:** ![sen](https://avatars.discourse-cdn.com/v4/letter/s/cc9497/32.png) [@sen](https://discuss.elastic.co/u/sen)\
**Post date:** [July 11, 2016, 4:06pm UTC](https://discuss.elastic.co/t/elk-stack-and-the-lambda-architecture/41149/2 "2016-07-11T16:06:57Z")

</div>

I am also interested in this question. If someone has an idea..

---

<div class="post-metadata">

**Author:** ![cknz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cknz/32/9640_2.png) [@cknz](https://discuss.elastic.co/u/cknz)\
**Post date:** [April 13, 2017, 4:21am UTC](https://discuss.elastic.co/t/elk-stack-and-the-lambda-architecture/41149/3 "2017-04-13T04:21:31Z")

</div>

You would be well advised to include a queueing technology (eg. Kafka) to your ELK architecture. You should be sure to separate out the concerns of reception, buffering and enrichment, which could be satisfied by logstash|kafka|logstash respectively. You could (perhaps should) have multiple topics for the reception layer to write to, which could then be viewed as a topic(s) for logstash enrichment, but you might then have other topics which lead to something like a Flink deployment, which I daresay would handle inter-event state better than logstash; that Flink deployment could then write back to the (a) topic that leads to logstash.

Elasticsearch can certainly be a good place to sink the results of real-time and batch jobs.

Kibana as a serving layer.... I think that would depend heavily on how fast you needed the results to be generated. In general, I think the performance bounds of Kibana would be too unconstrained, but with a suitably fixed/constrained dashboard, it might be usable. Perhaps more likely you _could_ have something that just queries Elasticsearch (although Kibana does a _lot_ for you)

I am (reading between the lines) seeing more capabilities in things like Beats to write directly to more different queues. I think you could comfortably say that ELK can be part of a Lambda architecture, but it would be stretch to say that by itself it would a suitable for a full-on entreprise-level data-streaming architecture.

(Disclaimer: I know perhaps less than 1 paragraph of a Lambda Architecture)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:43pm UTC](https://discuss.elastic.co/t/elk-stack-and-the-lambda-architecture/41149/4 "2017-07-06T13:43:23Z")

</div>


