Currently, I am wondering whether the whole ELK Stack can be mapped to the Lambda Architecture or not, without regarding other technologies? My idea was that Elasticsearch can be settled to the Serving Layer, whereas Logstash can be used in the Speed & Batch Layer, but at this point I am not clearly sure, because the Speed layer normally uses technologies like Apache Spark or Storm. And Kibana as the tool for the serving layer.
I am also interested in this question. If someone has an idea..
You would be well advised to include a queueing technology (eg. Kafka) to your ELK architecture. You should be sure to separate out the concerns of reception, buffering and enrichment, which could be satisfied by logstash|kafka|logstash respectively. You could (perhaps should) have multiple topics for the reception layer to write to, which could then be viewed as a topic(s) for logstash enrichment, but you might then have other topics which lead to something like a Flink deployment, which I daresay would handle inter-event state better than logstash; that Flink deployment could then write back to the (a) topic that leads to logstash.
Elasticsearch can certainly be a good place to sink the results of real-time and batch jobs.
Kibana as a serving layer.... I think that would depend heavily on how fast you needed the results to be generated. In general, I think the performance bounds of Kibana would be too unconstrained, but with a suitably fixed/constrained dashboard, it might be usable. Perhaps more likely you could have something that just queries Elasticsearch (although Kibana does a lot for you)
I am (reading between the lines) seeing more capabilities in things like Beats to write directly to more different queues. I think you could comfortably say that ELK can be part of a Lambda architecture, but it would be stretch to say that by itself it would a suitable for a full-on entreprise-level data-streaming architecture.
(Disclaimer: I know perhaps less than 1 paragraph of a Lambda Architecture)