# Big data: Log analysis with ELK and Spark **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [April 16, 2018, 9:01am UTC](https://discuss.elastic.co/t/big-data-log-analysis-with-elk-and-spark/128142 "2018-04-16T09:01:57Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Malek\_Soltani](https://avatars.discourse-cdn.com/v4/letter/m/7c8e57/32.png) [@Malek\_Soltani](https://discuss.elastic.co/u/Malek_Soltani)\ **Post date:** [April 16, 2018, 9:01am UTC](https://discuss.elastic.co/t/big-data-log-analysis-with-elk-and-spark/128142/1 "2018-04-16T09:01:57Z")
Hi, I'm new to ELK stack and Big data. Now, i'm trying to build a one node Log Analytics tool based on ELK+Kafka+Spark Machine Learning. I'm trying to implement the lambda architecture with the following layers: **Batch** : Spark **Speed** : Logstash+ES **Serving** : Kibana I want to check if this is the right implementation or do you have a better architecture. If you have a useful Tutorial, i would be thankful. ---
**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Post date:** [April 16, 2018, 8:13pm UTC](https://discuss.elastic.co/t/big-data-log-analysis-with-elk-and-spark/128142/2 "2018-04-16T20:13:31Z")
This looks like a perfectly reasonable approach. Please note that there are a few differences between how the Hadoop connector works with how Logstash approaches things. These aren't documented anywhere since they are separate projects, but it's something that is worth being aware of. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 14, 2018, 8:14pm UTC](https://discuss.elastic.co/t/big-data-log-analysis-with-elk-and-spark/128142/3 "2018-05-14T20:14:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.