# Architecture validation

**URL:** <https://discuss.elastic.co/t/architecture-validation/248940>\
**Category:** Elasticsearch\
**Created:** [September 17, 2020, 8:13am UTC](https://discuss.elastic.co/t/architecture-validation/248940 "2020-09-17T08:13:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vero](https://avatars.discourse-cdn.com/v4/letter/v/e19b73/32.png) [@vero](https://discuss.elastic.co/u/vero)\
**Post date:** [September 17, 2020, 8:13am UTC](https://discuss.elastic.co/t/architecture-validation/248940/1 "2020-09-17T08:13:26Z")

</div>

Hi,

I'm writing to you, because I need to your help / points of view about the architecture that I choose.  
In fact, I just started a new project, I'm in preparation architecture stage.  
Normally, I have, 4 stages: Collect data, storage, treatement data (We'll use Machine learning models also), data visualisation.

So, I will give you a vision about the architecture that we have choosen :

Industry phase ==\> MQTT ==\> Logstash ==\> Elasticsearch ==\> Kibana

My question is, how the data scientists will do their work ? I mean, because we don't have spark for example. Which framework is available to connect it with elastcisearch ? and to use by the data scientists. Because, our data scientist will use Python as program lanaguage ?

I need some clarification about the data science stage in our architecture from you the experts. Thanks a lot in advance

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 17, 2020, 10:22pm UTC](https://discuss.elastic.co/t/architecture-validation/248940/2 "2020-09-17T22:22:21Z")

</div>

What sort of things do the data scientists need to do?

---

<div class="post-metadata">

**Author:** ![vero](https://avatars.discourse-cdn.com/v4/letter/v/e19b73/32.png) [@vero](https://discuss.elastic.co/u/vero)\
**Post date:** [September 18, 2020, 1:31pm UTC](https://discuss.elastic.co/t/architecture-validation/248940/3 "2020-09-18T13:31:38Z")

</div>

we have a use case : predictive maintenance . And may be after that we'll have more use cases.

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [September 18, 2020, 2:51pm UTC](https://discuss.elastic.co/t/architecture-validation/248940/4 "2020-09-18T14:51:47Z")

</div>

May be [eland](https://github.com/elastic/eland) is what you are looking for  
You can also use [tranform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) to prepare your entity centrics indices  
[TSVB](https://www.elastic.co/guide/en/kibana/current/TSVB.html) is also very usefull

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 16, 2020, 2:52pm UTC](https://discuss.elastic.co/t/architecture-validation/248940/5 "2020-10-16T14:52:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.