Architecture validation

Hi,

I'm writing to you, because I need to your help / points of view about the architecture that I choose.
In fact, I just started a new project, I'm in preparation architecture stage.
Normally, I have, 4 stages: Collect data, storage, treatement data (We'll use Machine learning models also), data visualisation.

So, I will give you a vision about the architecture that we have choosen :

Industry phase ==> MQTT ==> Logstash ==> Elasticsearch ==> Kibana

My question is, how the data scientists will do their work ? I mean, because we don't have spark for example. Which framework is available to connect it with elastcisearch ? and to use by the data scientists. Because, our data scientist will use Python as program lanaguage ?

I need some clarification about the data science stage in our architecture from you the experts. Thanks a lot in advance

1 Like

What sort of things do the data scientists need to do?

we have a use case : predictive maintenance . And may be after that we'll have more use cases.

May be eland is what you are looking for
You can also use tranform to prepare your entity centrics indices
TSVB is also very usefull