Hesitation between three Elasticsearch-based Architectures

Hello everyone,
Let me present to you my Use Case:
I am trying to develop an algorithm that chooses the best communication channel(SMS, RCS, Messenger, Whatsapp, E-mail) and the best moment to communicate with the clients. There are two phases for this algorithm:

  • The prediction phase in which it will, based on usage and segmentation criteria, pick the right channel.

  • The adaptation phase, in which it will receive everyday logs (CDRs) of the client's actions and usages and try to adapt.(Machine Learning Process)

I am hesitating between three Elasticsearch-based architectures, can you please give me some advice, and if you have any documentation or Architecture to suggest please feel free.

1st: Logstash(Producer) -> Kafka -> Logstash(Consumer) -> Elasticsearch -> Eland(Machine Learning)

2nd: Logstash -> Spark -> Elasticsearch -> Eland(Machine Learning)

3rd: Logstash -> Kafka -> Spark (Do Machine Learning) -> Elasticsearch

Thank you in advance.

I'm not familiar with Spark and Eland. However I am pretty sure for anyone to help you they will need to understand your (planned) usage of the elastic stack.

What usecase(s) do you plan to carry out?

Thank you for your comment, I have just edited my post.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.