Using ElasticSearch as a Microservice

This is a design question. We have messages pushed to Kafka and Kafka can pipe it to Elastic using Kafka connect. This is what I am considering for message search. Kafka can send these messages but it also forwards it to the Elasticsearch service. Messages are multi-lingual.

We also need API access to these messages. This is not search. A screen for a particular user will have messages for that user. Traditionally a message micro service is used for this purpose. But I though I could unify the search service and this API.

Can this be done using Elasticsearch ? I can avoid a separate Micro service for this as all the messages are in Elastic now. I plan to explore how Elasticsearch can be hosted in Kubernetes as all our services are there.

The only reason for have a Spring Data Elastic Micro Service is to check some business rules. Has the message been viewed already ? How to expire the messages ? Has the user been blacklisted ?


The retrieval of messages for a user would be akin to a search, just with a filter for that user right? So Elasticsearch can do this.

Yes. But the layer before Elasticsearch has to take care of checking blacklisted customers and other security restrictions. I assume the Elasticsearch cluster in Kubernetes is better if accessed directly instead of a Spring Data Elastic Micro Service. Any architectural pattern guidance is documented in books etc. ?

Elasticsearch has inbuilt, free security features. It doesn't do access blacklisting as such, but you could implement a list of those in an index that you query first with your code, then anything that passes can run another query to get the info it needs.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.