Using the Elastic Stack as database for big data

Elastic Stack Elasticsearch
1

Hi Guys,

I wanted to know if it is a good idea to use the elastic stack as database for big data. The data is used to do data science stuff later.

Further the database should scale and enable a lot of read and write access.

What are the limits of the elastic stack?

T

2

That's a very broad question, but Elasticsearch can scale to billions of records while maintaining sub-second response times.

3

And keep in mind that Elasticstack is great at ingesting data and VERY bad at updating it, so that's a primary limitation for any data set that needs anything more than trivial updates.

But as @warkolm mentions, the rest depends a lot on the data, how it would arrive, be queried, etc. You do see more 'big data' projects on Elasticsearch, especially with Machine Learning, scalability, etc. but also non-trivial to manage at scale (well, that's true of every large system).

4

It's not bad, it does it perfectly fine.
It is relatively expensive though.

5

Updates are generally fine as long as they are not frequent.