Set up and maintain ES as primary database

Hello,

the chosen database for our current project is Elasticsearch as it is handy enough for JSON documents and our project involves only JSON documents (data-wise).

I'm a bit concerned because no one in the company is fluent in elasticsearch. I have questions regarding 2 things:

  • as I understand, we will have only a VM to deploy all of our modules (.jar mostly) and elasticsearch. How much RAM and CPU should we give to ES in a context where at most 60.000 requests could be sent for create/update and between [0-60.000] requests could be sent for search/get (this 60.000 number is conservative, I would expect something like a peak load of 10.000 users maximum, thus resulting in approximately 10.000 get/search requests)
  • I know snapshots can be scheduled every half hour or so to secure the data (that frequency seems ok to me, relative to the value of the data).
  • How do I know how to size my elasticsearch database ? What are the main parameters to consider ? For example when I test on my machine I see health is YELLOW and for example I don't know how to go from there to GREEN.

Sorry this post might be messy but I want to make sure elasticsearch is a suitable solution as a primary database, otherwise we need to look somewhere else.

Thanks :slight_smile:

May I suggest you look at the following resources about sizing:

https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

And https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right

For example when I test on my machine I see health is YELLOW and for example I don't know how to go from there to GREEN.

It's because you have only one node running but you have a number of replicas equal to 1 (default value). Which means that replicas can't be allocated anywhere, hence that yellow status.
Either change the index settings and set the number of replicas to 0, or start a new node, or just ignore this. This is fine for tests.

1 Like

Thanks a lot! I will dig into these resources and learn.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.