Set up and maintain ES as primary database

Elastic Stack Elasticsearch
1

Hello,

the chosen database for our current project is Elasticsearch as it is handy enough for JSON documents and our project involves only JSON documents (data-wise).

I'm a bit concerned because no one in the company is fluent in elasticsearch. I have questions regarding 2 things:

  • as I understand, we will have only a VM to deploy all of our modules (.jar mostly) and elasticsearch. How much RAM and CPU should we give to ES in a context where at most 60.000 requests could be sent for create/update and between [0-60.000] requests could be sent for search/get (this 60.000 number is conservative, I would expect something like a peak load of 10.000 users maximum, thus resulting in approximately 10.000 get/search requests)
  • I know snapshots can be scheduled every half hour or so to secure the data (that frequency seems ok to me, relative to the value of the data).
  • How do I know how to size my elasticsearch database ? What are the main parameters to consider ? For example when I test on my machine I see health is YELLOW and for example I don't know how to go from there to GREEN.

Sorry this post might be messy but I want to make sure elasticsearch is a suitable solution as a primary database, otherwise we need to look somewhere else.

Thanks :slight_smile:

2

May I suggest you look at the following resources about sizing:

And https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right

For example when I test on my machine I see health is YELLOW and for example I don't know how to go from there to GREEN.

It's because you have only one node running but you have a number of replicas equal to 1 (default value). Which means that replicas can't be allocated anywhere, hence that yellow status.
Either change the index settings and set the number of replicas to 0, or start a new node, or just ignore this. This is fine for tests.

1 Like
3

Thanks a lot! I will dig into these resources and learn.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.