500mb memory used with no data added?

Elastic Stack Kibana
1

I have setup Elasticsearch and Kibana locally in a docker container with one node using docker compose.

I enabled self-monitoring in Kibana and see that the memory usage for Elastic is around 500mb out of 1gb and there is 15k documents, but I have not added any data yet. The amount of documents is still soaring and now at 20k - 60mb of data!

Kibana also has a memory usage around 500mb.

Is this all normal/defaults?? The DB im migrating that we have used for years only totals to 5mb of JSON, so 60mb of "overhead" is crazy. I also go to the indices page and the only index that has any documents is the es-monitoring, and thats about half of the "Document Count" at the top. Where are these documents coming from?

2

If you enabled self-monitoring then you added data, monitoring can be pretty noisy and generate a lot of documents as it will query the metrics every 10 seconds per default.

Also, Elasticsearch and Kibana have some internal indices that will also have some documents.

This seems normal, Elasticsearch is not light-weight, per default it will try to use half of the memory of the system and it needs memory to run well.

What is the DB you are migrating from and what is the reason for the migration? Elasticsearch is not a Database per se, it is a search engine and a document storage, so you cannot simple compare with other databases because they are internally different.

Also, depending on what you want to do, you will not be able to do in Elasticsearch, for example, it is pretty common to see people starting to use Elasticsearch and then after some time find out that it does not have JOIN capabilities.

As mentioned, there are some internal indices that will not show up per default.

3

Currently we have a SQL database but want to switch to MongoDB which is why I mentioned migration.

The documents will be pretty nested with lots of text data. For example:

{
  "rootCauses": [
    {
      "description": "...",
      "actions": [
        {
          "summary": "..."
        }
      ]
    },
    // ...
  ]
}

We want to be able to search for any text in these and account for typos. (and possibly vector search). Is this a good use case for Elastic and can it index that type of data?