Memory usage in index

Elastic Stack Elasticsearch
1

Hello,
I am new to ELasticSearch..I created an index and populated with around 500thousand documents.
How can I check what % of my documents are in RAM and how much is stored in disk?
Thanks
Rajeev

2

Welcome!

Everything is stored on disk.

1 Like
3

Thank you :blush:

4

Hey,
Is there any way I can store in RAM to speed up response time?

5

Files on disk are cached in the operating system page cache, so having lots of RAM available will speed up queries this way. Elastic also uses off-heap memory for some data in additional to the allocated heap.

1 Like
6

What is the current response time you have?

7

@dadoonet It is around 150 to 200 ms for simple queries. Even if I run same query why the response time varies every time?

8

Will every document in index is cached?
When queried , will the results fetched from disk(secondary memory) or page cache?

9

Could you share your query and the response?

10

Curl call:
curl -X GET "localhost:9200/test/_search?q=JamesTown&pretty"

Response:

    {
      "took" : 169,
      "timed_out" : false,
      "_shards" : {
        "total" : 5,
        "successful" : 5,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : 2,
        "max_score" : 7.391895,
        "hits" : [
          {
            "_index" : "test",
            "_type" : "_doc",
            "_id" : "airport_5745",
            "_score" : 7.391895,
            "_source" : {
              "airportname" : "Jamestown Regional Airport",
              "city" : "Jamestown",
              "country" : "United States",
              "faa" : "JMS",
              "geo" : {
                "alt" : 1498.0,
                "lat" : 46.9297,
                "lon" : -98.6782
              },
              "icao" : "KJMS",
              "id" : 5745,
              "type" : "airport",
              "tz" : "America/Chicago"
            }
          },
          {
            "_index" : "test",
            "_type" : "_doc",
            "_id" : "airport_7059",
            "_score" : 7.391895,
            "_source" : {
              "airportname" : "Chautauqua County-Jamestown",
              "city" : "Jamestown",
              "country" : "United States",
              "faa" : "JHW",
              "geo" : {
                "alt" : 525.0,
                "lat" : 42.153333,
                "lon" : -79.258056
              },
              "icao" : "KJHW",
              "id" : 7059,
              "type" : "airport",
              "tz" : "America/New_York"
            }
          }
        ]
      }
    }

FYI my use case is FullTextSearch. User may search on any field.

11

If you have only 500k documents, I'd use only one shard. That could help.

12

I will try it.
Thanks

13

Can we get to know how much % of docs are in page cache if the RAM is less?

14

Hey, I tried with only one shard, It is taking almost same time sometimes more.

15

What kind of hardware do you have?

16

Using Virtual Machine with ~32 GB RAM

17

What kind of drives do you have?

18

Normal HDD, not SSD

19

That might explain then.
Is it still slow after some runs? It should be cached by the OS at some point.

20

Yes, After some runs it's better. But for some new queries it is again taking long time.
Thanks for your support.