ECE - Searchable object storage

Elastic Stack Elasticsearch
1

Hello,

I'm using ECE, and trying to create a script that would give the "Searchable object storage" of a Frozen Instance, that is stored in S3. I have tried to get it through the API : /api/v1/deployments/{deployment_id}. And when I check the specific instance object from the JSON :

{
                                "instance_name": "",
                                "instance_configuration": {
                                    "id": "",
                                    "name": "",
                                    "resource": "memory"
                                },
                                "service_version": "",
                                "instance_set_id": "",
                                "healthy": ,
                                "container_started": ,
                                "service_running": ,
                                "maintenance_mode": ,
                                "logical_zone": "",
                                "zone": "",
                                "allocator_id": "",
                                "memory": {
                                    "instance_capacity": 2375,
                                    "memory_pressure": 26,
                                    "native_memory_pressure": 82
                                },
                                "disk": {
                                    "disk_space_available": 71250,
                                    "disk_space_used": 64393,
                                    "storage_multiplier": 30.0
                                },
                                "service_roles": [
                                    "data"
                                ],
                                "node_roles": [
                                    "data_frozen"
                                ],
                                "instance_overrides": {}
                            }

I can't figure out why in the UI it shows :

Searchable object storage 405.18 GB.

As I'm not finding any formula to end up with this number. Maybe I'm missing something when trying to recreate the formula?

2

Hi @Catalin1 Welcome to the community.

Can you share which UI screenshot you are talking about?

Keep in mind for Frozen, there are 2 parts to the Storage

The local disk storage on the frozen node that acts as a cache (Typically a couple of TBs)
and
The Searchable Snapshots stored in the S3 get queried as needed. (Perhaps many TBs)

It is not clear exactly what you are trying to calculate.

And since you are ECE (Enteprise Subscription) and have a support subscription, I would suggest perhaps opening a support ticket for this.

If you are talking about this... It is all calculated by ratios...

In this Example
(Which is Elastic Cloud and your ECE may be configured differently)

Screenshot 2024-12-11 at 11.52.18 AM
Screenshot 2024-12-11 at 11.52.18 AM2296×840 87.9 KB

The Node is 8GB RAM (4 GB JVM)
The RAM to Local Disk Ratio is 1 : 95 = 8GB x 90 = 720GB
The RAM to S3 / Searchable is 1:1600 = 8 * 1600 = 12,500 GB = 12.5TB

With yours...

"memory": {
                                    "instance_capacity": 2375,
                                    "memory_pressure": 26,
                                    "native_memory_pressure": 82
                                },
                                "disk": {
                                    "disk_space_available": 71250,
                                    "disk_space_used": 64393,
                                    "storage_multiplier": 30.0
                                },

You will need to consult with your Admin, who created the instance templates for how they are set up ... to get that RAM to S3 ratio ... (above is the 30 is RAM to Local Disk Ratio... math works out) and, again, perhaps open a ticket...

If I take that 405.18GB and divide it by the RAM

405.18GB *1024 / 2375 = ~175... so that may be your Ratio for Frozen searchable snapshots... which seems a little low but may be what your team is using

3

Hello @stephenb , and thank you for your response.

image
image1295×275 12.3 KB

This is the UI that I was speaking about, so basically what I'm trying to do, is to get that number (405.18GB) into my script, I thought it's possible through the JSON object of the instance, but I guess it is being calculated in a different way, so I would love to find out how, so I can also use it for all the frozen instances.