Hardware profiles - days of data available for fast access

My team is using Elastic Cloud (Azure cloud provider) and we recently noticed that the hardware profiles have changed when creating a new deployment. Previously, we had chosen the I/O optimized profile which is no longer present. When looking at the new profiles, one confusing phrase that it uses is "ingestion use cases with 7-10 days of data available for fast access". Can anyone explain to what "available for fast access" is referring? Is that just referring to the RAM and cached results?

In our use case, all of our data is Hot data, not log/time-series data. It's basically record tracking data where any record can be updated at any time. Peak ingestion can be up to millions of records per hour (including updates and new records), peak searches could be in the 30-50 requests per second range. Data is likely in the range of multiple terabytes. We're also making significant use of aggregations across this data to calculate various metrics.

In our internal testing, CPU has been the the typical bottleneck for ingestion/search causing us to scale up and the new hardware profiles (General and CPU optimized) look to provide more CPU resources which looks attractive, but I would just like to understand the intent and meaning behind that "available for fast access" wording before we decide on a new profile.

Hi @sz_bb Thanks for using Elastic Cloud

In short "fast access" is relative depending on your specific use case.

I/O Optimized has been replaced with Storage Optimized.. the docs I linked below specify what replaced what.

Perhaps is should say "faster access relative to the other profiles"

Search Access is driven by an number of factors as you are probably aware of.

Number / Concurrent Search, Types of Searches, Complexity of Searches, Number complexity of the size of the results, Well tuned searches vs poorly tuned searches etc...

From the underlying Node / HW Profile what support these is the CPU and RAM to process the searches and Fast IO SSD AND the Ratio of CPU to RAM / Disk Storage ratio. Often CPU is the limiting factor on Highly Intensive Search Use Cases

So with respect to Azure HW Profiles the CPU Optimized provide the most CPU to RAM / Disk ratio.

You can look at those ratios here and instance configurations here

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.