Elasticsearch deployment

[porscheme (porsche)]

We wanted to build Elasticsearch on cloud Azure.
Source data size: 50TB
Replicas count: 3
Shard count: 1000

For doing this, we are considering below two VM SKUs.

1. L32sv2 is very processing we wanted to know SLAs for this VM SKU
2. Any experience with using this SKU is really helpful

DS5v2 – Intel Xeon processor
RAM: 56 GB
vCPU(s): 16
Managed Disk: Premium SSD

L32s v2: AMD EPYC 7551 processor
RAM: 256 GB
vCPU(s): 32
Local NVMe premium SSD disks attached to VMs (ephemeral)

[warkolm (Mark Walkom)]

Welcome to our community!

Are you saying you will have 2 nodes, with 3000 shards?

[porscheme (porsche)]

No, we are thinking of using 10 nodes with 1000 shards.
We want each shard to be 75GB.

But the real question here is...
By default, I was assuming we’d have to use managed disks (using DS5v2 VM SKU) as we couldn’t rely on the local temporary storage. That could get blown up at any time. Attached managed storage seems to be the way most people talk about supporting this scenario online.

However, the Lsv2-series has both temporary storage and NVMe disk. The spec sheet for it talks about it being ideal for “Big Data, SQL, and NoSQL databases.”. Which seems to fit our problem space

Should use DS5v2 (with managed disks) or L32s (NVMe storage)