Machine Configuration for elasticsearch as a database

I am trying to move the SQL database into elasticsearch. Successfully flattened the data from SQL into elasticsearch.
These is the details of the index which was created from SQL to elastic Doc'sCount : 49201833, Storage Size : 13.7gb.
This will be increasing. Our backend is in python. Whenever we try to pull the data through scroll API all my machine RAM is getting used and our flask is getting restarted. Below is the machine which we are using. What should be our ideal machine configuration for that data size?

H/W path Device Class Description

                            system     t3a.2xlarge

/0 bus Motherboard
/0/0 memory 64KiB BIOS
/0/4 processor AMD EPYC 7571
/0/1 memory 31GiB System memory
/0/100 bridge 440FX - 82441FX PMC [Natoma]
/0/100/1 bridge 82371SB PIIX3 ISA [Natoma/Triton II]
/0/100/1.3 generic 82371AB/EB/MB PIIX4 ACPI
/0/100/3 display, Inc.
/0/100/4 storage, Inc.
/0/100/4/0 /dev/nvme0 storage Amazon Elastic Block Store
/0/100/4/0/1 /dev/nvme0n1 disk 429GB NVMe namespace
/0/100/4/0/1/1 /dev/nvme0n1p1 volume 399GiB EXT4 volume
/0/100/5 eth0 network Elastic Network Adapter (ENA)

It's not clear what system specs you have in this cluster sorry.

Sorry for the delay in response. System is AWS EC2 instance of type t3a.2xlarge.
32 GB RAM, 8 core CPU, 400GB SSD harddisk.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.