Hardware that can last longer and IOPS needed


(Charlie) #1

Hello.
I am planning to build local cluster with following assumptions and hardware which I already have.

Worker nodes:
Number of worker nodes: 2
RAM: 64 GB DDR4 physical per node, 32 GB per heap.
CPU: AMD Ryzen™ Threadripper 1900X (on both nodes)
Max data size stored on the elastic cluster: 3TB
Document count: 5,000,000,000
HDDs: Per node 4x2TB HGST spinning drives with RAID10

Master node / ingestion node:
1 master node, 8GB RAM DDR4, i3-7100U CPU @ 2.40GHz, Samsung SSD 850 256 GB

Shard will be 20 GB with one replica.

loading the data ---> MASTER ---> SWITCH --> Two worker nodes
The connection between them would be 1Gbps.

The first load will perform approximately 700 GB of data, later every day ingestion will be 800 MB.
Querying the data will be only by the Kibana with the refresh interval of 1 minute. Occasionally API search.

If I will load the data over the master that will do the preprocessing of bulk requests, do I have the need to remove the HDDs from RAID10 configuration and replace it with RAID0 NVME drives / SSDs?
Will the SSD IOPS speed help to ingest data faster and better than on the worker node?

Or should I go straight to buying set of 2x 1TB M2 drives per node or 2x1TB SSDs? If yes, what drive you could recommend that I will not have to throw away after 4 months or something?

Thanks for your feedback in advance.


(Christian Dahlqvist) #2

Having a single master node is a bad idea. Instead also make the data nodes master eligible so you have a total of 3 master eligible nodes. Remember to set minimum_master_nodes to 2 according to these guidelines. Then send all traffic, indexing and querying, directly to the data nodes as these are larger and more powerful. Loading all data over the smaller master node risks making it a bottleneck and if it was the only master eligible node it could also cause instability.


(Charlie) #3

Thanks, that confirms my future setup.


(ddorian43) #4

Are you using ECC RAM ?


(Charlie) #5

Yes, it does support ECC.

$dmidecode --type 7

dmidecode 3.0

Getting SMBIOS data from sysfs.
SMBIOS 3.0 present.

Handle 0x002A, DMI type 7, 19 bytes
Cache Information
Socket Designation: L1 - Cache
Configuration: Enabled, Not Socketed, Level 1
Operational Mode: Write Back
Location: Internal
Installed Size: 768 kB
Maximum Size: 768 kB
Supported SRAM Types:
Pipeline Burst
Installed SRAM Type: Pipeline Burst
Speed: 1 ns
Error Correction Type: Multi-bit ECC
System Type: Unified
Associativity: 8-way Set-associative

Handle 0x002B, DMI type 7, 19 bytes
Cache Information
Socket Designation: L2 - Cache
Configuration: Enabled, Not Socketed, Level 2
Operational Mode: Write Back
Location: Internal
Installed Size: 4096 kB
Maximum Size: 4096 kB
Supported SRAM Types:
Pipeline Burst
Installed SRAM Type: Pipeline Burst
Speed: 1 ns
Error Correction Type: Multi-bit ECC
System Type: Unified
Associativity: 8-way Set-associative

Handle 0x002C, DMI type 7, 19 bytes
Cache Information
Socket Designation: L3 - Cache
Configuration: Enabled, Not Socketed, Level 3
Operational Mode: Write Back
Location: Internal
Installed Size: 16384 kB
Maximum Size: 16384 kB
Supported SRAM Types:
Pipeline Burst
Installed SRAM Type: Pipeline Burst
Speed: 1 ns
Error Correction Type: Multi-bit ECC
System Type: Unified
Associativity: 32-way Set-associative

$dmidecode --type memory

Handle 0x0028, DMI type 16, 23 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: None
Maximum Capacity: 512 GB
Error Information Handle: 0x0027
Number Of Devices: 8

Handle 0x002F, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x002E
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: Unknown
Set: None
Locator: DIMM 0
Bank Locator: P0 CHANNEL A
Type: Unknown
Type Detail: Unknown
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: Unknown
Asset Tag: Not Specified
Part Number: Unknown
Rank: Unknown
Configured Clock Speed: Unknown
Minimum Voltage: Unknown
Maximum Voltage: Unknown
Configured Voltage: Unknown

Handle 0x0031, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x0030
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: Unknown
Set: None
Locator: DIMM 1
Bank Locator: P0 CHANNEL A
Type: Unknown
Type Detail: Unknown
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: Unknown
Asset Tag: Not Specified
Part Number: Unknown
Rank: Unknown
Configured Clock Speed: Unknown
Minimum Voltage: Unknown
Maximum Voltage: Unknown
Configured Voltage: Unknown

Handle 0x0033, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x0032
Total Width: 64 bits
Data Width: 64 bits
Size: 16384 MB
Form Factor: DIMM
Set: None
Locator: DIMM 0
Bank Locator: P0 CHANNEL B
Type: DDR4
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: 00000000
Asset Tag: Not Specified
Part Number: CMV16GX4M1A2133C15
Rank: 2
Configured Clock Speed: 1067 MHz
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V

Handle 0x0036, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x0035
Total Width: 64 bits
Data Width: 64 bits
Size: 16384 MB
Form Factor: DIMM
Set: None
Locator: DIMM 1
Bank Locator: P0 CHANNEL B
Type: DDR4
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: 00000000
Asset Tag: Not Specified
Part Number: CMV16GX4M1A2133C15
Rank: 2
Configured Clock Speed: 1067 MHz
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V

Handle 0x0039, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x0038
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: Unknown
Set: None
Locator: DIMM 0
Bank Locator: P0 CHANNEL C
Type: Unknown
Type Detail: Unknown
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: Unknown
Asset Tag: Not Specified
Part Number: Unknown
Rank: Unknown
Configured Clock Speed: Unknown
Minimum Voltage: Unknown
Maximum Voltage: Unknown
Configured Voltage: Unknown

Handle 0x003B, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x003A
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: Unknown
Set: None
Locator: DIMM 1
Bank Locator: P0 CHANNEL C
Type: Unknown
Type Detail: Unknown
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: Unknown
Asset Tag: Not Specified
Part Number: Unknown
Rank: Unknown
Configured Clock Speed: Unknown
Minimum Voltage: Unknown
Maximum Voltage: Unknown
Configured Voltage: Unknown

Handle 0x003D, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x003C
Total Width: 64 bits
Data Width: 64 bits
Size: 16384 MB
Form Factor: DIMM
Set: None
Locator: DIMM 0
Bank Locator: P0 CHANNEL D
Type: DDR4
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: 00000000
Asset Tag: Not Specified
Part Number: CMV16GX4M1A2133C15
Rank: 2
Configured Clock Speed: 1067 MHz
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V

Handle 0x0040, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0028
Error Information Handle: 0x003F
Total Width: 64 bits
Data Width: 64 bits
Size: 16384 MB
Form Factor: DIMM
Set: None
Locator: DIMM 1
Bank Locator: P0 CHANNEL D
Type: DDR4
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 2133 MHz
Manufacturer: Unknown
Serial Number: 00000000
Asset Tag: Not Specified
Part Number: CMV16GX4M1A2133C15
Rank: 2
Configured Clock Speed: 1067 MHz
Minimum Voltage: 1.2 V
Maximum Voltage: 1.2 V
Configured Voltage: 1.2 V


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.