Node sync fails and cluster goes to "red"

Hello David,

Thank you for replying.

I do not have snapshots for the cluster. It unfortunately started with the intention to take snapshots which failed : https://discuss.elastic.co/t/backup-snapshot-strategy-for-a-two-data-node-cluster/

Here is the dmseg log from the node where the data is available (secondary node) . It in fact has new a SSD I got 2 weeks back :worried:

[ 9002.852451] sd 32:0:1:0: [sdb] tag#36 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9002.852462] sd 32:0:1:0: [sdb] tag#36 Sense Key : Hardware Error [current]
[ 9002.852475] sd 32:0:1:0: [sdb] tag#36 Add. Sense: Internal target failure
[ 9002.852478] sd 32:0:1:0: [sdb] tag#36 CDB: Read(10) 28 00 21 fa 09 00 00 01 00 00
[ 9002.852480] blk_update_request: critical target error, dev sdb, sector 570034432 op 0x0:(READ) flags 0x80700 phys_seg 9 prio class 0
[ 9003.028072] sd 32:0:1:0: [sdb] tag#37 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9003.028086] sd 32:0:1:0: [sdb] tag#37 Sense Key : Hardware Error [current]
[ 9003.028089] sd 32:0:1:0: [sdb] tag#37 Add. Sense: Internal target failure
[ 9003.028091] sd 32:0:1:0: [sdb] tag#37 CDB: Read(10) 28 00 21 fa 0a 00 00 01 00 00
[ 9003.028094] blk_update_request: critical target error, dev sdb, sector 570034688 op 0x0:(READ) flags 0x80700 phys_seg 9 prio class 0
[ 9003.294889] sd 32:0:1:0: [sdb] tag#42 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9003.294898] sd 32:0:1:0: [sdb] tag#42 Sense Key : Hardware Error [current]
[ 9003.294901] sd 32:0:1:0: [sdb] tag#42 Add. Sense: Internal target failure
[ 9003.294903] sd 32:0:1:0: [sdb] tag#42 CDB: Read(10) 28 00 21 fa 09 98 00 00 08 00
[ 9003.294906] blk_update_request: critical target error, dev sdb, sector 570034584 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 9003.467311] sd 32:0:1:0: [sdb] tag#43 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9003.467324] sd 32:0:1:0: [sdb] tag#43 Sense Key : Hardware Error [current]
[ 9003.467326] sd 32:0:1:0: [sdb] tag#43 Add. Sense: Internal target failure
[ 9003.467329] sd 32:0:1:0: [sdb] tag#43 CDB: Read(10) 28 00 21 fa 09 98 00 00 08 00
[ 9003.467332] blk_update_request: critical target error, dev sdb, sector 570034584 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0

Will I be at least able to recovery the cluster with my research data?

Just to add information here, I store OS&Elasticsearch on a single disk and all of the data on a separate physical disk