Hello everyone,
I am fairly new to this whole ES thing, but I am getting through the trial by fire on my first misconfiguration. I ended up doing a snapshots from my multi node cluster and restored it to a single node to fix some other stuff.
Anyway the issue I am having is when I do the restore from the snapshot I can take a look and see the index and shard present, green, and containing gigabytes of data; however, I am seeing that I am missing around half the data for specific windows of time. This is a SIEM implementation and I have at least 3 separate data holes. The first two are around 2 days long, and another 1 days long. I will note the I am not seeing any errors on the restore.
Here is the restore curl:
Restore curl:
curl -k -X POST -u <user>:password "https://localhost:9200/_snapshot/my_backup10/my_snapshot_0830/_restore?wait_for_completion=true&pretty" -H 'Content-Type: application/json' -d' { "indices": "<index1>,<index2>", "include_global_state": false, "rename_pattern": "so-(.+)", "rename_replacement": "so-$1-r10pp", "include_aliases": false } '
I have to list each and every index that I would like to restore, if I leave it blank it only restores the hidden index starting with a period (.). I am changing the name so I do not have an collisions.
Original Server:
Restored Server:
Curling the shards: curl -k -XGET -u user:pass "https://localhost:9200/_cat/shards?" | grep 8.19
Any idea what I could be doing wrong that basically 90% of all the data was restored, but I had these 'hole' every once and a while? Any help would be very much appreciated.