Hi all,
I've been running ES 5 for some time (say a month) now and all has been well, but last night all my indices were deleted. I see this in the log (the second line):
---- 8< ----
[2017-01-11T21:23:04,357][WARN ][o.e.d.z.p.PublishClusterStateAction] [master] timed out waiting for all nodes to process published state ...
[2017-01-11T21:23:08,473][WARN ][o.e.c.s.ClusterService ] [master] cluster state update task [delete-index ... ] took [34.1s] above the warn threshold of 30s
---- 8< ----
Thing is, i didn't do it on purpose, and i can't think anyone else did it maliciously. What could i have done wrong?
It's an 8 node cluster:
---- 8< ----
$ curl -X GET localhost:9200/_cat/nodes
10.6.204.38 55 99 21 1.24 1.83 1.99 di - data-ip-10-6-204-38
10.6.204.47 69 97 99 12.72 11.79 11.40 di - data-ip-10-6-204-47
10.6.204.6 42 99 99 10.36 10.62 10.63 di - data-ip-10-6-204-6
10.6.204.46 63 94 99 9.89 10.96 10.77 di - data-ip-10-6-204-46
10.6.204.43 57 99 22 1.98 2.28 2.37 di - data-ip-10-6-204-43
10.6.204.33 24 99 99 15.38 12.45 11.71 di - data-ip-10-6-204-33
10.6.204.58 25 81 76 6.46 6.76 6.86 mdi * master
10.6.204.44 65 99 22 3.02 2.35 2.11 di - data-ip-10-6-204-44
---- 8< ----
Version info:
---- 8< ----
[master] version[5.0.1], pid[27750], build[080bb47/2016-11-11T22:08:49.812Z], OS[Linux/3.13.0-74-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_91/25.91-b14]
---- 8< ----
Port 9200 is open to the internet. Might it be malicious?
Thanks!