Sorry for the silly question, but it is possible to delete Elasticsearch documents forever?
I'm in the migration process 1.7 -> 2.3 and when I run the migration plugin all my indices were listed with fields with dots. After analysis, I identified these fields with dots were generated by a plugin installed on my Graylog server and that has no more use for me.
Instead of bothering to rename these fields, I decided to go for something more practical and delete all documents with these fields.
So I ran:
for i in `curl -s -XGET localhost:9200/_all/ | jq -r "keys[]"`; do
echo -n "Excluindo JMX do índice $i: "
curl -XDELETE localhost:9200/$i/_query?q=label:JMX
echo
done
then...
#curl 'localhost:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
green open graylog2_2 3 1 19925062 75050 15.7gb 7.8gb
green open graylog2_5 3 1 20001004 6792 18.3gb 9.1gb
green open graylog2_21 3 1 19952265 48034 17.3gb 8.6gb
green open graylog2_34 3 1 19968203 32270 17.9gb 8.9gb
green open graylog2_26 3 1 19950723 50344 17.5gb 8.7gb
green open graylog2_6 3 1 19976215 24772 18.3gb 9.1gb
green open graylog2_29 3 1 19962504 37908 17.9gb 8.9gb
green open graylog2_27 3 1 19953287 47122 17.5gb 8.7gb
green open graylog2_10 3 1 19928390 71802 17.4gb 8.7gb
green open graylog2_25 3 1 19950258 51176 17.5gb 8.7gb
green open graylog2_0 3 1 19980897 19551 15gb 7.5gb
green open graylog2_33 3 1 19965352 35020 17.8gb 8.9gb
green open graylog2_35 3 1 18677635 40233 17.1gb 8.4gb
green open graylog2_9 3 1 19958295 42350 17.2gb 8.6gb
green open graylog2_24 3 1 19956467 44080 17.4gb 8.7gb
green open graylog2_18 3 1 19956737 45109 17.4gb 8.7gb
green open graylog2_12 3 1 19955303 44811 17.4gb 8.7gb
green open graylog2_8 3 1 19956070 44784 17.9gb 8.9gb
green open graylog2_15 3 1 19926718 74654 17.4gb 8.7gb
green open graylog2_17 3 1 19957205 42997 17.3gb 8.6gb
green open graylog2_4 3 1 19988129 13418 18gb 9gb
green open graylog2_7 3 1 19955716 45062 17.8gb 8.9gb
green open graylog2_11 3 1 19945574 56475 18gb 9gb
green open graylog2_22 3 1 19965180 38857 17.4gb 8.7gb
green open graylog2_31 3 1 19957684 43390 17.6gb 8.8gb
green open graylog2_28 3 1 19960169 41356 17.3gb 8.6gb
green open graylog2_16 3 1 19953562 46613 17.4gb 8.7gb
green open graylog2_13 3 1 19946695 54502 17.5gb 8.7gb
green open graylog2_32 3 1 19958744 41386 17.7gb 8.8gb
green open graylog2_19 3 1 19958161 43828 17.5gb 8.7gb
green open graylog2_1 3 1 4529877 4007 3.5gb 1.7gb
green open graylog2_20 3 1 19957822 43082 17.4gb 8.7gb
green open graylog2_3 3 1 4458723 7190 3.5gb 1.7gb
green open graylog2_14 3 1 19955648 45826 17.4gb 8.7gb
green open graylog2_23 3 1 19957960 43160 17.3gb 8.6gb
green open graylog2_30 3 1 19960653 40325 17.8gb 8.9gb
... and finally:
#curl -XPOST localhost:9200/_optimize?only_expunge_deletes=true
When I do a search on Graylog, I can not find these documents anymore. Great! But when I run the migration plugin again, it keeps warning that I own fields with dots and are the same fields.
Where did I go wrong?
