_reindex has suddenly stopped working inexplicably

[mrodent (Mike)]

I've been using the _reindex endpoint for some time now, about 2 months, without any problems.

I'm running ES 8.6.2 on port 9500. The OS is W10.

For some reason, as of this morning, any attempts to reindex are just not ending. The indices in question are about 20 MB or so: I've let the operation run on for as much as 3 minutes. I've tried running the command from the program (Python) but also from Insomnia.

I've also checked that the index I'm trying to reindex definitely does exist. I've also restarted the ES server. Other ES server commands appear to be working fine.

I've tried rebooting.

I notice that when I cancel the operation after many seconds, and then go GET _cat/indices I now see the new index which I was attempting to create with status "red" and apparently no documents (or stated size).

Any suggestions about what might be causing this?

[jessgarson]

Thanks for reaching out @mrodent. I have a few follow up questions here:

• I am correct in understanding that you have't received any errors and it just seems to hang?
• Are all your attempts with data of the same size?
• Do you have a code sample you can send over?

This blog post may also be helpful too.

[DavidTurner]

@mrodent

I now see the new index which I was attempting to create with status "red"

These docs describe how to troubleshoot this situation.

[mrodent (Mike)]

Thanks both. Turned out the disk was too full and had exceeded the "high watermark". Had never heard of this before!