Reindexing due to index incompatibility gone wrong, can't find task result

Elastic Stack Elasticsearch
, ,
1

Hello!

Just for context, my mission is to “migrate” a cluster from ES 7.7.1 to OS 2.15 via snapshot restore. The main issue I encountered is the index version incompatibility, because several indices were created in ES version 6.x.x.
Correct me if I’m wrong, but the only way to resolve this issue is to reindex such indices, right?
Anyway I managed to mess up the reindexing process somehow. TL;DR seems like reindexing of an index I chose (note: the index is constantly getting written into) failed at the end, but I cannot find the task result inside .tasks index. I was able to check the result when the reindex task was in progress, but the next day when I thought reindexing finished (by optimistically looking at the number of documents between two indices), I forgot to look at the task result (a document inside .tasks).

The reindex command:

curl -XPOST -H 'Content-Type: application/json' 'http://localhost:9200/_reindex?wait_for_completion=false' -d '{
   "source": {
    "index": "index-2019-1"
   },
   "dest": {
      "index": "index-2019-2"
   }
  }'

The command output was task ID, which I could search for with curl -XGET "localhost:9200/_tasks/WVDz_yHpRia9DXkQ3alLZg:114775?pretty" when reindex was in-progress, but a few (3-4) days after there was no such document/task inside this index anymore. I’m trying to reproduce the issue, but I can’t - all newly completed tasks are visible as documents inside the .tasks index.

Has anyone encountered the same/similar issue with ES before?

2

Welcome!

As we don't support this fork here, I'd recommend that you ask on their forums.

1 Like
3

Thx @dadoonet , but the migration part ES->OS was just for context. The reindexing I've done was local, ES only. Which means, the issue with .tasks document "disappearance" is potential ES issue only.

Basically, my questions is: any idea what to check for .tasks, what options should I look at, is there anything additional to be done for this index? For example, IDK if settings for .tasks index were changed in the past, so I'm pasting them:

/home/ubuntu# curl 'localhost:9200/.tasks/_settings?pretty'
{
  ".tasks" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "auto_expand_replicas" : "0-1",
        "provided_name" : ".tasks",
        "creation_date" : "1729161928080",
        "priority" : "2147483647",
        "number_of_replicas" : "1",
        "uuid" : "vcDwnx6uRWa6iqu7AW9Jgg",
        "version" : {
          "created" : "7070199"
        }
      }
    }
  }
}

If by "fork" you meant something else, then my bad, please explain further.

4

I don't remember the internals but may be at some point the document is removed?