Reindexing docs to new index using alias name with "is_write_index" clause (without reindex API)

Elastic Stack Elasticsearch
1

Hi

Elastic 6.8.7

Let's say I have single index a. Then there is for example 1 document with id: 1.
(in real life there would be hundred thousands of documents of course)

Then at some time, let's say we need to reindex the whole index (source is not enabled, so I can't use reindex API).
So what I do, I create new index b (with same mappings as a), and create alias, so that new index b is write index.

This is how I create alias.

    POST /_aliases
    {
      "actions": [
        {
          "add": {
            "index": "a",
            "alias": "a_index",
            "is_write_index": false
          }
        },
        {
          "add": {
            "index": "b",
            "alias": "a_index",
            "is_write_index": true
          }
        }
      ]
    }

Then I will forward all reindex requests to this alias name "a_index" and my document "id: 1" is indexed into write index "b".

Now if I search this doc "id: 1", the result gives me 2, since this doc is present in both indexes.

    GET a_index/_search?q=_id:1
    ...
    "hits" : {
    "total" : 2,
    .....

Now, my question is, is it not possible, that after the doc with "id: 1" has bee reindexed into index "b", then this doc is automatically deleted from index "a" ?

Raul

2

@raulk89 in short, no, it isn't really possible to automatically delete the doc with id:1 from index a after being indexed into index b using just Elasticsearch.

3

Thanks. Makes sense.
I was asking because last year I did mass indexing (few million documents), with custom made indexing program. And there I did the same - index alias, one is write index and indexing program used alias name.
And there I saw, that as docs in new index are increasing, the docs from old one, were decreasing.
So at first I thought it was some cool elastic feature.
Few days back I did some more mass indexing with custom made program (not the same program), then there happened no such thing. After the indexing, all the docs were present on both indexes.

But now I realized, that the first indexing program, it had that feature built into, that first it will check if the doc exists and if so, then delete it and then index this doc.

Raul

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.