Losing documents while reindexing 4 million documents

Elastic Stack Elasticsearch
1

I am re-indexing 4 million documents. When I create a new index without any mappings (default mappings) it works fine. Means every document get indexed into the new one from source index.

But when I create new index with my new mappings where I added a char filter and copying some fields to a common field. I am loosing around 500000 documents. Please help

2

have you checked the tasks API to retrieve the status of the task for further information, or checked the reindex response? Just adding a char filter should not result in something like this, but maybe something else in your mapping has changed.

3

Thanks Alexander for your response. Yes I checked the reposne it do not give any error. I have total 4 million plus documents. But everytime it stops at the same count 3921999. And after that task just exit.
Thing is when I map index with default mappings every document get indexed. But I changed added porter stem, added a field to which I am copying an another field, added synonyms. Its really strange for me. I am checking re-indexing status with task api. Could not see error. It takes around 20 min to index but in the last I do not a response what happened.

4

I figured it out. Actually I was using keyword field for my description field which may contain some no alphabatic chars. I removed and it got fine.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.