Reindex api and date field

[guilherme_maranhao]

Dear all,

I've used _reindex and all the date fields were changed to "type": "text". In the source index, they are all declared as follow:

my_date_field": { "type": "date", "format": "yyyy-MM-dd HH:mm:ss" }

The reindex json was:

{
  "source": {
    "index": "my_source_index"
  },
  "dest": {
    "index": "my_dest_index"
  },
  "script": {
    "inline": "ctx._source.remove('another_field')"
  }
}

My elastic version is 6.2.1.

Does anybody know why that happened?

Thanks a lot,

Guilherme

[cdekker]

Make sure you set up your (empty) destination index including mappings first. Else it is left for ES to guess the mapping for each field which it seems to do incorrectly in your case.

[guilherme_maranhao]

Thank you, @cdekker, but I couldn't find this option in the _reindex documentation page (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html).

However, according to the "Important note", if I want to duplicate the source index settings, it's supposed to run the index creation prior to the _reindex execution, isn't it?

Thanks,

Guilherme

[guilherme_maranhao]

Actually, the mappings and settings replication should be default for the _reindex. Why would you want to reindex (copy) an index without the source settings???

[cdekker]

It is literally in the first paragraph of the documentation you linked

Reindex does not attempt to set up the destination index. It does not copy the settings of the source index. You should set up the destination index prior to running a _reindex action, including setting up mappings, shard counts, replicas, etc.

Reindex gives you complete freedom over the destination index, which is why it doesn't set up the destination index for you. For example, we use it when a field mapping changes. In those cases you actually want to manually set up a different destination index mapping before copying over all data.

Other 'cloning' operations such as _shrink and _split do set up the destination index for you. But in those cases it is also clearly documented.

EDIT: For our use case we have also disabled dynamic index creation and dynamic mapping (adding fields automatically). It saved us a lot of confusing situations by throwing an error instead of magically creating unknown fields

[guilherme_maranhao]

Thank you, @cdekker

@cdekker

Other 'cloning' operations such as _shrink and _split do set up the destination index for you. But in those cases it is also clearly documented.

I didn't know those operations. Maybe I'll try them in the next time.

@cdekker

Reindex gives you complete freedom over the destination index, which is why it doesn't set up the destination index for you. For example, we use it when a field mapping changes. In those cases you actually want to manually set up a different destination index mapping before copying over all data.

It makes sense.

Regards!

Guilherme