Reindexing: Move mapping up a level

Elastic Stack Elasticsearch
1

Greetings!
I am trying to remap an index that inadvertently had a top level 'doc' node.
So, the documents were indexed like this :

"_source" : {
          "doc" : {
            "week_starting" : "2020-05-24",
            "origin_zip3" : "020",
            "dest_zip3" : "722", ...

when the doc node should have been excluded from the import.

Is there a way to reindex and remap values? Its about 37 million records, so something in place would be great.

Here is kind of a snippet that makes sense to me, if doable, but I can't figure out if anything like this is workable.

  POST _reindex
 {
 	"size": 1,
    "source": {
      "index": [
        "source_index_1"
      ],
      "_source": [
        "doc"
      ]
    },
    "dest": {
      "index": "dest_index"
 	 {can I do mapping here with simple object notation?:  (field1 = source.doc.field1)}
    }
  }
2

You can use a Painless script in the redinex process to remap your fields. Here is an example for a remap that i had to do to map the old host field into a new host.name field:

    {
  "source": {
    "index": "temp1"
  },
  "dest": {
    "index": "temp2"
  }
  ,"script": {
    "source": "if (ctx._source.host instanceof String) {String host=ctx._source.host;ctx._source.host=['name':host];}"
  }
}

see https://www.elastic.co/guide/en/elasticsearch/painless/7.10/painless-lang-spec.html for the specification of Painless

Hope this helps

3

Thanks so much!

Got it.

I was having trouble figuring out the mapping from one ctx object to another.

Thanks again.