Rename nested fields with spaces using Ingest and Reindex

Elastic Stack Elasticsearch
1

UPDATE: I'm more stumped than before because I was able to update some fields with spaces, and not others. As a concrete example, intervention_time.operation.H OURS gets updated to intervention_time.operation.hours just fine, but N. of Instance doesn't get updated to N. of Instances. What could be happening here?

Hi, everyone!

I have a very messy index that I want to organize to have the same fields across all documents. For instance, I have documents with a field called "Date" and others with "1 Date", because the fields are being created automatically using a script that parses through text files.

I wrote a script that sees what fields can be merged and creates an Ingest node with some renames, and then the documents are re-indexed with Reindex and the Ingest node created. This works fine for simple fields or fields without spaces, but when I try something like

    {
        "rename": {
            "field": "exampleField.bad name",
            "target_field": "exampleField.goodName"
        }
    }

I receive the following error:

      "failures" : [
        {
          "index" : "newIndex",
          "type" : "_doc",
          "id" : "123",
          "cause" : {
            "type" : "illegal_argument_exception",
            "reason" : "field [exampleField.bad name] doesn't exist"
          },
          "status" : 400
        }

I noticed that this happens with nested fields containing spaces (nested fields without spaces and fields with spaces but without the dot separator work fine). I tried escaping the spaces, using 'field name' and \"field name\", and considered using a script, but nothing worked. I'm completely stumped. I know that putting spaces on fields isn't ideal, but I already have this index and I want to organize it. What can I do?

P.S.: Since I don't know if the proposed solution will work for my other problem, I might as well mention it. Similarly, I have some nested fields in arrays, such as

    {"myField": [
      {"field 1": "value1",
       "field 2": "value2"},
      {"field 1": "value3",
       "field 2": "value4"}
    ]}

I was renaming these fields with a foreach, and it was working fine, but the same problem occurred: the Ingest can't access fields with spaces. In this case, I was inserting "_ingest._value.field 1" in the field option, and was receiving the same error.

2

After some more research, I solved my problem by using the dot expander processor with the problematic fields. That solved everything!