Rename nested fields with spaces using Ingest and Reindex

UPDATE: I'm more stumped than before because I was able to update some fields with spaces, and not others. As a concrete example, intervention_time.operation.H OURS gets updated to intervention_time.operation.hours just fine, but N. of Instance doesn't get updated to N. of Instances. What could be happening here?

Hi, everyone!

I have a very messy index that I want to organize to have the same fields across all documents. For instance, I have documents with a field called "Date" and others with "1 Date", because the fields are being created automatically using a script that parses through text files.

I wrote a script that sees what fields can be merged and creates an Ingest node with some renames, and then the documents are re-indexed with Reindex and the Ingest node created. This works fine for simple fields or fields without spaces, but when I try something like

        "rename": {
            "field": "exampleField.bad name",
            "target_field": "exampleField.goodName"

I receive the following error:

      "failures" : [
          "index" : "newIndex",
          "type" : "_doc",
          "id" : "123",
          "cause" : {
            "type" : "illegal_argument_exception",
            "reason" : "field [exampleField.bad name] doesn't exist"
          "status" : 400

I noticed that this happens with nested fields containing spaces (nested fields without spaces and fields with spaces but without the dot separator work fine). I tried escaping the spaces, using 'field name' and \"field name\", and considered using a script, but nothing worked. I'm completely stumped. I know that putting spaces on fields isn't ideal, but I already have this index and I want to organize it. What can I do?

P.S.: Since I don't know if the proposed solution will work for my other problem, I might as well mention it. Similarly, I have some nested fields in arrays, such as

    {"myField": [
      {"field 1": "value1",
       "field 2": "value2"},
      {"field 1": "value3",
       "field 2": "value4"}

I was renaming these fields with a foreach, and it was working fine, but the same problem occurred: the Ingest can't access fields with spaces. In this case, I was inserting "_ingest._value.field 1" in the field option, and was receiving the same error.

After some more research, I solved my problem by using the dot expander processor with the problematic fields. That solved everything!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.