Reindexing the data from an existing index to new index with Completion fields

[Vaibhav_Sinha]

Hey Team,

I need some help reindexing the data from an existing Index (keyword/string-based fields) to a new index with fields of type Completion.

I have created the new index to be used for Completion Suggesters with field mapping as such:

 PUT product_completion_suggestions
{
  "settings": {
    "analysis": {
      "analyzer": {
        "suggestion_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "shingle"
          ]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "title": {
        "type": "completion",
        "analyzer": "suggestion_analyzer"
      },
      "id": {
        "type": "integer"
      },
      "brand": {
        "type": "completion",
        "analyzer": "suggestion_analyzer"
      },
      "category": {
        "type": "completion",
        "analyzer": "suggestion_analyzer"
      },
      "genericproductname": {
        "type": "completion",
        "analyzer": "suggestion_analyzer"
      },
      "variety": {
        "type": "completion",
        "analyzer": "suggestion_analyzer"
      }
    }
  }
}

After creating the index, I am looking to reindex the data from an existing index into this new index via the reindex API as such:

POST _reindex?wait_for_completion=true
{
  "source": {
    "index": "product-v2",
    "_source": [
       "stockcode",
       "genericproductname",
       "brand",
       "variety"
       ],
    "query": {
      "bool": {
        "must": [
          {
            "exists": {
              "field": "displayname"
            }
          },
          {
            "exists": {
              "field": "genericproductname"
            }
          },
          {
            "exists": {
              "field": "brand"
            }
          }
        ]
      }
    }
  },
  "dest": {
    "index": "product_v2_completion_suggestions"
  },
  "script": {
    "source": """def concat = ' '; concat = concat + (ctx._source.brand != null ? ctx._source.brand : ''); concat = concat + ' '; concat = concat + (ctx._source.genericproductname != null ? ctx._source.genericproductname : (ctx._source.variety != null ? ctx._source.variety : '')); ctx._source.title.input = new def[]{concat.trim()};"""
  }
}

Note: 'title' is a dynamically calculated field, and the field's value is a concatenation of three fields, i.e. brand, genericproductname and variety. Also that I am mapping 'keyword' type fields to Completion types in the ReIndex API call above as mentioned before.

But I keep getting compile time error:

script_stack" : [
      "ctx._source.title.input = new def[]{concat.trim()};",
      "                 ^---- HERE"
    ],
    "script" : "def concat = ' '; concat = concat + (ctx._source.brand != null ? ctx._source.brand : ''); concat = concat + ' '; concat = concat + (ctx._source.genericproductname != null ? ctx._source.genericproductname : (ctx._source.variety != null ? ctx._source.variety : '')); ctx._source.title.input = new def[]{concat.trim()};",
    "lang" : "painless",
    "position" : {
      "offset" : 282,
      "start" : 265,
      "end" : 316
    },
    "caused_by" : {
      "type" : "null_pointer_exception",
      "reason" : "Cannot invoke \"Object.getClass()\" because \"callArgs[0]\" is null"
    }
  },
  "status" : 400

My intention is to index the data into new Completion fields as such:

"title": {
 "input": "some_text",
"weight": 10
}

via the reindex script, but I cannot achieve it clearly, and some how my script above is incorrect. I need to have a default weight assigned to the fields as they are being reindexed. I couldn't find any documentation regarding mapping data from existing fields into new completion fields while capturing the input and weight fields during reindexing.

Any help or assistance here will be much appreciated.

Thanks.