Check for field's existence in script processor ingest node

[meetdave2611997 (Meet Dave)]

Hello,

I am using ingest node for processing documents and I am using script processor for processing documents. Now my requirement is to check whether the field exists and based on existence process goes further.

I am using below code but it's not working.

if (ctx.field1) {
     // process 1
}else{
    // process 2
}

field1 is not present in all document. some of the documents may not have that field.
How can i check whether field exists in current doc or not?

Thanks,
Meet

[rjernst (Ryan Ernst)]

Currently ctx is a Map, so you can use containsKey:

if (ctx.containsKey('field1')) {
     // process 1
}else{
    // process 2
}

[meetdave2611997 (Meet Dave)]

@rjernst

I tried the above solution but didn't work. Here is my script

POST _scripts/threat_preference-script
{
  "script":{
    "lang":"painless",
    "source":"""
    if(ctx.containsKey('threat.id')){
      ctx.threat_preference = "With Threat"
    }else{
      ctx.threat_preference = "Without Threat"
    }
    """
  }
}

Here is my ingest pipeline

POST _ingest/pipeline/test
{
"description": "Pipeline to process scripted fields.",
"processors": [
    {
    "script": {
        "lang": "painless",
        "on_failure": [
        {
            "set": {
            "field": "threat_preference-error",
            "value": "{{ _ingest.on_failure_message }}"
            }
        }
        ],
        "id": "threat_preference-script"
    }
    }
]
}

After ingesting some document, I see each document contains Without threat value in threat_preference field even though threat.id field exist

[rjernst (Ryan Ernst)]

Unfortunately in this case, ctx is a map of maps, so using the full path to the field will not work. You will need to check both maps, eg:

if (ctx.containsKey('threat')) {
  if (ctx['threat'].containsKey('id')) {
    // has threat.id field
  }
}

[meetdave2611997 (Meet Dave)]

@rjernst Thank you very much. That worked!