Nested Field Check fails in if condition

Elastic Stack Elasticsearch
1

Hello Team:

I am using elasticsearch mapping as below :

  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "age": {
        "type": "integer"
      },
      "department": {
        "type": "text"
      },
      "salary": {
        "type": "float"
      },
      "appEvent": {
        "properties": {
          "eventName": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }

We have created a pipeline as

  "processors": [
    {
      "set": {
        "field": "_index",
        "value": "debug-employee",
        "if": "ctx?.appEvent != null && ctx?.appEvent?.eventName== \u0027debug_info\u0027"
      }
    }
  ]

Now when we insert record =>

{
  "name": "Hans Raj",
  "age": 25,
  "department": "IT",
  "roleName": "devops",
  "salary": 24000,
  "appEvent.eventName": "debug_info"
}'

Pipeline is NOT routing the document to respective index.

While if i use single field , same pipeline logic works perfectly fine.

Could someone please help me in fixing the code of pipeline.

Thank You

2

I created this one

{
  "debug_jcappdetails_routing" : {
    "description" : "Route documents to another index if appevent.eventname is debug",
    "processors" : [
      {
        "set" : {
          "field" : "_index",
          "value" : "debug-jcappdetails",
          "if" : "ctx['appEvent.eventName'] == 'DEBUG_INFO'"
        }
      }
    ]
  }
}

But this too is not working ...

Someone could please help me ..

3

Hi @tusharnemade

You mapped it as an object but are indexing it as a single field so the if is not working.

In this case you would have to do the if as below:

POST /_ingest/pipeline/_simulate?verbose=true
{
  "pipeline": {
    "description": "_description",
    "processors": [
      {
        "set": {
          "field": "_index",
          "value": "debug-employee",
          "if": """
          ctx['appEvent.eventName'] == 'debug_info'
          """
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "name": "Hans Raj",
        "age": 25,
        "department": "IT",
        "roleName": "devops",
        "salary": 24000,
        "appEvent.eventName": "debug_info"
      }
    }
  ]
}

If you want to use objects, you need to correctly index your document. See below:

POST /_ingest/pipeline/_simulate?verbose=true
{
  "pipeline": {
    "description": "_description",
    "processors": [
      {
        "set": {
          "field": "_index",
          "value": "debug-employee",
          "if": """
          ctx?.appEvent?.eventName == 'debug_info'
          """
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "name": "Hans Raj",
        "age": 25,
        "department": "IT",
        "roleName": "devops",
        "salary": 24000,
        "appEvent": {                   <---------- HERE 
          "eventName": "debug_info"
        }
      }
    }
  ]
}
1 Like
4

Thank You @RabBit_BR for your response ..

It has helped me to understand better my issue.

I would like to mention , in our case , while creating the pipeline we have to mention like below code in UNICODE Values

{
  "pipeline": {
    "description": "_description",
    "processors": [
      {
        "set": {
          "field": "_index",
          "value": "debug-employee",
          "if": "ctx[\u0027appEvent.eventName\u0027] == \u0027debug_info\u0027"   << THIS ONE \u0027 for SINGLE quote
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "name": "Hans Raj",
        "age": 25,
        "department": "IT",
        "roleName": "devops",
        "salary": 24000,
        "appEvent.eventName": "debug_info"
      }
    }
  ]
}'```
5

This Code worked ... as Indexing is done as Single Field ...

1 Like