Transform --> Default a pivot groupBy keyName is does not exist

Elastic Stack Elasticsearch
1

In a transform under the section of "pivot": and "group_by": I would like to default a value if keyName does not exist. Example:

<if query_parameters.partnerId does not exists:
   partnerId=null
else:
   partnerId=query_parameters.partnerId/>

I tried to do the following:

 "partnerId": {
        "terms": {
          "field": "query_parameters.partnerId",
           "missing": "NULL"
        }

But it fails with this error when I provision the transform via API. If I remove "missing": "NULL" all works well but I need to set the groupBy keyName to null:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "x_content_parse_exception",
        "reason" : "[1:124] [data_frame_terms_group] unknown field [missing]"
      }
    ],
    "type" : "x_content_parse_exception",
    "reason" : "[135:5] [data_frame_transform_config] failed to parse field [pivot]",
    "caused_by" : {
      "type" : "x_content_parse_exception",
      "reason" : "[135:5] [data_frame_transform_pivot] failed to parse field [group_by]",
      "caused_by" : {
        "type" : "x_content_parse_exception",
        "reason" : "[1:124] [data_frame_terms_group] unknown field [missing]"
      }
    }
  },
  "status" : 400
}

How can I make it work when a keyName does not exists?

Thanks,
Lp

2

unfortunately transform does not support missing, but it supports missing_bucket

"missing_bucket": true

This will not skip buckets where partnerId==null, but create them. If you want to change the value null to something else, you can use an ingest pipeline to post process and set a value.

3

Thanks for the trick. It worked. :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.