Ingest log if field exists

Elastic Stack Kibana
1

Hello,

I'm trying to set a condition in my phpfpm pipeline that sends the log to a different index if field log.context.extra.asyncapi.name exists but is not doing anything. It keeps sending the log to the phpfpm index instead of asyncapi-phpfpm.
This is the pipeline I'm using:

{
  "description": "Basic ingest pipeline for phpfpm index",
  "processors": [
    {
      "set":{
         "field":"_index",
         "value":"phpfpm"
      }
    },
    {
      "json": {
        "if" : "ctx.containsKey('message') && ctx.message.startsWith('{')",
        "field": "message",
        "target_field": "log"
      }
    },
    {
      "set":{
         "if": "ctx.containsKey('log.context.extra.asyncapi.name')",
         "field":"_index",
         "value":"asyncapi-phpfpm"
      }
    }
}

That field exists but is not redirecting the log to asyncapi-phpfpm index for some reason.

Any help will be very appreciated. Thanks

2

Hello Mario,

I think your if-condition is wrong. I think it needs to be ctx?.log?.context?.extra?.asyncapi?.containsKey('name')

Best regards
Wolfram

3

i think the condition is wrong try this by verifying that the name field exists and !=null and second thing that you should not directly add the _index value as phpfpm try adding if the index value != asyncapi-phpfpm' this way both will be working perfectly

{
  "description": "Basic ingest pipeline for phpfpm index",
  "processors": [
    {
      "json": {
        "if": "ctx.containsKey('message') && ctx.message.startsWith('{')",
        "field": "message",
        "target_field": "log"
      }
    },
    {
      "set": {
        "if": "ctx.log?.context?.extra?.asyncapi?.name != null",
        "field": "_index",
        "value": "asyncapi-phpfpm"
      }
    },
    {
      "set": {
        "if": "ctx._index != 'asyncapi-phpfpm'",
        "field": "_index",
        "value": "phpfpm"
      }
    }
  ]
}