How to add new fields in ingest pipeline based on conditons?

[sidharth_vijayakumar]

Need to add a new field called app_statuscode for in filebeat. The condition is that if message field contains 404 or 502 the value of app_statuscode must be set to failed.
If the Message field contains 200 OK then the value must be set to successful.
Can someone help me out ?

[aaron-nimocks]

Here is one way to do it.

If status == 200 then create a new field called app_status with value of successful.

If status == 404 or 502 then create a new field called app_status with value of failed.

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "set": {
          "if": "ctx.app_statuscode == '200'",
          "field": "app_status",
          "value": "successful"
        }
      },
      {
        "set": {
          "if": "ctx.app_statuscode == '404' || ctx.app_statuscode == '502'",
          "field": "app_status",
          "value": "failed"
        }
      }      
    ]
  },
  "docs": [
    {
      "_source": {
        "app_statuscode": "404"
      }
    },
    {
      "_source": {
        "app_statuscode": "502"
      }
    },
    {
      "_source": {
        "app_statuscode": "200"
      }
    }
  ]
}

Output

  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "app_status" : "failed",
          "app_statuscode" : "404"
        },
        "_ingest" : {
          "timestamp" : "2021-11-08T15:20:13.04282Z"
        }
      }
    },
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "app_status" : "failed",
          "app_statuscode" : "502"
        },
        "_ingest" : {
          "timestamp" : "2021-11-08T15:20:13.042826Z"
        }
      }
    },
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "app_status" : "successful",
          "app_statuscode" : "200"
        },
        "_ingest" : {
          "timestamp" : "2021-11-08T15:20:13.042829Z"
        }
      }
    }
  ]

[sidharth_vijayakumar]

Hi,
Actually, the scenario is a bit more complicated. there is a field called message and it has value called status. Need to check if there is a status inside message if it has any of these status codes then set the value or if it does not have status then nothing to be done. Should i use grok here ??

Sample value of message field :
{action:show,count:208,duration:6.38ms,status:200}