Operators in Ingest Pipeline

Elastic Stack Elasticsearch
, ,
1

Hello, I am trying to parse and compare values through ingest pipeline, but couldn't do it,
I was running below code, grok is running fine, but couldn't be able to compare value in set condition.

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": ["%{HOSTNAME:host.name},%{WORD:host.driveletter},%{INT:host.driveusage}"]
        },
        "set": {
          "field": "summary.down",
          "value": 1,
          "if": "ctx.host.name == 'CALCWZ88C3'"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "message": "CALCWZ88C3,C,13"
      }
      
    }
    
  ]
}

getting below issues

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "script_exception",
            "reason": "runtime error",
            "script_stack": [
              "ctx.host.name == 'CALCWZ88C3'",
              "        ^---- HERE"
            ],
            "script": "ctx.host.name == 'CALCWZ88C3'",
            "lang": "painless",
            "position": {
              "offset": 8,
              "start": 0,
              "end": 29
            }
          }
        ],
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "ctx.host.name == 'CALCWZ88C3'",
          "        ^---- HERE"
        ],
        "script": "ctx.host.name == 'CALCWZ88C3'",
        "lang": "painless",
        "position": {
          "offset": 8,
          "start": 0,
          "end": 29
        },
        "caused_by": {
          "type": "null_pointer_exception",
          "reason": "cannot access method/field [name] from a null def reference"
        }
      }
    }
  ]
}

Could you please help ?

I have one more question , is there < and > than operators in ingest pipeline you can use, if yes please what's the syntax, I have to compare value which comes from grok ouput

2

Hi @errupeshmca,
When I tried running your simulated command I was getting the same errors you saw, but actually creating the pipeline worked fine. Please try these steps:

  1. Create the pipeline
PUT _ingest/pipeline/experiment
{
  "processors" : [
    {
      "grok": {
          "field": "message",
          "patterns": ["%{HOSTNAME:host.name},%{WORD:host.driveletter},%{INT:host.driveusage}"]
        },
        "set": {
          "field": "summary.down",
          "value": 1,
          "if": "ctx.host.name == 'CALCWZ88C3'"
        }
    }
  ]
}
  1. Post test message
POST /your_index_name_here/_doc?pipeline=experiment
{
 "message": "CALCWZ88C3,C,112"
}
  1. Verify it looks good
GET /your_index_name_here/_search

If all goes well you should see something like this:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "your_index_name_here",
        "_type" : "_doc",
        "_id" : "84gkd24YBQ_bO945337D",
        "_score" : 1.0,
        "_source" : {
          "summary" : {
            "down" : 1
          },
          "host" : {
            "driveusage" : "112",
            "name" : "CALCWZ88C3",
            "driveletter" : "C"
          },
          "message" : "CALCWZ88C3,C,112"
        }
      }
    ]
  }
}

In regards to your question on comparison operators see this documentation.

3

Thanks Andrew that works, and thanks for the documentation links.