Else conditional in ingest pipeline

Elastic Stack Elasticsearch
1

Hello, I want to make else conditional on my ingest pipeline, here's pipeline I've made:

PUT _ingest/pipeline/pipeline-test
{
  "processors": [
    {
      "pipeline":{
        "description": "check field1 to output to pipeline1",
        "if": "ctx?.field1 == 'value1'",
        "name": "pipeline1"
      }
    },
    {
      "pipeline": {
        "description": "check field1 to output to pipeline2",
        "if": "ctx?.field1 == 'value2'",
        "name": "pipeline2"
      }
    },
    {
      "drop": {
        "description": "drop everything else",
        "if": "ctx?.field1 != 'value1' && ctx?.field1 != 'value2'"
      }
    }
    ]
}

is there any solution to make else conditional so I can drop everything except the field with value1 and value2. I don't want to add more parameter inside drop processor if someday I have to add more pipeline.

Thanks in advance

2

anyone have insight about this?

3

Hi @alfianaf

If / Else statements can be used in the Logstash Modules like this example.

Same applies with the filter module specifically

4

hello, sorry before, but I dont use logstash nor filebeat. I use ingest pipeline on my Elasticsearch directly, if you want to know the reference I used is from here

5

Hi @zx8086 I think @alfianaf is referring to ingest pipelines not logstash pipelines.

@alfianaf I am not exactly clear what you are asking.

Ingest pipelines are executed in order they are defined.

I am not exactly clear what you are asking can you show me "pseudo" code or something what you want to accomplish?

The drop processor will drop the entire message not just fields... so the above drop will drop every event that that field1 is not value1 or value2

if you are looking to drop all the fields except field1 that is something completely different.

6

hello @stephenb , exactly I want to drop the entire message, so I want to use drop that works as much as "else", I want to drop every message else than those 2 pipeline, since I saw on documentation that drop pipeline use "if", what "if" in my "drop" pipeline can I put to get "else" behaviour?

7

I'm still not completely sure What you want to do but there is no else at the top level processor control.

The if is actually inside each processor not outside... so there is no concept of else outside / above processor itself as far as I am aware.

8

if you watch carefully what I write on the pipeline I've made, I actually made "drop" pipeline to act as "else" from another 2 pipeline, I didn't really want to use "else" outside processor, but I just want to drop every document else than the 2 pipeline mentioned above. But it is too much if I have more than 2 pipeline, so I have to write much field on my "if" inside my "drop" pipeline

9

Agree, That is why I was a bit confused in the beginning because it seemed to be doing exactly what you were describing.

I'm just explaining that the conditionals are inside the processors not outside, there is no control flow outside the processors they are simply executed in order, And there is no else as requested in the title of the thread

10

I see, so it is not possible to do the thing I asked, thanks for your explanation

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.