Field reference from _source for conditional output

hello, I'm new to logstash conditional, I want to make different index output based on some field reference on my logstash

here is my index example :

{
  "_index": "iris-new-2021.09",
  "_type": "_doc",
  "_id": "EKS5EHwBUrXRxI7i7tvA",
  "_version": 1,
  "_score": null,
  "_source": {
    "input": {
      "type": "syslog"
    },
    "@timestamp": "2021-09-23T03:36:30.421Z",
    "tags": [
      "beats_input_codec_plain_applied"
    ],
    "openshift": {
      "message": "\tat com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:554)",
      "level": "unknown",
      "viaq_msg_id": "YTlkYzdhNzItM2U2NC00NmRiLWFmZTItNDc2M2FiZWExZTJj",
      "kubernetes": {
        "container_image_id": "image-registry.openshift-image-registry.svc:5000/iris-uat/iris-batch-swcon-prod@sha256:d4d74861c96f728af3cb953aa77e0f4c7d220541fb82a048a76f546b4e5a3c94",
        "pod_id": "96a3657c-a2e7-4918-9f83-10be97123270",
        "container_name": "iris-batch-swcon-prod",
        "master_url": "https://kubernetes.default.svc",
        "container_image": "image-registry.openshift-image-registry.svc:5000/iris-uat/iris-batch-swcon-prod@sha256:d4d74861c96f728af3cb953aa77e0f4c7d220541fb82a048a76f546b4e5a3c94",
        "host": "devocpworker04.ocpdev.dti.co.id",
        "pod_name": "iris-batch-swcon-prod-1-s57zc",
        "flat_labels": [
          "app=iris-batch-swcon-prod",
          "deployment=iris-batch-swcon-prod-1",
          "deploymentconfig=iris-batch-swcon-prod"
        ],
        "namespace_name": "iris-uat",
        "namespace_id": "0da8b310-ceaf-4446-a645-11f3b792d572"
      },
      "pipeline_metadata": {
        "collector": {
          "ipaddr4": "10.58.81.34",
          "name": "fluentd",
          "version": "1.7.4 1.6.0",
          "received_at": "2021-08-01T03:23:39.492821+00:00",
          "inputname": "fluent-plugin-systemd"
        }
      },
      "docker": {
        "container_id": "5001c9085439a436eb3ef1f146b0e390003e01f93dda1a269b9d2fae3a8a4286"
      },
      "hostname": "devocpworker04.ocpdev.dti.co.id",
      "@timestamp": "2021-08-01T03:23:38.794143+00:00"
    }
  },
  "fields": {
    "openshift.@timestamp": [
      "2021-08-01T03:23:38.794Z"
    ],
    "openshift.pipeline_metadata.collector.received_at": [
      "2021-08-01T03:23:39.492Z"
    ],
    "@timestamp": [
      "2021-09-23T03:36:30.421Z"
    ]
  },
  "sort": [
    1632368190421
  ]
}

I want to use [_source][openshift][kubernetes][namespace_name] as reference to make index output

here's what I've tried :
1.

if "iris-uat" in "%{[openshift][kubernetes][namespace_name]}" {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-uat-%{+YYYY.MM}"
		}
	}
if "iris-uat" in "[openshift][kubernetes][namespace_name]" {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-uat-%{+YYYY.MM}"
		}
	}
if "iris-uat" in [openshift][kubernetes][namespace_name] {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-uat-%{+YYYY.MM}"
		}
	}

am I missing something, I've also tried delete the [openshift] and only use the rest of sub field instead, but I still couldn't get the output

feel free to ask for more information regarding my question if you don't understand about what I'm asking

Thanks

if "iris-uat" in "%{[openshift][kubernetes][namespace_name]}" {

You cannot use a sprintf reference in a conditional. This is a substring match.

if "iris-uat" in "[openshift][kubernetes][namespace_name]" {

That is a also substring match against a string, so if "ift][kuber" in "[openshift][kubernetes][namespace_name]" { would evaluate to true, but that never will.

if "iris-uat" in [openshift][kubernetes][namespace_name] {

I would expect that to work. Instead of showing us the result from an index search can you show us an event output from logstash with

output { stdout { codec => rubydebug } }
1 Like

fyi, I make a else conditional to output to other index, from every conditional I've used, it is forwarded to the "else" condition, so to make it clear, the "if" condition was not met

Yes, I understood that. It is an unexpected result. That is why I asked for additional data.

I used the bottom one from you've recommended,
sorry if my information before is not complete enough
here was my config :

if "iris-uat" in [openshift][kubernetes][namespace_name] {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-uat-%{+YYYY.MM}"
		}
	}
else if "iris" in [openshift][kubernetes][namespace_name] {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-dev-%{+YYYY.MM}"
		}
	}

I think the anomaly happened because "iris" and "iris-uat" is counted as same because I used "in" conditional,
rather I use "==" conditional to add tag and then use "in" conditional for the output

here's my code on my filter section:

		if [openshift][kubernetes][namespace_name] == "iris" {
			mutate { add_tag => "iris" }
		} else if [openshift][kubernetes][namespace_name] == "iris-uat" {
			mutate { add_tag => "iris-uat" }
		}

and this on my output section :

if "iris" in [tags] {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-dev-%{+YYYY.MM}"
		}
	}
	else if "iris-uat" in [tags] {
		elasticsearch {
			hosts => ["xxx:9200"]
            index => "iris-uat-%{+YYYY.MM}"
		}
	}
    else {
        #stdout { codec => rubydebug }
        elasticsearch {
            hosts => ["xxx:9200"]
            index => "iris-new-%{+YYYY.MM}"
        }
    }

Now it is fixed, thanks for your explanation, because now I can clearly know how the syntax work

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.