If conditional on nested field with IP address

Elastic Stack Logstash
1

I'm trying to create a simple if conditional on the host.name field if it matches an IP address. This is originating from a syslog source and is a static IP. I've tried == with quotes around the IP, escaping the octet dots, no forward slashes around the IP, =~ with quotes but none work. Also tried [host.name] with but same result. Any ideas?

    filter {
      if "syslog" in [tags] {
        if [host][name] =~ /10.10.1.1/ {
         mutate { add_tag => ["fw"] }
        }
      }
    }
2

Are you sure "syslog" is in [tags]? What does an event look like when you use

output { stdout { codec => rubydebug } }
3

Below is the json from a log that should be getting this tag applied. It does get the syslog tag originally. Could it have something to do with host.name not being in the original message because it's coming via syslog?

  "_index": "logstash-0-2020.13",
  "_type": "doc",
  "_id": "KQTTCXEB95N6Cn5a0123",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2020-03-23T23:56:48.664Z",
    "event_time_nsm_received": "2020-03-23T23:56:48.805Z",
    "host": {
      "name": "10.10.1.1"
    },
    "@version": "1",
    "message": "<23> inbound/pass1: unknown[2.2.3.4] 1585008384-1080013-5673-222263-1 1585008384 1585008387 RECV - - 2 83 - [-]\n",
    "tags": [
      "syslog"
    ],
    "uuid": "7c76ff22-478b-4fd9-9955-ef73b17f850e",
    "event_time_received": "2020-03-23T23:56:49.019Z"
  },
  "fields": {
    "@timestamp": [
      "2020-03-23T23:56:48.664Z"
    ],
    "event_time_nsm_received": [
      "2020-03-23T23:56:48.805Z"
    ],
    "event_time_received": [
      "2020-03-23T23:56:49.019Z"
    ]
  },
  "highlight": {
    "host.name": [
      "@kibana-highlighted-field@10.10.1.1@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1585007808664
  ]
}
4

If the event look like the _source you show then I would absolutely expect this to work, although escaping the octet dots would improve it.

The order of filters matters. Could you be testing for the tag before you add it?

5

In our naming of filters the syslog input begins with 0003_input_syslog.conf while the filter listed initially is 0950_identify_syslog.conf so I think it should apply later.

6

That sounds right.

I am running out of suggestions. You could try --config.debug --log.level debug --config.test_and_exit and review the debug logs to see if the ordering is what we expect.

7

So it looks like it's something to do specifically with a comparison to the [host][name] field. I modified the conditional to match something in the message field instead and I was able to get the fw tag added so it's not an ordering issue with the syslog. Unfortunately, the hostname doesn't appear in the message field so I can't use that. It makes me think the [host][name] may be added on later but I'm not sure when or there is a unique way to compare an IP to a nested field.

  if "inbound/pass1" in [message] {

This did add the fw tag but inbound/pass doesn't appear in all logs.

  "_index": "fw-0-2020.13",
  "_type": "doc",
  "_id": "k5CrDHEB5y9_RVCo5MjE",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2020-03-24T13:12:03.636Z",
    "event_time_nsm_received": "2020-03-24T13:12:03.737Z",
    "host": {
      "name": "10.10.1.1"
    },
    "@version": "1",
    "message": "<23> inbound/pass1: unknown[1.2.3.4] 1585056097-1080013-5673-228526-1 1585056097 1585056102 RECV - - 2 83 - [-]\n",
    "tags": [
      "syslog",
      "fw"
    ],
    "uuid": "6c773ac3-9a69-4d08-bea2-99ab1b85d4fa",
    "event_time_received": "2020-03-24T13:12:04.029Z"
  },
  "fields": {
    "@timestamp": [
      "2020-03-24T13:12:03.636Z"
    ],
    "event_time_nsm_received": [
      "2020-03-24T13:12:03.737Z"
    ],
    "event_time_received": [
      "2020-03-24T13:12:04.029Z"
    ]
  },
  "sort": [
    1585055523636
  ]
}
8

When using a generator input events have a [host] field that is a string, not an object, so I have to remove that before adding [host][name]

input { generator { count => 1 lines => [ '' ] } }
filter {
    mutate { remove_field => [ "[host]" ] }
    mutate { add_field => { "[host][name]" => "10.10.1.1" } }
    if [host][name] =~ /10\.10\.1\.1/ {
        mutate { add_tag => ["fw"] }
    }
}

gets me

      "host" => {
    "name" => "10.10.1.1"
},
      "tags" => [
    [0] "fw"
],
9

So apparently [host][name] is really just [host]. I actually didn't have to drop and add it back. This line below now works and gives me host.name with the proper tag.

if [host] == "10.10.1.1" { ...

Is [host] a top level field that by itself becomes [host][name] assuming no other nested fields are applied?

10

beats use a [host][name] object. A lot of logstash inputs use a [host] string. Combining the two can result in mapping exceptions in elasticsearch.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.