ES|QL query to match a CIDR in a lookup index

Elastic Search
1

Hi there.

I am trying to create an ES|QL query that will match firewall records with a source.ip field against subnets listed as CIDR in a lookup table. Basically I’m trying to find events that originate from certain CIDR blocks.

This is as close as I got:

from "logs-panw*" | where rule.name == "BLOCKED MFN OUTBOUND" 

|lookup JOIN "air-byod-network" on source.ip

where both the logs-panw* and the air-byod-network indices contain a field named source.ip.

In the panw index it is an ip field, and in the air index it is an ip_range field. But this gave me an error message:

[esql] > Unexpected error from Elasticsearch: verification_exception - Found 1 problem line 2:36: Cannot use field [source.ip] with unsupported type [ip_range]

Is this because the two fields in the join have to be the same type?

Any ideas on how to perform this sort of join/lookup between an Ip address and a CIDR block?

Thanks, Ross

2

The data type of both fields needs to match, you need the data type in your lookup index to be an ip data type, not an ip_range.

You will need to expand the CIDR and have one ip address per document, I have a similar lookup index working in this way.

3

Yeah, that’s not going to work for us. We have about a hundred subnets we need to match against, and the subnets change, so creating and maintaining thousands of individual IP records is not going to happen.

Thanks anyway.

4

I know it’s bad karma to respond to your own question, but just in-case anyone else needs to do this…

I recreated my lookup index:

put air-byod-network
{
    "settings": {
        "index": {
            "mode": "lookup"
        }
    },
  "mappings": {
    "properties":{
      "name": {
        "type": "keyword"
      },
      "source.ip": {
        "type": "ip_range"
      }
    }
  }
}

and used a python script to populate it with docs like:

“name”: “net1”,
”source.ip”: “10.1.2.3/24”

I then created an enrichment policy:

put /_enrich/policy/byod-networks
{
  "range": {
    "indices": "air-byod-network",
    "match_field": "source.ip",
    "enrich_fields": "name"
  }
}

and then ran the policy:

put /_enrich/policy/byod-networks/_execute

(remember to re-execute the policy every time the contents of the lookup index (air-byod-network in my case) changes.

They by using this ES|QL I was able to filter on just the firewall events that did not happen on my byod networks:

from "logs-panw*" | where rule.name == "BLOCKED MFN OUTBOUND"
| ENRICH byod-networks ON source.ip
| WHERE name is null

that is, the field “name” will be null when the source.ip address in the firewall log does not match one of the CIDR ranges in the lookup index field source.ip

Hope this helps someone else.

Ross