Best way to enrich by IP subnets

Elastic Stack Elasticsearch
1

I want to enrich incoming ECS-compatible documents with a tag if they come from VPN IPs.

From my testing, the enrich processor cannot deal with IP-ranges. However, i want to use IP-ranges and not manually (or by a script) spell out thousands of IPs for the enrich policy. What's the best strategy to achieve this?

Summary

PUT test-ip-range-lookup
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"properties": {
"ip": {
"type": "ip_range"
}
}
}
}

POST test-ip-range-lookup/_doc/vpn-ips
{
"ip": "192.168.0.0/16",
"tags": ["vpn-ip"]
}

PUT _enrich/policy/test-ip-enrich
{
"match": {
"indices": "test-ip-lookup",
"match_field": "ip",
"enrich_fields": ["tags"]
}
}

POST _enrich/policy/test-ip-enrich/_execute

POST _ingest/pipeline/_simulate
{
"docs": [
{
"_source": {
"host": {
"ip": "192.168.0.1"
}
}
}
],
"pipeline": {
"processors": [
{
"enrich": {
"policy_name": "test-ip-enrich",
"field": "host.ip",
"target_field": "tags",
"ignore_missing": true
}
}
]
}
}

2

You're correct about the enrich processor. Support for range queries has been requested, among other features.

What's sending the documents to the cluster?

If it's something you've built yourself, implement the enrichment query and add to the document before indexing it.

If you can't change the code, set the output to logstash and implement a pipeline with an Elasticsearch filter to do the enrichment.

3

Ah thanks, thats a good idea I hadn't thought of before.

I think I'd rather wait for the enrich processor to support other datatypes though. Since I'm only using the vpn-tag for one watcher use-case, I'll just use the terms query for now (it supports searching for CIDR-strings against IP fields)

5

Just wanted to let you know that Elasticsearch 7.16 was released with this functionality, see Example: Enrich your data by matching a value to a range | Elasticsearch Guide [7.16] | Elastic for an example. Thank you for your feedback.