How to use Filebeat pipelines for both source and dest GeoIP

I am ingesting network data, and trying to get both the destination and source IP address to populate as geoips. I am able to get one of them to work while using Filebeat pipeline, and elasticsearch's GeoIP plugin, but not both.

Below is a copy of the filebeat.yml with the pipeline section, and the elasticsearch pipelines.

filebeat.yml
using GEOIP elastic search plugin
output.eleasticsearch:
hosts: [”x.x.x.x”]
index: “fa-%{+yyyy-MM-dd}”
template.enable: false
pipelines:
- pipeline: “DestIPGeo”
- pipeline: “SrcIPGeo”

PUT _ingest/pipeline/SrcIPGeo
{
“processors”: [
“geoip”: {
“field”: “SourceIP”,
“target_field” : “SrcGeoIP”
}
}
]
}

PUT _ingest/pipeline/DestIPGeo
{
“processors”: [
“geoip”: {
“field”: “DestinationIP”,
“target_field” : “DestIPGeo”
}
}
]

When I do this, only one of them works. I have tried doing just the src or just the dest, and they work, but not both pipelines enabled at the same time. Do I need to use logstash for this?

Filebeat will choose only one pipeline. The pipelines setting is used to select one pipeline based on configurable conditions or string formatters (e.g. if first string pipeline access non-present field, the next pipeline setting will be tested). You will have to put both lookups into one pipeline.

Steffens,

I got it!

Thanks for the reply.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.