How to use Filebeat pipelines for both source and dest GeoIP

[Clueless (Clue)]

I am ingesting network data, and trying to get both the destination and source IP address to populate as geoips. I am able to get one of them to work while using Filebeat pipeline, and elasticsearch's GeoIP plugin, but not both.

Below is a copy of the filebeat.yml with the pipeline section, and the elasticsearch pipelines.

filebeat.yml
using GEOIP elastic search plugin
output.eleasticsearch:
hosts: [”x.x.x.x”]
index: “fa-%{+yyyy-MM-dd}”
template.enable: false
pipelines:
- pipeline: “DestIPGeo”
- pipeline: “SrcIPGeo”

PUT _ingest/pipeline/SrcIPGeo
{
“processors”: [
“geoip”: {
“field”: “SourceIP”,
“target_field” : “SrcGeoIP”
}
}
]
}

PUT _ingest/pipeline/DestIPGeo
{
“processors”: [
“geoip”: {
“field”: “DestinationIP”,
“target_field” : “DestIPGeo”
}
}
]

When I do this, only one of them works. I have tried doing just the src or just the dest, and they work, but not both pipelines enabled at the same time. Do I need to use logstash for this?

[steffens (Steffen Siering)]

Filebeat will choose only one pipeline. The pipelines setting is used to select one pipeline based on configurable conditions or string formatters (e.g. if first string pipeline access non-present field, the next pipeline setting will be tested). You will have to put both lookups into one pipeline.

[Clueless (Clue)]

Steffens,

I got it!

Thanks for the reply.