I am ingesting network data, and trying to get both the destination and source IP address to populate as geoips. I am able to get one of them to work while using Filebeat pipeline, and elasticsearch's GeoIP plugin, but not both.
Below is a copy of the filebeat.yml with the pipeline section, and the elasticsearch pipelines.
filebeat.yml
using GEOIP elastic search plugin
output.eleasticsearch:
hosts: [”x.x.x.x”]
index: “fa-%{+yyyy-MM-dd}”
template.enable: false
pipelines:
- pipeline: “DestIPGeo”
- pipeline: “SrcIPGeo”
PUT _ingest/pipeline/SrcIPGeo
{
“processors”: [
“geoip”: {
“field”: “SourceIP”,
“target_field” : “SrcGeoIP”
}
}
]
}
PUT _ingest/pipeline/DestIPGeo
{
“processors”: [
“geoip”: {
“field”: “DestinationIP”,
“target_field” : “DestIPGeo”
}
}
]
When I do this, only one of them works. I have tried doing just the src or just the dest, and they work, but not both pipelines enabled at the same time. Do I need to use logstash for this?