GeoIP enrich IP addresses broken if fileting with include_fields

[dmgeurts (Djerk Geurts)]

I'm logging DNS packets without dns.response_code: NOERROR. This part works fine.

But when I try to smile down the logged data, by adding a filter to drop all but some selected fields, GeoIP breaks. GeoIP can't be used in the Elastic Agent Packetbeat processor, so I'm a bit stuck without this as a workaround.

The following works fine:

- drop_event:
    when:
      equals:
        dns.response_code: NOERROR

And this works great for reducing the data volume, but breaks GeoIP too:

 include_fields:
    fields:
      - source
      - server.bytes
      - server.ip
      - dns.question.name
      - dns.question.type
      - dns.question.etld_plus_one
      - dns.response_code
      - network.transport
      - network.type

Is this a bug, or a feature?

I guess I could add the GeoIP data in Kibana, but I'd prefer to log the data properly in the first place.

[andrewkroh]

You are dropping the source object and the geoip processor makes use of the source.ip field so you will never get any source.geo info.

elastic/integrations/blob/9dcd6eac196eed413eff898c1ecbc46fe1d745e1/packages/network_traffic/data_stream/dns/elasticsearch/ingest_pipeline/geoip.yml#L5

You could create an ingest pipeline named logs-network_traffic.dns@custom which would run after the geoip pipeline to drop the source field. Fleet will automatically invoke your custom pipeline at the end of the built-in one.