Background: at my organization, we heavily use Logstash and kafka to deal with buffering and spikes in the traffic flowing to ElasticSearch, as well as scaling processing horizontally.
Because of this, the "ingest" pipelines in Filebeats aren't directly useful. So I've investigated the ingest-convert.sh script that comes with Logstash, and mostly have it working (side note: it would be nice if the docs were updated to note that the Beats ingest "logic" is now in YAML instead of JSON like the converter wants).
However, I've found that some fields don't appear to be getting correctly parsed by the converter, and that's the reason for this post.
When I convert the Filebeat iis access ingest module (Filebeat v7.10.2) using the Logstash 7.10.2 ingest converter, fields enclosed in {{ and }} don't get mapped -- they're passed through verbatim (like "{{source.ip}}"). I'm pretty sure some cleanup of the output is expected, but this particular one can easily become tedious...
Is this expected behavior, to pass "{{field.name}}" through verbatim into the Logstash config? It sure seems like a bug to me.
Thanks,
- Daniel