Can I turn the path (or parts of it) into a variable?

[jclose (Jason)]

I want to ship some syslog out from our syslog server into Elastic, and obviously Filebeat works for this.

But I want to send part of the path as a variable within the beat.

So, for instance, my path will look like /data/syslog/firewall/2017/05/25/10.0.0.1/[whatever].log, which maps to something like /data/syslog/%{event_type}/%{+YYYY}/%{+MM}/%{+dd}/%{host_ip}

I would like to ship that %{host_ip} within my beat as a variable. It's handy, because if I have a ton of different hosts, or a ton of different event types, I can wildcard out my filebeat configuration.

I would love to use the 'fields' ability to pass that host IP or event type as a variable. Any way for me to do that?

[ruflin]

This is currently not possible on the beats side, but you can do that in Logstash or Ingest and add the fields there, as you have the full path shipped with each event. Would that work for you?

[jclose (Jason)]

Not really, as I was trying to use it as a workaround to the issue I posted here. Right now, the only way I've gotten the auditd module to work is via the Pipeline used by Filebeat. I tried to force Logstash to use the same pipeline, and it didn't work

My thinking was, since Logstash wasn't working for me, I can send stuff straight to file in Logstash (via file output, or I could just send it to the syslog process on the box, which is listening).

Then, I could point Filebeat to those files and ship, defining the pipeline. It was really an attempt at a workaround of that other issue. But it would still be handy to have.