Filebeat custom index

[richard_N (Richard)]

I'm running filebeat 7.2 and trying to capture some exchange logs and put them in their own custom index but they always seem to end up in the default filebeat index. Below is my filebeat.yml. I've tried setting the setup.template options to several different things with no luck. I want them to go into an index 'filebeat-exchange-*'

filebeat.inputs:

• type: log
  enabled: true
  paths:
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Hub\Connectivity*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Frontend\ProtocolLog\Connectivity*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Frontend\ProtocolLog\SmtpSend*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Frontend\ProtocolLog\SmtpReceive*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Mailbox\Connectivity\Delivery*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Mailbox\Connectivity\Submission*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Mailbox\ProtocolLog\SmtpReceive\Delivery*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Mailbox\ProtocolLog\SmtpReceive\Submission*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Mailbox\ProtocolLog\SmtpSend\Delivery*.LOG
  • C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Mailbox\ProtocolLog\SmtpSend\Submission*.LOG

setup.template:
name: "filebeat"
pattern: "filebeat-*"
overwrite: true

output.elasticsearch:
hosts:
- es01:9200
- es02:9200
- es03.9200
index: "filebeat-exchange-%{[agent.version]}-%{+yyyy.MM.dd}"

tags: ["mail", "exchange"]

[rugenl (Len Rugen)]

I think ILM may be the indirect cause here... With the 7.x versions ILM is enabled (says auto), so the default for the ilm_rollover_alias is "filebeat", and that field takes precidence over the index name.

Check the logs for the first startup, see if it tried to do things with "lifecycle management".

To verify if it's ILM confusion, you can set ilm_enabled: false.

We are sending exchange logs thru Logstash because there are a lot of fields to parse and process.... and most of their logs have a different format and the thought of common fields names is complete foreign to M$.