Filebeat version- 7.3
Logstash version- 7.3
OS- Windows
Filebeat.yml file-
filebeat.inputs:
-
type: log
enabled: true#Paths that should be crawled and fetched. Glob based paths.
paths:- c:\Program Files\filebeat-7.3.0-windows-x86_64\filebeat-7.3.0-windows-x86_64\logFiles\ESMPurchaseAPILogs\Purchase_Api_Log*
filebeat.config.modules:
#Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
#Set to true to enable config reloading
reload.enabled: false
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
name:
#The tags of the shipper are included in their own field with each
#transaction published.
tags: ["esm-purchase-api-log"]
output.logstash:
hosts: ["localhost:5044"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
Logstash config file:
input {
beats{
port=>5044
}
}
output {
if "esm-purchase-api-log" in [tags] {
elasticsearch {
hosts => ["localhost:9200"]
index => "esm-purchase-api-log-%{+YYYY.MM.dd}"
}
}
}
But it does not create any index as specified in logstash config file. Also I do not see any data reading my application logs.
However, if I directly set output of my filebeat to elasticsearch, then it starts reading my data under filebeat-* common index. However, I want to create different index for different types of files(application log files, iis log files, etc.) I am sending through filebeat. And I want to achieve it with below kind of setup:
Filebeat-> Logstash -> ElasticSerach ->Kibana
I have also enabled logstash in filebeat using command - filebeat.exe modules enable logstash. I do not want to enable iis.yml module in filebeat. I want to specify path to pick iis log files as I have done above in code.
Any help appreciated!