Hi All,
I am pretty new to the ELK stack.
I am trying to create multiple indices for different logs coming from the application.
I am not using logstash and I am getting json logs from the app.
But when I am trying to use the output.elasticsearch.indices option my setup is failing.
Below is the error message:
Exiting: error initializing publisher: wrong type,
expect map accessing 'output.elasticsearch.indices.0.when.contains' (source:'/etc/filebeat/filebeat.yml')
I have tried multiple ways and this sounded promising but i am now stuck here.
Below is my filebeat.yml file i am using.
Also if the above idea will not work where do i configure in the filebeat.yml or is there anything i am missing here?
"""
###################### Filebeat Configuration Example #########################
#------------------------------ Log input --------------------------------
filebeat.inputs:
-
type: log
paths:
-
/home/mapr/fullpath(cant publish)*.log
-
/home/mapr/fullpath(cant publish).log
-
/home/fullpath(cant publish)*.log
json.message_key: job_id
-
#============================= Filebeat modules ===============================
filebeat.config.modules:
Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
Set to true to enable config reloading
reload.enabled: true
Period on which files under path should be checked for changes
reload.period: 10s
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
setup.dashboards.enabled: true
================================== Outputs ===================================
---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
Boolean flag to enable or disable the output module.
enabled: true
hosts: ["10.66.96.152:9200"]
Protocol - either http (default) or https.
protocol: "https"
Authentication credentials - either API key or username/password.
username: "admin"
password: "elastic123"
#index: "xyz-%{[beat.version]}-%{[fields.flow_type])-%{+yyyy.MM.dd}"
indices:
- index: "dimensions-%{[agent.version]}-%{+yyyy.MM.dd}"
when.contains:
flow_type:"dimensions" - index: "facerd-%{[agent.version]}-%{+yyyy.MM.dd}"
when.contains:
flow_type:"facerd" - index: "reports-%{[agent.version]}-%{+yyyy.MM.dd}"
when.contains:
flow_type:"reports"
ssl.enabled: true
ssl.verification_mode: none
ssl.certificate_authorities: ["/etc/filebeat/cacert.pem"]
=================================== Kibana =================================
setup.kibana:
host: "10.66.96.152:5601"
protocol: "https"
username: "admin"
password: "elastic123"
ssl.enabled: true
ssl.verification_mode: none
ssl.certificate_authorities: ["/etc/filebeat/cacert.pem"]
setup.ilm.enabled: false
setup.template.enabled: true
setup.template.name: "%{[beat.version]}"
setup.template.pattern: "%{[beat.version]}-*"
"""
Would be really grateful if I get any pointers on this.
Thanks,
H
