Multiple filebeat instances sending logs

Hi, I have installed filebeat on multiple servers to pick system logs.
The filebeat for server a works fine.
filebeat.yml

  • type: log
    enabled: true
    paths:
    • /var/log/jenkins/jenkins.log
      exclude_files: ['.gz$']
      multiline.pattern: '[1]+\s[0-9]{1,2},\s[0-9]{4}\s[0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}\s(?:AM|am|PM|pm)'
      multiline.negate: true
      multiline.match: after
      fields:
      type: jenkins-server
      fields_under_root: true

logstash/conf.d/pipeline.conf
output
{
if [type] == "jenkins-server" {
elasticsearch {
hosts => ["{{ elk_ip }}:9200"]

            user => "{{ elk_user }}"
            password => "{{ elk_password }}"

            action => "index"
            index => "jenkins_syslog%{+YYYY.MM.dd}"
     }
   }

}
Sends log as expected. For the second server the filebeat input is somewhat like this

  • type: log
    enabled: true
    paths:
    • /var/log/elasticsearch/*.log
      fields:
      type: elasticsearch_syslog
      fields_under_root: true
  • type: log
    enabled: true
    paths:
    • /var/log/kibana/*.log
      fields:
      type: kibana_syslog
      fields_under_root: true

When i add the following code in pipeline.conf
if [type] == "elasticsearch_syslog" {
elasticsearch {
hosts => ["43.204.205.20:9200"]

            user => "elastic"
            password => "minutus"

            action => "index"
            index => "elasticsearch_syslog%{+YYYY.MM.dd}"
     }
   }
   if [type] == "kibana_syslog" {
      elasticsearch {
            hosts => ["43.204.205.20:9200"]

            user => "elastic"
            password => "minutus"

            action => "index"
            index => "kibana_syslog%{+YYYY.MM.dd}"
     }
   }

The indexes aren't getting created.
Is this the correct way to do it? or should i use a separate pipeline for every filebeat input, if yes how? do i need to add if else?


  1. a-zA-Z ↩︎