Filebeats on ElasticBean Stalk


(Mathew Mathew) #1

Trying to setup filebeats on elastic bean stalk to forward to the AWS Elasticsearch service.
Getting this error when I try to start filebeats.

Starting filebeat: 2015/11/20 17:01:23.891713 main.go:42: CRIT Config error: Error reading config file: YAML config parsing failed on /etc/filebeat/filebeat.yml: yaml: unmarshal errors:
line 7: cannot unmarshal !!map into []config.ProspectorConfig. Exiting.

-- This by elasticbean stalk .ebextension file. Installing the filebeats service

files:
    "/etc/filebeat/filebeat.yml":
        mode: "000755"
        owner: root
        group: root
        content: |
            filebeat:
                # List of prospectors to fetch data.
                prospectors:
                # Each - is a prospector. Below are the prospector specific configurations
                # Paths that should be crawled and fetched. Glob based paths.
                # For each file found under this path, a harvester is started.
                    paths:
                        - "/var/log/eb-docker/containers/eb-current-app/*.log"
                    input_type: log

            output:
                ### Elasticsearch as output
                elasticsearch:
                    # Array of hosts to connect to.
                    # Scheme and port can be left out and will be set to the default (http and 9200)
                    # In case you specify and additional path, the scheme is required: http://localhost:9200/path
                    # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
                    hosts: ["search-XXXXX-w4peuekyqtmgn4ofx25co6a7ne.eu-west-1.es.amazonaws.com:80"]

            logging:
                to_files: true
                files:
                    path: "/var/log/mybeat"
                    rotateeverybytes: 10485760

commands:
  1_command:
     command: "curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.0.0-rc2-x86_64.rpm"
     cwd: /home/ec2-user
  2_command:
     command: "rpm -ivh --replacepkgs filebeat-1.0.0-rc2-x86_64.rpm"
     cwd: /home/ec2-user
  3_command:
     command: "/etc/init.d/filebeat start"

(Tudor Golubenco) #2

I noticed a missing - in the configuration of filebeat. You need a correctly indented - to start each prospector section, see the example here.


(Khalid) #3

Thank you. Found the right answer at the right time.


(system) #4