How to configure Filebeat -> ELK Docker Container correctly?

[zoram (ThomasK)]

Dear All,

I have already installed on my local computer ELK Docker Container and Filebeat.

I could read my local glassfish log directly from Kibana dashboard. But I want that the log file should be forwarding to ELK Container "Logstash" first.

under my ELK Container I found this conf.d folder with the following files

root@4569cf1f66ab:/etc/logstash/conf.d# ls
01-lumberjack-input.conf  02-beats-input.conf  10-syslog.conf  11-nginx.conf  30-output.conf 

My current configuration look like here.

filebeat.yml file

################### Filebeat Configuration Example #########################

############################# Filebeat ######################################
filebeat:
  # List of prospectors to fetch data.
  prospectors:
        - /Users/mmlug/Documents/university/project/runtime/glassfish-3.1/glassfish/domains/domain1/logs/server.log
        #- c:\programdata\elasticsearch\logs\*
      input_type: log
      exclude_lines: ["^DBG"]
	  
output:
  ### Elasticsearch as output
  elasticsearch:
	hosts: ["localhost:9200"]

  ### Logstash as output
  #logstash:
    # The Logstash hosts
   	#hosts: ["localhost:5044"]

Step 1 Starting Docker  : sudo docker-compose up elk
Step 2 : sudo ./filebeat -e -c filebeat.yml -d "publish"

When I enable "Logstash" as a output, I always got an errors as below...

Loading config file error: YAML config parsing failed on filebeat.yml: yaml: line 279: found character that cannot start any token. Exiting.

Many thanks in advance.

Best Regards,
Thomas.

[ruflin]

Could it be that your filebeat.yml is much longer then what you posted above? There seems to be a problem on lin 279. Most of the time the issue with yaml is using tabs instead of spaces.

[zoram (ThomasK)]

hi ruflin,

Thanks.! Yes, I could fixed the error.
After editing "02-beats-input.conf" under ":/etc/logstash/conf.d" I could now see the glassfish logs in kibana.

Do you know how I can proper display glassfish multiple line logs in Kibana?

Best Regards,
Thomas

[ruflin]

That is probably what you are looking for: https://www.elastic.co/guide/en/beats/filebeat/5.0/multiline-examples.html#multiline-examples

[zoram (ThomasK)]

thanks @ruflin