FileBeat Optimization

[manojvenkat (Manoj)]

Dear All,

I have Installed Filebeat on my application server and ELK stack on another server. I have configured filebeat output to Elasticsearch and I started seeing logs on Kibana. But in few minutes logs stopped shipping. I restarted Filebeat and Application server and again logs started to display on Kibana for few minutes. Could someone please help me what could be the reason for this. Do I need to optimize/fine tune Filebeat or elasticsearch? I am a newbie to ELK. Appreciate your support. Thanks.

The output in filebeat.yml is configured to Elasticsearch only and commented out other fields.

[ruflin]

Can you share your filebeat config and log output? In addition, which versions of FB, Elasticsearch and Kibana are you using?

[manojvenkat (Manoj)]

Hi Ruflin,

Below versions are used,

filebeat version: 1.3.1
elasticsearch version:2.4
kibana version :4.4.2

[manojvenkat (Manoj)]

Ruflin, I am not able to paste the full yml file here.

Below is what configured in output,

  elasticsearch:
    # Array of hosts to connect to.
    # Scheme and port can be left out and will be set to the default (http and 9200)
    # In case you specify and additional path, the scheme is required: http://localhost:9200/path
    # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
    hosts: ["10.30.66.227:9200"]

I have not made any other custom changes to YML file.

[ruflin]

You can use a gist to paste the full log and config and then link it here. Without the log files it is very hard to figure out what is going on.