Filebeat not sending any log files

Elastic Stack Beats
1

Hello,

I'm encountering a weird issue with the Filebeat on one of our systems.
I've configured it identical to all other systems, but for some reason this one box is not sending out any log files.
I've also noticed that the Filebeat app is not producing any entries in it's log file whatsoever.

The system is a Ubunty Trusty setup with Filebeat installed as package.
The configuration for this service is as follows:

filebeat:
  prospectors:
    -
      paths:
        - /home/riskmethods/riskmethods/current/log/staging.log
      encoding: utf-8
      input_type: log
      fields:
        origin: staging
        technology: Ruby on Rails
      document_type: log
      registry_file: /var/lib/filebeat/registry
    -
      paths:
        - /home/riskmethods/riskmethods/current/log/unicorn.log
      encoding: utf-8
      input_type: log
      fields:
        origin: staging
        technology: Unicorn
      document_type: log
      registry_file: /var/lib/filebeat/registry
output:
  logstash:
    hosts: ["logging.riskmethods.net"]
    tsl:
      timeout: 15
      certificate_authorities: /etc/nginx/wildcard.riskmethods.net.cer
      certificate: /etc/nginx/wildcard.riskmethods.net.pem
      certificate_key: /etc/nginx/wildcard.riskmethods.net.key
shipper:
  name: staging
  tags: ["playground"]
logging:
  level: warning
  to_files: true
  files:
    path: /var/log/filebeat
    name: beat.log
    keepfiles: 7
    rotateeverybytes: 10485760 # 10 MB

But the /var/log/filebeat/beat.log file was not created, and when I created it manually it remains empty. Piping all output from -e to a file reveals nothing as well.
Ping'ing the end destinations work and Firewalls are configured.

Kinda at a loss here as to why this box is not sending anything.
Log files to follow are there and being filled with entries.

1 Like
2

Fixed it.
The problem was that I've forgotten to add the port to the hosts section for the output.
So filebeat was sending it to somewhere unknown :confused:

Working like a charm now

3

Great.

And FYI, if you need more verbosity from logging you can configure it in the YAML config file.

logging:
  level: warning
  to_files: true
  files:
    path: /var/log/filebeat
    name: beat.log
    keepfiles: 7
    rotateeverybytes: 10485760 # 10 MB
  level: debug 
  selectors: ["*"]

That will give you all possible logging (super verbose, but good for debugging). See the logging documentation for more details: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-details.html#_level

4

@andrewkroh

Step1:

I have installed Elastic search 2.3 , Kibana 4.5, Logastash 2.3 and shield 2.3 on single server.

created using esusers command line I have created one admin user with admin role and another user with user role.

Right now I am able to login to Kibana with user credentials created.

Step 2:

Loaded Kibana Dashboards
Load Filebeat Index Template in Elasticsearch
installed filebeat and configured it on client server.

step3 : Tried to test filebeat installation and I get below error.

curl -XGET "http://ipaddress:9200/filebeat-/_search?pretty" -u admin -p
Enter host password for user 'admin':xxx
{
"error" : {
"root_cause" : [ {
"type" : "index_not_found_exception",
"reason" : "no such index",
"index" : "[filebeat-]"
} ],
"type" : "index_not_found_exception",
"reason" : "no such index",
"index" : "[filebeat-*]"
},
"status" : 404
}

Please help me in fixing this issue

5

Please start a new thread.