Filebeat not taking files from directory and sending to Logstash


(Andrew Apple) #1

The filebeat instance I am running wont seem to send files in a directory to my logstash instance. Below is the filebeat.yml config file which I use a Shell Script to automatically change according to the type of file I want it to take in. I have the FIlebeat service restart every time I change it and still have had zero luck. I also added the logstash config file I use to handle inputs in case that is the issue but I am fairly sure it's filebeat as I cannot seem to get it to output to either logstash or just a file. Any advice on how to troubleshoot this? Everything in the config file is left the same except for the changed path under the "Paths:", the document type, and the logstash output. I had to delete a lot of the config file to make this post due to the character limit but the rest of the file is unchanged from the default configuration.

`
################### Filebeat Configuration Example #########################

############################# Filebeat ######################################
filebeat:
prospectors:
-
paths:
- /home/elkadmin/PlasoData/
input_type: log
document_type: l2tcsv
###############################################################################
############################# Libbeat Config ##################################

Base config file used by all other beats for using libbeat features

############################# Output ##########################################
output:

Logstash as output

logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
# Number of workers per Logstash host.
#worker: 1
# Set gzip compression level.
#compression_level: 3
# Optional load balance the events between the Logstash hosts
#loadbalance: true
# Optional index name. The default index name depends on the each beat.
# For Packetbeat, the default is set to packetbeat, for Topbeat
# top topbeat and for Filebeat to filebeat.
#index: filebeat
# Optional TLS. By default is off.

`

Logstash Config File:
input { beats { port => 5044 #ssl => true #ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" #ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" } }


(Magnus B├Ąck) #2
- /home/elkadmin/PlasoData/

Is this really what's in your configuration file? You need to supply a filename pattern, not a directory.

If that doesn't help, consult the Filebeat logs. You may want to increase the log level.


(system) #3

This topic was automatically closed after 21 days. New replies are no longer allowed.