Filebeat startup fail

[thps (Torsten Lorenz)]

Hello everyone,

i´m trying to setup elastic-stack as a logserver with elastic search, logstash and filebeat. I´ve tested it on poc-machines with root-acces. Everything works fine. On our productive enviroment i got in some trouble with filebeat.

I didn´t get it to start. If i try to start it with:

arsadmin@vls20507:/var/log/filebeat>sudo systemctl start filebeat
arsadmin@vls20507:/var/log/filebeat>sudo systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
Loaded: loaded (/usr/lib/systemd/system/filebeat.service; disabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Mo 2019-03-25 07:45:27 CET; 2s ago
 Docs: https://www.elastic.co/products/beats/filebeat
Process: 18734 ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml - 
path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs 
/var/log/filebeat (code=exited, status=1/FAILURE)
Main PID: 18734 (code=exited, status=1/FAILURE)

Mär 25 07:45:27 vls20507 systemd[1]: Unit filebeat.service entered failed state.
Mär 25 07:45:27 vls20507 systemd[1]: filebeat.service failed.
Mär 25 07:45:27 vls20507 systemd[1]: filebeat.service holdoff time over, scheduling restart.
Mär 25 07:45:27 vls20507 systemd[1]: start request repeated too quickly for filebeat.service
Mär 25 07:45:27 vls20507 systemd[1]: Failed to start Filebeat sends log files to Logstash or 
directly to Elasticsearch..
Mär 25 07:45:27 vls20507 systemd[1]: Unit filebeat.service entered failed state.
Mär 25 07:45:27 vls20507 systemd[1]: filebeat.service failed.

No logfile will be created....

if i try to start it directly with
/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -
path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs
/var/log/filebeat

nothing happens, but a logfile will be created:
arsadmin@vls20507:/var/log/filebeat>tail filebeat
2019-03-25T07:51:06.432+0100 INFO elasticsearch/client.go:165 Elasticsearch url:
http://[xxxx... edit...]:9200
2019-03-25T07:51:06.432+0100 INFO [publisher] pipeline/module.go:110 Beat name:
vls20507
2019-03-25T07:51:06.432+0100 INFO instance/beat.go:403 filebeat start running.
2019-03-25T07:51:06.432+0100 INFO [monitoring] log/log.go:117 Starting metrics logging
every 30s
2019-03-25T07:51:06.433+0100 INFO registrar/registrar.go:134 Loading registrar data
from /var/lib/filebeat/registry
2019-03-25T07:51:06.433+0100 INFO registrar/registrar.go:141 States Loaded from
registrar: 0
2019-03-25T07:51:06.433+0100 INFO crawler/crawler.go:72 Loading Inputs: 1
2019-03-25T07:51:06.433+0100 INFO crawler/crawler.go:106 Loading and starting Inputs
completed. Enabled inputs: 0
2019-03-25T07:51:06.433+0100 INFO cfgfile/reload.go:150 Config reloader started
2019-03-25T07:51:06.433+0100 INFO cfgfile/reload.go:205 Loading of config files
completed.

i´ve configured the elastic search in the filebeat.yml, but it even wont work with the logstash server.
Are there any ideas?

greetz thps

[kvch (Noémi Ványi)]

Have you edited the unit file with comes with Filebeat?
Could you please enable debug logging in the configuration and show the logs of Filebeat service? If there is no log, command line debug logs might be useful.

[thps (Torsten Lorenz)]

Mmh i´ve only edited the filebeat.yml. Nothing more...

I´ve set the debug level to "debug". If I try to start it via systemctl- the same issue , no logifle.
If I start the filebeat directly a log will be written. I attached the logfile.

Greetz

(Attachment filebeat.txt is missing)

[thps (Torsten Lorenz)]

Attachement ....