Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch

Elastic Stack Beats
1

Hi,
I have installed ELK just now. And I am trying to get the log file from my ELK machine. Kibana,Elastic and logstash works fine. But , somehow I am not able to get logs using filebeats.

"Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch.."

root@xxxxxxxx~# systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2018-10-22 22:14:03 +03; 17min ago
     Docs: https://www.elastic.co/products/beats/filebeat
  Process: 6350 ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat
 Main PID: 6350 (code=exited, status=1/FAILURE)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

ELK 6.4
Debian

2

Systemd messages are not helpful here. Check filebeat logs or run filebeat on console.

3 
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Main process exited, code=exited, status=1/FAILURE
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Unit entered failed state.
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Failed with result 'exit-code'.
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Service hold-off time over, scheduling restart.
Oct 23 22:17:09 pardus systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Start request repeated too quickly.
Oct 23 22:17:09 pardus systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Unit entered failed state.
Oct 23 22:17:09 pardus systemd[1]: filebeat.service: Failed with result 'exit-code'.
lines 1-16/16 (END)
4 
2018-10-20T23:41:50.453+0300	INFO	instance/beat.go:544	Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2018-10-20T23:41:50.454+0300	INFO	instance/beat.go:551	Beat UUID: 40222142-a424-46c7-9eca-e19fc89f3b52
2018-10-20T23:41:50.454+0300	INFO	[seccomp]	seccomp/seccomp.go:116	Syscall filter successfully installed
2018-10-20T23:41:50.454+0300	INFO	[beat]	instance/beat.go:768	Beat info	{"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "40222142-a424-46c7-9eca-e19fc89f3b52"}}}
2018-10-20T23:41:50.454+0300	INFO	[beat]	instance/beat.go:777	Build info	{"system_info": {"build": {"commit": "e193f6d68b25b7ddbe3a3ed8d60bc07fea1ef800", "libbeat": "6.4.2", "time": "2018-09-26T12:42:46.000Z", "version": "6.4.2"}}}
2018-10-20T23:41:50.454+0300	INFO	[beat]	instance/beat.go:780	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.10.3"}}}
2018-10-20T23:41:50.455+0300	INFO	[beat]	instance/beat.go:784	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-10-20T22:48:17+03:00","containerized":false,"hostname":"pardus","ips":["127.0.0.1/8","::1/128","192.168.1.37/24","fe80::ab5c:371b:46c:6c1a/64"],"kernel_version":"4.16.0-041600-generic","mac_addresses":["18:67:b0:b4:a1:18","48:d2:24:a8:8c:47"],"os":{"family":"","platform":"pardus","name":"Pardus GNU/Linux","version":"17.3 (onyedi)","major":17,"minor":3,"patch":0,"codename":"onyedi"},"timezone":"+03","timezone_offset_sec":10800,"id":"69a4503face14c52b99a624179d0b36e"}}}
2018-10-20T23:41:50.456+0300	INFO	[beat]	instance/beat.go:813	Process info	{"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 4420, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2018-10-20T23:41:49.710+0300"}}}
2018-10-20T23:41:50.456+0300	INFO	instance/beat.go:273	Setup Beat: filebeat; Version: 6.4.2
2018-10-20T23:41:50.456+0300	INFO	instance/beat.go:327	filebeat stopped.
2018-10-20T23:41:50.516+0300	ERROR	instance/beat.go:743	Exiting: error initializing publisher: missing required field accessing 'output.elasticsearch.hosts'
5 
|2018-10-23T23:30:29.155+0300|ERROR|pipeline/output.go:100|Failed to connect to backoff(async(tcp://localhost:5044)): dial tcp 127.0.0.1:5044: connect: connection refused|
|---|---|---|---|
|2018-10-23T23:30:29.155+0300|INFO|pipeline/output.go:93|Attempting to reconnect to backoff(async(tcp://localhost:5044)) with 5 reconnect attempt(s)|`Preformatted text`
6

Please proplery format logs and config files using the </> button in the editor window.

Filebeat logs complain about the output not being configured. The setting output.elasticsearch.hosts is missing. Can you share your config file?

7 
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
    ssl  => false
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}`Preformatted text`
8

Can you share your filebeat config? The error message is complaining about Elasticsearch output in filebeat, yet you send to Logstash.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.