Start FileBeat using publish

Elastic Stack Beats
1

Hello everyone,

Im not sure what the difference is between starting filebeat using: service start filebeat AND ./filebeat -e -c /etc/filebeat/filebeat.yml -d "publish"

the latter seems to work better than the first one tho (to me atleast) any idea why and how can i make it run in the background?

Any help or guidance is appreciated, thanks in advance.

Cheers!

2

Hello, thanks for reaching out about filebeat. Could you let us know some more information about your filebeat implementation?

  • Which version of filebeat are you using?
  • Which distribution and version of Linux are you using?
  • Do you get any errors when starting filebeat as service? filebeat defaults to logging to /var/log/filebeat.
  • Which user do you using when running service filebeat start?

Thanks.

3

Hello @Michael_Madden,

im using the latest version 7.3.2
ubuntu 18.04
no errors while running filebeat as a service
im running it as root user.

should i be using a different user, aka regular user?

Thank you

4

Hi,

I wonder if the logs are getting stored in journald? After starting the service with service filebeat start, could you try the following commands?

ps auxw | grep filebeat
systemctl status filebeat
journalctl -u filebeat.service
5

should i run them as root user or regular user?

6

i ran them as root user on one of the servers and i got a huge list of logs for the journalctl command.

7

My main goal is to send the log files using filebeat to logstash. but if i run the filebeat and logstash as a service instead of the whole command it doesnt work so i use the whole command instead: ./filebeat -e -c /etc/filebeat/filebeat.yml -d "publish" and ./logstash -f /etc/logstash/conf.d/logs.conf

is there any way to not have to use the whole command?

8

You can run the ps command as a regular user. I wanted to see if any filebeat processes were running.

9

Does the output of journalctl -u filebeat.service seem related to filebeat? Are there any indicators why the service cannot be started?

Also, is logstash also running on the same virtual machine / server as filebeat?

10

no logstash is on a different server. And yeah the output of journalctl seems related to filebeat. i see this warning in the output tho not sure if its related:

Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning

in my filebeat.yml file i commented out the elasticsearch output because i am using logstash instead.

11

i know this may sound ignorant but is there anyway to configure the service start filebeat command to run in a certain way that i want it to?

12

Hello, I spun up a docker container for Ubuntu 18.04 to ensure the configuration shipped with filebeat 7.3.2 allowed you to run filebeat as a service.

I'd recommend starting with a default configuration to ensure filebeat starts as as a service. It looks like the service should be started with service filebeat start instead of service start filebeat.

root@8acca7f35df6:~# curl https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.3.2-amd64.deb -O -s
root@8acca7f35df6:~# dpkg -i filebeat-7.3.2-amd64.deb
Selecting previously unselected package filebeat.
(Reading database ... 4570 files and directories currently installed.)
Preparing to unpack filebeat-7.3.2-amd64.deb ...
Unpacking filebeat (7.3.2) ...
Setting up filebeat (7.3.2) ..

root@8acca7f35df6:~# service filebeat start
2019-09-23T18:44:57.833Z	INFO	instance/beat.go:607	Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2019-09-23T18:44:57.836Z	INFO	instance/beat.go:615	Beat ID: 141a9a46-88fe-4032-a912-9991f19613f6
2019-09-23T18:44:57.838Z	INFO	[beat]	instance/beat.go:903	Beat info	{"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "141a9a46-88fe-4032-a912-9991f19613f6"}}}
2019-09-23T18:44:57.838Z	INFO	[beat]	instance/beat.go:912	Build info	{"system_info": {"build": {"commit": "5b046c5a97fe1e312f22d40a1f05365621aad621", "libbeat": "7.3.2", "time": "2019-09-06T13:49:32.000Z", "version": "7.3.2"}}}
2019-09-23T18:44:57.838Z	INFO	[beat]	instance/beat.go:915	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.12.4"}}}
2019-09-23T18:44:57.840Z	INFO	[beat]	instance/beat.go:919	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-09-23T17:33:30Z","containerized":true,"name":"8acca7f35df6","ip":["127.0.0.1/8","172.17.0.2/16"],"kernel_version":"4.9.184-linuxkit","mac":["02:42:ac:11:00:02"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.3 LTS (Bionic Beaver)","major":18,"minor":4,"patch":3,"codename":"bionic"},"timezone":"UTC","timezone_offset_sec":0}}}
2019-09-23T18:44:57.841Z	INFO	[beat]	instance/beat.go:948	Process info	{"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 2803, "ppid": 2802, "seccomp": {"mode":"filter"}, "start_time": "2019-09-23T18:44:57.359Z"}}}
2019-09-23T18:44:57.841Z	INFO	instance/beat.go:292	Setup Beat: filebeat; Version: 7.3.2
2019-09-23T18:44:57.841Z	INFO	[index-management]	idxmgmt/std.go:178	Set output.elasticsearch.index to 'filebeat-7.3.2' as ILM is enabled.
2019-09-23T18:44:57.841Z	INFO	elasticsearch/client.go:170	Elasticsearch url: http://localhost:9200
2019-09-23T18:44:57.842Z	INFO	[publisher]	pipeline/module.go:97	Beat name: 8acca7f35df6
Config OK

root@8acca7f35df6:~# ps auxw | grep filebeat
root      2817  0.0  0.0   9300   636 pts/0    S    18:44   0:00 /usr/share/filebeat/bin/filebeat-god -r / -n -p /var/run/filebeat.pid -- /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
root      2818  0.4  0.3 967152 30724 pts/0    Sl   18:44   0:00 /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
13

yeah you see when i type service filebeat start i dont get the same lines that you do, it just doesnt show anything in a way indicating that it started and when i check the status service filebeat status it says that its running fine

14

ok here is the output of ps auxw | grep filebeat on a fresh filebeat installation on a new server without changing the configs whatsoever

root     10502  0.1  0.8 751800 32776 ?        Ssl  19:06   0:00 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
root     10534  0.0  0.0  14856  1108 pts/1    S+   19:06   0:00 grep --color=auto filebeat
15

Thanks. When you issue service filebeat start are you using the default filebeat.yml or a customized version?

I could be possible that the startup process is different in a docker container versus a virtual machine or physical server.

16

no im using the default filebeat.yml file, this server is an AWS EC2 instance btw, could that be why?

17

and if i run service filebeat status it shows me the last 20 lines of logs or so showing what has been done and to what server it trying to connect to (and it does) but thats it, it doesnt send any log files to the logstash server

18

Hello,

Perhaps the filebeat instance is having issues communicating to Logstash? I'd refer to the following faq to debug the communication between filebeat and logstash.

https://www.elastic.co/guide/en/beats/filebeat/7.1/faq.html#connection-problem

19

to be honest i dont think thats the issue, because, if run filebeat using this command ./filebeat -e -c /etc/filebeat/filebeat.yml -d "publish" everything works fine, but one thing that i noticed when i ran the ps auxw | grep filebeat at the end i see this: -path.logs /var/log/filebeat but thats not the log files i had configured in the filebeat.yml file any idea oh how to fix this?

20

FIXED IT!
so apparently after i changed the filebeat.references.yml file to fetch the log files that i wanted then start filebeat normally using service filebeat start it actually sent the files and everything was working perfectly!

Thank you so very much good sir, your awesome!