Setting Logstash to run as a service and start on reboot

(Brad) #1

When I log out of an interactive terminal Filebeat and Logstash seem to run in the foreground and stop. Elasticsearch and Kibana services are running OK. This just might be that my Linux fu is not up to scratch but how do I get Logstash and Filebeat to run as a service and start on reboot.

#2

Hi @bradfordaemorton,

how have you installed Filebeat and Logstash and how do you start them? Also, which OS are you using?

(Brad) #3

Thanks A_B,

I am running Ubuntu.

Install process is below:

Logstash

java –version

sudo apt install default-jre

java –version

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch sudo apt-key add -

sudo apt-get install apt-transport-https

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

sudo apt-get update && sudo apt-get install logstash

cd /etc/logstash/

echo “” > first-pipeline.conf

sudo vim first-pipeline.conf

input {

beats {

port => "5044"

}

}

filter {

grok {

match => { "message" => "%{COMBINEDAPACHELOG}"}

}

geoip {

source => "clientip"

}

}

output {

elasticsearch {

hosts => [ "localhost:9200" ]

}

}

cd /usr/share/ logstash /bin

./logstash -f /etc/logstash/first-pipeline.conf --config.test_and_exit

./logstash -f /etc/logstash/first-pipeline.conf --config.reload.automatic

Filebeats

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add –

sudo apt-get install apt-transport-https

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

sudo apt-get update && sudo apt-get install filebeat

sudo update-rc.d filebeat defaults 95 10

cd /etc/filebeat/

echo “” > filebeat1.yml

sudo vim filebeat1.yml

filebeat.prospectors:

- type: log

paths:

- /path/to/file/logstash-tutorial.log

output.logstash:

hosts: ["hostname:5044"]

cd /usr/share/filebeat/bin

sudo ./filebeat -e -c /etc/filebeat/filebeat1.yml -d "publish"

#4

You are definitely starting them in the foreground :smiley:

As you install with apt-get you should get SystemD unit files created (at least on Debian, I think Ubuntu is the same these days).

So if you start Filebeat and Logstash with systemctl start they will start as services in the background

e.g.

# systemctl start logstash.service
# systemctl status logstash.service
● logstash.service - logstash
   Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2019-02-05 16:09:51 UTC; 1 weeks 2 days ago
 Main PID: 27033 (java)
    Tasks: 137 (limit: 4915)
   CGroup: /system.slice/logstash.service
           └─27033 /usr/bin/java -Dfile.encoding=UTF-8 -Djava.awt.headless=true -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseC

You will have to put your configuration files where the unitfile expects them though

e.g. for Logstash

# cat /etc/systemd/system/logstash.service
[Unit]
Description=logstash

[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings=/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=102400

[Install]
WantedBy=multi-user.target

Note --path.settings=/etc/logstash. I think the default location for the logstash conf file is /etc/logstash/logstash.conf (that might be wrong though).

Same goes for Filebeat.

Just to add... I have my Logstash conf in three separate files

# find /etc/logstash/conf.d/
/etc/logstash/conf.d/
/etc/logstash/conf.d/input
/etc/logstash/conf.d/output
/etc/logstash/conf.d/filter

And use them with the SystemD unit file above. There should be no other magic going on.

And for good measure the Filebeat unit file (should be the stock file as far as I know)

# cat /etc/systemd/system/multi-user.target.wants/filebeat.service
[Unit]
Description=filebeat
Documentation=https://www.elastic.co/guide/en/beats/filebeat/current/index.html
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
Restart=always

[Install]
WantedBy=multi-user.target

So put your Filebeat configuration in /etc/filebeat/filebeat.yml and start Filebeat with

# systemctl start filebeat.service
1 Like
(Brad) #5

Thanks for the help. The problems seems to be isolated to logstash - filebeats, elastic and kibana all reboot and start services successfully.

The behaviour I notice is after a reboot the logstash service doesn't start and if manually start the service no data is sent to elastic or visible in Kibana. Maybe it is reading the wrong configuration files?

I can run the following command and it all works: /usr/share/logstash/bin# ./logstash -f /etc/logstash/logstash.yml --config.reload.automatic

Status of logstash service is below:
/usr/share/logstash/bin# systemctl status logstash.service
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset
Active: active (running) since Fri 2019-02-15 09:38:55 UTC; 1s ago
Main PID: 19565 (java)
Tasks: 13 (limit: 2361)
CGroup: /system.slice/logstash.service
└─19565 /usr/bin/java -Xms1g -Xmx1g -XX:+UseParNewGC -XX:+UseConcMark

Feb 15 09:38:55 ubuntu-s-1vcpu-2gb-sgp1-01 systemd[1]: logstash.service: Service
Feb 15 09:38:55 ubuntu-s-1vcpu-2gb-sgp1-01 systemd[1]: logstash.service: Schedul
Feb 15 09:38:55 ubuntu-s-1vcpu-2gb-sgp1-01 systemd[1]: Stopped logstash.
Feb 15 09:38:55 ubuntu-s-1vcpu-2gb-sgp1-01 systemd[1]: Started logstash.

#6

The Logstash config file is not YAML... What happens is you name it /etc/logstash/logstash.conf or /etc/logstash/logstash.d/input?

1 Like
(Brad) #7

OK, I did some more investigation. I have managed to get the service to start and start automatically on reboot. I had to set the service to enable.

I still have a problem as the service constantly restarts and never sends data elasticsearch. There must be a bad config or a file missing.

I did a bit of reading and found that the .conf files are meant to be in /etc/logstash/conf.d but I still have issues. Is there a log file to check for errors? Also, is there a place to download the default .yml and .conf files located in the /etc/logstash/ directory?

#8

I don't think there are any default config files for Logstash...

Start as minimal as possible

Input

input {
  beats {
    port => 5044
  }
}

Filter

filter {}

Output

output {
  elasticsearch {
        hosts => ["10.1.1.1:9200", "10.1.1.2:9200"]
        index => "logstash-%{+YYYY.MM.dd}"
  }
}
(system) closed #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.