Run Logstash in the background on Ubuntu

[hHelen (Helen)]

I already have logstash running on the termial with this command:

sudo -Hu logstash /usr/share/logstash/bin/logstash --path.settings=/etc/logstash -f logstash-simple.conf

I can see files scrolling past on the terminal, so closing it will kill the service.
So, my question is how do I run my command above so that my terminal is free - and I can continue working on it? Basically, I need a way to start this command in the background:

sudo -Hu logstash /usr/share/logstash/bin/logstash --path.settings=/etc/logstash -f logstash-simple.conf

Any help would be appreciated.
Helen

[magnusbaeck (Magnus Bäck)]

https://www.elastic.co/guide/en/logstash/current/running-logstash.html

[hHelen (Helen)]

Thanks, I saw that link, but no where does it explain how to point to a config file. It assumes that that one is using the default config - at least that's my understanding.

So, how does running this command below know where my config file is?

sudo systemctl start logstash.service

I'm sorry if I sound daft but it doesn't show how to run a specific config file.

Thanks again for your help.
Helen

[Jevgenij]

Trying to solve the exact same question.

As far as I see, you can't pass parameters to service while using systemctl.

So the only way I found so far is using logstash.yml.

I haven't tried it yet, though.

[Jevgenij]

Ok, I've created logstash.yml under my LS_HOME/config directory.

You can find yours by running in terminal:

sudo cat /etc/default/logstash

logstash.yml file content in my case is:

pipeline.workers: 1
path.config: "/home/me/logstash.conf"

And it works perfectly while running Logstash with no parameters set from command line:

sudo -Hu logstash /usr/share/logstash/bin/logstash

But no luck with sudo systemctl start logstash.service.

It says:

[2018-07-25T12:48:56,717][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.2"}
[2018-07-25T12:48:56,837][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*.conf"}
[2018-07-25T12:48:56,847][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2018-07-25T12:48:57,103][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Next thing I've done - I've copied my logstash.conf to /etc/logstash/conf.d/ and got Logstash service working. But its results were like the pipeline.workers: 1 wasn't set.

So it looks like Logstash service completely ignores logstash.yml settings file.

Any ideas?

[tgaudin]

When you install Logstash with the deb or rpm package, its configuration is in /etc/logstash/logstash.yml, with the pipelines definition in /etc/logstash/pipelines.yml.
It is then picked up automatically when you start Logstash with systemctl start logstash.service

[Jevgenij]

@tgaudin, thanks for collaboration!

Just moved my logstash.yml to /etc/logstash and got logstash service running correctly.

The only issue now is that running Logstash from command line fails with warning:

Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults

Have no idea why can't it find logstash.yml in /etc/logstash.

But anyway, I believe that topicstarters question was answered.

[tgaudin]

You can run it with --path.settings /etc/logstash on the CLI then, that's what I use to check if my configuration is syntactically correct (/usr/share/logstash/bin/logstash -t --path.settings /etc/logstash/ --config.debug --log.level debug).

But I think I see this warning all the time, it still picks up the configuration and runs correctly afterwards.

[hHelen (Helen)]

Thanks for your comment and sample code.
I will try your suggestion and see if that helps. You would expect this to be very simple but instead people are just pointing you to links that doesn't actually address your issue.

Thanks again, will let you know if it works for me.

Helen

[magnusbaeck (Magnus Bäck)]

Thanks, I saw that link, but no where does it explain how to point to a config file. It assumes that that one is using the default config - at least that's my understanding.

Well, if you start Logstash or any other Linux service via e.g. systemd it'll look in the predefined directories for configuration files. There's no way to pass an option to the service startup command to point Logstash elsewhere (however, if you change configuration files that systemd pays attention to you can make it run other files).

See also Logstash Directory Layout | Logstash Reference [8.11] | Elastic.