Confused and concerned

[QuizzyRascal]

Hi all,
I spent my weekend building a CentOS 7 ELK (6.2.1) implementation to collect syslogs (redirecting 514 udp/tcp up to 5000). Everything in the end was bullet proof and I came into work feeling great.

It didn't last.

I have a slightly different CentOS 7 build (hardened a bit) and the exact same steps resulted in all services running but when I do a netstat -ntlp I don't see port 5000 listening.

So I ran logstash from the command line like this:
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/syslog.conf --verbose

I got the following error message:
[ERROR] 2018-02-19 16:46:51.171 [LogStash::Runner] Logstash - java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (LoadError) Could not load FFI Provider: (NotImplementedError) FF

I googled and it said that I might fix this by ensuring /tmp didn't have noexec as a flag.

I changed that and now from the command line the logstash listener runs.

But not from sudo SystemCtl start logstash (even after a daemon-reload and reboots). If I run netstat -ntl repeatedly I see port 9600 pop up and then disappear but never port 5000 which is set in my syslog.conf file in conf.d

I have been googling away and apparently people have resolved this by setting
-Djava.io.tmpdir to a writable directory.

Problem is I have no idea where to actually set this, can anyone tell me where to set it please?

I need a little help as I just don't know what to do to get the service option to run.

Can anyone suggest a way to make logstash listen from the service please?

All help is truly appreciated as I am a newbie,

Thanks,
QR

[QuizzyRascal]

SOLVED!
The issue appears to be a combination of things:
The build of CentOS 7 I have been given has noexec on /tmp which was an issue that stopped command line execution of logstash.
The permissions around /etc/logstash and /etc/logstash.conf.d
These were set to root only, which might be fine for others but for me was a bad do.
I executed the following command:
setfacl -Rm d:u:logstash:rwX,u:logstash:rwX /etc/logstash
Which set rw on the directory and conf.d
Before when I was running Verbose from the command line I was getting a warning that the service could not find logstash.yml but it would start using a default.
When starting from SystemCtl the service would start but not listen
So after the line above I rebooted and made sure the logstash service was running and did a netstat -ntl
And to my delight discovered that logstash was now listening on port 5000 as hoped and that the listener persisted.
So at the moment this is a solved issue. I hope this helps someone else