Installing self hosted with systemd configuration

I think the docs need to be updated. The current docs only specify how to create the app-search systemd service. There is no mentions of other child services that may need to be created or started. I just followed the instructions to build a 7.6 box and if I start app-search via the service the GUI doesn't start. But if I start it via /usr/share/app-search/bin/app-search then everything starts up and the GUI runs as well.

I installed app-search by downloading the RPM and doing yum install /path/to/downloaded/rpm

I'm posting more information to this thread in case anyone else runs into this problem. Let me set the stage. I download the 7.6 RPM version and installed it via yum install /path/to/rpm/package. Then followed the documentation to start app-search as a systemd service. I noticed that if I started the service using systemctl start app-search the GUI would not run but if I started the service on the command line the GUI would run. Spoiler alert, this turned out to be a permissions issue.

So here is how I diagnosed what the issue was, first I had to look into the logs for systemd using journalctl. When I looked in the logs I noticed the following lines:

Mar 02 08:14:28 elastic7-appsearch systemd[1]: app-search.service: main process exited, code=exited, status=143/n/a
Mar 02 08:14:28 elastic7-appsearch systemd[1]: Stopped Elastic App Search.
Mar 02 08:14:28 elastic7-appsearch systemd[1]: Unit app-search.service entered failed state.
Mar 02 08:14:28 elastic7-appsearch systemd[1]: app-search.service failed.

Looking further up in the logs I noticed the following:

Mar 02 08:14:11 elastic7-appsearch app-search[13200]: ERROR: org.jruby.exceptions.SystemCallError: (EACCES) Permission denied - /var/log/app-search/app-search.log
Mar 02 08:14:11 elastic7-appsearch app-search[13200]: org.jruby.exceptions.SystemCallError: (EACCES) Permission denied - /var/log/app-search/app-search.log

So app-search couldn't write to the log file when started as a service. I then looked into the log directory at /var/log/app-search and found the following file permissions:

[root@elastic7-appsearch app-search]# ls -l
total 8136
-rw-r--r-- 1 root root 20406 Mar 2 07:56 app-search.log
-rw-r--r-- 1 root root 1048608 Mar 2 07:49 app-search.log.0
-rw-r----- 1 root root 22745 Mar 2 07:56 filebeat
-rw-r----- 1 root root 4563441 Mar 1 12:24 filebeat.1
-rw-r----- 1 root root 9373 Feb 28 13:22 filebeat.2
-rw-r----- 1 root root 8235 Feb 28 12:56 filebeat.3
-rw-r--r-- 1 root root 376393 Mar 2 07:54 logstasher.log
-rw-r--r-- 1 root root 1049360 Feb 28 15:16 logstasher.log.0
-rw-r--r-- 1 root root 748719 Mar 2 07:55 stats.log
-rw-r--r-- 1 root root 441090 Mar 2 07:54 system.log

To test my theory I changed the file permissions to allow read/write for all scopes and changed the file settings to the following:

[root@elastic7-appsearch app-search]# chmod 666 *
[root@elastic7-appsearch app-search]# ls -l
total 8136
-rw-rw-rw- 1 root root 20406 Mar 2 07:56 app-search.log
-rw-rw-rw- 1 root root 1048608 Mar 2 07:49 app-search.log.0
-rw-rw-rw- 1 root root 22745 Mar 2 07:56 filebeat
-rw-rw-rw- 1 root root 4563441 Mar 1 12:24 filebeat.1
-rw-rw-rw- 1 root root 9373 Feb 28 13:22 filebeat.2
-rw-rw-rw- 1 root root 8235 Feb 28 12:56 filebeat.3
-rw-rw-rw- 1 root root 376393 Mar 2 07:54 logstasher.log
-rw-rw-rw- 1 root root 1049360 Feb 28 15:16 logstasher.log.0
-rw-rw-rw- 1 root root 748719 Mar 2 07:55 stats.log
-rw-rw-rw- 1 root root 441090 Mar 2 07:54 system.log

Success, the app-search service was able to start and I was able to run the GUI. I didn't like leaving the permissions open like that so I reverted the permissions to the restricted mode but change the file ownership to app-search:app-search as depicted below:

[root@elastic7-appsearch app-search]# ls -l
total 8220
-rw-r--r-- 1 app-search app-search 44186 Mar 2 08:09 app-search.log
-rw-r--r-- 1 app-search app-search 1048608 Mar 2 07:49 app-search.log.0
-rw-r----- 1 app-search app-search 22745 Mar 2 07:56 filebeat
-rw-r----- 1 app-search app-search 4563441 Mar 1 12:24 filebeat.1
-rw-r----- 1 app-search app-search 9373 Feb 28 13:22 filebeat.2
-rw-r----- 1 app-search app-search 8235 Feb 28 12:56 filebeat.3
-rw-r--r-- 1 app-search app-search 413574 Mar 2 08:09 logstasher.log
-rw-r--r-- 1 app-search app-search 1049360 Feb 28 15:16 logstasher.log.0
-rw-r--r-- 1 app-search app-search 755277 Mar 2 08:09 stats.log
-rw-r--r-- 1 app-search app-search 452562 Mar 2 08:09 system.log

Once again I discovered that the app-search service could start up properly and the GUI launched fine.

In hind sight the issue was probably self inflicted, after installing app-search I initially launched it from the command line to make sure it installed properly. This initial launch was run under the root user, therefore the logs were created with that user. Later trying to run the service which runs under the app-search user caused the service to fail because it couldn't write to the log files.

I thought I post this note in case anyone else runs into this issue.

1 Like

Thank you for posting the follow up :+1:

Thanks @bpamiri Worked like a charm :grinning: