I'm posting more information to this thread in case anyone else runs into this problem. Let me set the stage. I download the 7.6 RPM version and installed it via yum install /path/to/rpm/package. Then followed the documentation to start app-search as a systemd service. I noticed that if I started the service using systemctl start app-search the GUI would not run but if I started the service on the command line the GUI would run. Spoiler alert, this turned out to be a permissions issue.
So here is how I diagnosed what the issue was, first I had to look into the logs for systemd using journalctl. When I looked in the logs I noticed the following lines:
Mar 02 08:14:28 elastic7-appsearch systemd[1]: app-search.service: main process exited, code=exited, status=143/n/a
Mar 02 08:14:28 elastic7-appsearch systemd[1]: Stopped Elastic App Search.
Mar 02 08:14:28 elastic7-appsearch systemd[1]: Unit app-search.service entered failed state.
Mar 02 08:14:28 elastic7-appsearch systemd[1]: app-search.service failed.
Looking further up in the logs I noticed the following:
Mar 02 08:14:11 elastic7-appsearch app-search[13200]: ERROR: org.jruby.exceptions.SystemCallError: (EACCES) Permission denied - /var/log/app-search/app-search.log
Mar 02 08:14:11 elastic7-appsearch app-search[13200]: org.jruby.exceptions.SystemCallError: (EACCES) Permission denied - /var/log/app-search/app-search.log
So app-search couldn't write to the log file when started as a service. I then looked into the log directory at /var/log/app-search and found the following file permissions:
[root@elastic7-appsearch app-search]# ls -l
total 8136
-rw-r--r-- 1 root root 20406 Mar 2 07:56 app-search.log
-rw-r--r-- 1 root root 1048608 Mar 2 07:49 app-search.log.0
-rw-r----- 1 root root 22745 Mar 2 07:56 filebeat
-rw-r----- 1 root root 4563441 Mar 1 12:24 filebeat.1
-rw-r----- 1 root root 9373 Feb 28 13:22 filebeat.2
-rw-r----- 1 root root 8235 Feb 28 12:56 filebeat.3
-rw-r--r-- 1 root root 376393 Mar 2 07:54 logstasher.log
-rw-r--r-- 1 root root 1049360 Feb 28 15:16 logstasher.log.0
-rw-r--r-- 1 root root 748719 Mar 2 07:55 stats.log
-rw-r--r-- 1 root root 441090 Mar 2 07:54 system.log
To test my theory I changed the file permissions to allow read/write for all scopes and changed the file settings to the following:
[root@elastic7-appsearch app-search]# chmod 666 *
[root@elastic7-appsearch app-search]# ls -l
total 8136
-rw-rw-rw- 1 root root 20406 Mar 2 07:56 app-search.log
-rw-rw-rw- 1 root root 1048608 Mar 2 07:49 app-search.log.0
-rw-rw-rw- 1 root root 22745 Mar 2 07:56 filebeat
-rw-rw-rw- 1 root root 4563441 Mar 1 12:24 filebeat.1
-rw-rw-rw- 1 root root 9373 Feb 28 13:22 filebeat.2
-rw-rw-rw- 1 root root 8235 Feb 28 12:56 filebeat.3
-rw-rw-rw- 1 root root 376393 Mar 2 07:54 logstasher.log
-rw-rw-rw- 1 root root 1049360 Feb 28 15:16 logstasher.log.0
-rw-rw-rw- 1 root root 748719 Mar 2 07:55 stats.log
-rw-rw-rw- 1 root root 441090 Mar 2 07:54 system.log
Success, the app-search service was able to start and I was able to run the GUI. I didn't like leaving the permissions open like that so I reverted the permissions to the restricted mode but change the file ownership to app-search:app-search as depicted below:
[root@elastic7-appsearch app-search]# ls -l
total 8220
-rw-r--r-- 1 app-search app-search 44186 Mar 2 08:09 app-search.log
-rw-r--r-- 1 app-search app-search 1048608 Mar 2 07:49 app-search.log.0
-rw-r----- 1 app-search app-search 22745 Mar 2 07:56 filebeat
-rw-r----- 1 app-search app-search 4563441 Mar 1 12:24 filebeat.1
-rw-r----- 1 app-search app-search 9373 Feb 28 13:22 filebeat.2
-rw-r----- 1 app-search app-search 8235 Feb 28 12:56 filebeat.3
-rw-r--r-- 1 app-search app-search 413574 Mar 2 08:09 logstasher.log
-rw-r--r-- 1 app-search app-search 1049360 Feb 28 15:16 logstasher.log.0
-rw-r--r-- 1 app-search app-search 755277 Mar 2 08:09 stats.log
-rw-r--r-- 1 app-search app-search 452562 Mar 2 08:09 system.log
Once again I discovered that the app-search service could start up properly and the GUI launched fine.
In hind sight the issue was probably self inflicted, after installing app-search I initially launched it from the command line to make sure it installed properly. This initial launch was run under the root user, therefore the logs were created with that user. Later trying to run the service which runs under the app-search user caused the service to fail because it couldn't write to the log files.
I thought I post this note in case anyone else runs into this issue.
