Elasticsearch 1.4.4 Setup Problem

ali.q · December 10, 2016, 3:30pm

Hi all,

I'm totally new here, so please forgive me if I'm posing in the wrong category or ask a newbie question.

I've setup Elasticsearch 1.4.4 for a SugarCRM system and I'm not quite sure if it is working fine or not. The status of Elasticsearch is ambiguous. When running:

systemctl status elasticsearch.service -l

I get the following output:

elasticsearch.service - Starts and stops a single elasticsearch instance on this system
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled)
   Active: active (running) since Sat 2016-12-10 16:13:20 CET; 2min 44s ago
     Docs: http://www.elasticsearch.org
  Process: 8351 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch/elasticsearch.pid -Des.default.config=$CONF_FILE -Des.default.path.home=$ES_HOME -Des.default.path.logs=$LOG_DIR -Des.default.path.data=$DATA_DIR -Des.default.path.work=$WORK_DIR -Des.default.path.conf=$CONF_DIR (code=exited, status=0/SUCCESS)
 Main PID: 8357 (java)
   CGroup: /system.slice/elasticsearch.service
           └─8357 /bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Des.max-open-files=true -Delasticsearch -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.path.home=/usr/share/elasticsearch -cp :/usr/share/elasticsearch/lib/elasticsearch-1.4.4.jar:/usr/share/elasticsearch/lib/*:/usr/share/elasticsearch/lib/sigar/* -Des.default.config=/etc/elasticsearch/elasticsearch.yml -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch

Dec 10 16:13:20 localhost.localdomain systemd[1]: Unit elasticsearch.service entered failed state.
Dec 10 16:13:20 localhost.localdomain systemd[1]: Starting Starts and stops a single elasticsearch instance on this system...
Dec 10 16:13:20 localhost.localdomain systemd[1]: PID file /var/run/elasticsearch/elasticsearch.pid not readable (yet?) after start.
Dec 10 16:13:20 localhost.localdomain systemd[1]: Started Starts and stops a single elasticsearch instance on this system.
Dec 10 16:13:21 localhost.localdomain elasticsearch[8351]: [2016-12-10 16:13:21,042][INFO ][bootstrap                ] max_open_files [65535]

Now, while it says that the service is running, it also says that it has entered a failed state and elasticsearch.pid not readable (yet?) after start.

I'm not sure if this is normal. I've made a lot of Googling but couldn't find a definite answer.

I'm using CentOS 7

Really appreciate your input and sorry if the question is duplicate or something.
Thanks!

warkolm · December 11, 2016, 3:39am

Have a look in /var/log/elasticsearch/ for more logging info.

ali.q · December 12, 2016, 8:57am

Hi @warkolm

Thanks for your reply.

/var/log/elasticsearch/sugarcrm.log

looks like this:
[2016-12-12 08:50:02,999][INFO ][bootstrap ] max_open_files [65535]
[2016-12-12 08:50:03,040][WARN ][common.jna ] Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swap$
[2016-12-12 08:50:03,126][INFO ][node ] [Thunderclap] version[1.4.4], pid[4018], build[c88f77f/2015-02-19T13:05:36Z]
[2016-12-12 08:50:03,126][INFO ][node ] [Thunderclap] initializing ...
[2016-12-12 08:50:03,130][INFO ][plugins ] [Thunderclap] loaded [], sites []
[2016-12-12 08:50:05,548][INFO ][node ] [Thunderclap] initialized
[2016-12-12 08:50:05,549][INFO ][node ] [Thunderclap] starting ...
[2016-12-12 08:50:05,675][INFO ][transport ] [Thunderclap] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[$
[2016-12-12 08:50:05,699][INFO ][discovery ] [Thunderclap] sugarcrm/0nfaK8cIS0SE5XKZlsw0xQ
[2016-12-12 08:50:09,480][INFO ][cluster.service ] [Thunderclap] new_master [Thunderclap][0nfaK8cIS0SE5XKZlsw0xQ][localhost.localdom$
[2016-12-12 08:50:09,520][INFO ][http ] [Thunderclap] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[$
[2016-12-12 08:50:09,521][INFO ][node ] [Thunderclap] started
[2016-12-12 08:50:09,527][INFO ][gateway ] [Thunderclap] recovered [0] indices into cluster_state
[2016-12-12 09:21:12,127][INFO ][cluster.metadata ] [Thunderclap] [10cfac56ab356b20d08350dc84d0d78b_shared] creating index, cause [ap$
[2016-12-12 09:21:12,686][INFO ][cluster.metadata ] [Thunderclap] [10cfac56ab356b20d08350dc84d0d78b_shared] create_mapping [Contacts]
[2016-12-12 09:21:12,754][INFO ][cluster.metadata ] [Thunderclap] [10cfac56ab356b20d08350dc84d0d78b_shared] create_mapping [Accounts]
[2016-12-12 09:21:12,776][INFO ][cluster.metadata ] [Thunderclap] [10cfac56ab356b20d08350dc84d0d78b_shared] create_mapping [Opportuni$
[2016-12-12 09:21:12,795][INFO ][cluster.metadata ] [Thunderclap] [10cfac56ab356b20d08350dc84d0d78b_shared] create_mapping [Cases]
[2016-12-12 09:21:12,811][INFO ][cluster.metadata ] [Thunderclap] [10cfac56ab356b20d08350dc84d0d78b_shared] create_mapping [Notes]

systemctl status elasticsearch.service -l

Looks like this:

● elasticsearch.service - Starts and stops a single elasticsearch instance on this system
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2016-12-12 08:50:02 CET; 1h 1min ago
Docs: http://www.elasticsearch.org
Process: 4012 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -d -p /var/run/elasticsearch/elasticsearch.pid -Des.default.config=$CONF_FILE -Des.default.path.home=$ES_HOME -Des.default.path.logs=$LOG_DIR -Des.default.path.data=$DATA_DIR -Des.default.path.work=$WORK_DIR -Des.default.path.conf=$CONF_DIR (code=exited, status=0/SUCCESS)
Main PID: 4018 (java)
CGroup: /system.slice/elasticsearch.service
└─4018 /bin/java -Xms1g -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Des.max-open-files=true -Delasticsearch -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.path.home=/usr/share/elasticsearch -cp :/usr/share/elasticsearch/lib/elasticsearch-1.4.4.jar:/usr/share/elasticsearch/lib/:/usr/share/elasticsearch/lib/sigar/ -Des.default.config=/etc/elasticsearch/elasticsearch.yml -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch

Dec 12 08:50:02 localhost.localdomain systemd[1]: Starting Starts and stops a single elasticsearch instance on this system...
Dec 12 08:50:02 localhost.localdomain systemd[1]: PID file /var/run/elasticsearch/elasticsearch.pid not readable (yet?) after start.
Dec 12 08:50:02 localhost.localdomain systemd[1]: Started Starts and stops a single elasticsearch instance on this system.

I'm still concerned about the "elasticsearch.pid not readable (yet?) after start" message.

FYI: When I run curl -X GET 'http://localhost:9200' I get the following reponse which looks fine:

{
"status" : 200,
"name" : "Thunderclap",
"cluster_name" : "sugarcrm",
"version" : {
"number" : "1.4.4",
"build_hash" : "c88f77ffc81301dfa9dfd81ca2232f09588bd512",
"build_timestamp" : "2015-02-19T13:05:36Z",
"build_snapshot" : false,
"lucene_version" : "4.10.3"
},
"tagline" : "You Know, for Search"
}

Any input or help is much appreciated.
Thanks!

warkolm · December 12, 2016, 11:35pm

Everything looks ok, other than that PID issue, but it doesn't seem to be causing major issues.

ali.q · December 13, 2016, 6:46am

Thanks for your response @warkolm

Is there a way I can make sure the PID message is totally normal?
How can I correct it? Or isn't that necessary at all?

Thanks again!

jprante · December 13, 2016, 12:28pm

See https://github.com/elastic/elasticsearch/issues/11594 for the /var/run/elasticsearch/elasticsearch.pid issue.

ali.q · December 13, 2016, 12:55pm

Thanks @jprante

I'm using elasticsearch 1.4.4 (I need to use it since it is a requirement for SugarCRM 7.7)

When running ls -al /var/run/elasticsearch I get the following:

drwxr-xr-x  2 elasticsearch elasticsearch   60 Dec 13 11:50 .
drwxr-xr-x 38 root          root          1200 Dec 13 11:52 ..
-rw-r--r--  1 elasticsearch elasticsearch    3 Dec 13 11:50 elasticsearch.pid

So, the folder and file both belong to elasticsearch. I don't know if the rights are enough?
Looking at the page you pointed to I see that this issue was introduced in version 1.6.

I'm concerned about the Dec 12 08:50:02 localhost.localdomain systemd[1]: PID file /var/run/elasticsearch/elasticsearch.pid not readable (yet?) after start. message that I get when I run a systemctl status elasticsearch -l

I'm not sure whether this is normal? I'm facing a lot of issues with SugarCRM and want to make sure elastic is working fine.

When I do a curl -X GET 'http://localhost:9200' I get the standard response:
{
"status" : 200,
"name" : "SugarCRM Node",
"cluster_name" : "sugarcrm",
"version" : {
"number" : "1.4.4",
"build_hash" : "c88f77ffc81301dfa9dfd81ca2232f09588bd512",
"build_timestamp" : "2015-02-19T13:05:36Z",
"build_snapshot" : false,
"lucene_version" : "4.10.3"
},
"tagline" : "You Know, for Search"
}

Is there a way to make sure elastic is 100 percent functional and not making any issues?
How can I make sure that the pid not readable message isn't causing any problems.

Thanks a lot!

jprante · December 13, 2016, 2:22pm

As @warkolm said, the PID issue is just a glitch, it does not influence Elastiscearch. It is a warning message by the systemctl start script, not by Elastcsearch. In the answers of https://github.com/elastic/elasticsearch/issues/11594 you can learn how to change /var/run/elasticsearch/elasticsearch.pid into /var/run/elasticsearch.pid in the systemctl script. This will make the warning disappear.

ali.q · December 13, 2016, 2:45pm

Thanks a lot @jprante & @warkolm

You saved my day. Now, I'm at least sure about elastic and can look elsewhere!

Have a nice day and thanks again

system · January 10, 2017, 2:45pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.