Service stop each 15 or 20 minutes

john_felipe_urrego_m · May 15, 2015, 5:45pm

274.9 MB of 2 GB Used / 1.73 GB Free

And each 20 minutes i need do this:

sudo /etc/init.d/elasticsearch start
curl -XGET 'localhost:9200'

Please give me instructions
Tnks

tylerjl · May 15, 2015, 8:18pm

Hi there!

The first step here would be to find information regarding why elasticsearch is stopping. Look in /var/log/elasticsearch for logs, you should find logs that match your cluster name. Start at the most recent logs and you should find information regarding why elasticsearch stopped.

Also, what Linux distribution are you running, and are you running this machine locally or with a cloud provider like Amazon EC2?

john_felipe_urrego_m · May 18, 2015, 6:16pm

/var/log/elasticsearch/elasticsearch.log:

[2015-05-15 13:41:59,546][INFO ][node ] [Polaris] version[1.5.2], pid[24783], build[62ff986/2015-04-27T09:21:06Z]
[2015-05-15 13:41:59,649][INFO ][node ] [Polaris] initializing ...
[2015-05-15 13:41:59,702][INFO ][plugins ] [Polaris] loaded [], sites []
[2015-05-15 13:42:13,185][INFO ][node ] [Polaris] initialized
[2015-05-15 13:42:13,186][INFO ][node ] [Polaris] starting ...
[2015-05-15 13:42:13,482][INFO ][transport ] [Polaris] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/198.38.86.148:9300]}
[2015-05-15 13:42:13,740][INFO ][discovery ] [Polaris] elasticsearch/zk4QOPRzTx-jkIAnofS0iw
[2015-05-15 13:42:17,519][INFO ][cluster.service ] [Polaris] new_master [Polaris][zk4QOPRzTx-jkIAnofS0iw][vps][inet[/198.38.86.148:9300]], reason: zen-disco-join (elected_as_master)
[2015-05-15 13:42:17,578][INFO ][http ] [Polaris] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/198.38.86.148:9200]}
[2015-05-15 13:42:17,578][INFO ][node ] [Polaris] started
[2015-05-15 13:42:18,852][INFO ][gateway ] [Polaris] recovered [2] indices into cluster_state

/var/log/elasticsearch/elasticsearch.log.2015-05-14:

[2015-05-14 14:04:47,459][INFO ][node [2015-05-14 14:04:47,460][INFO ][node [2015-05-14 14:04:47,473][INFO ][plugins [2015-05-14 14:05:23,613][INFO ][node [2015-05-14 14:05:23,614][INFO ][node [2015-05-14 14:05:24,053][INFO ][transport [2015-05-14 14:05:24,073][INFO ][discovery [2015-05-14 14:05:27,162][INFO ][cluster.service [2015-05-14 14:05:27,177][INFO ][http [2015-05-14 14:05:27,181][INFO ][node [2015-05-14 21:20:43,419][INFO ][node [2015-05-14 21:20:43,436][INFO ][node [2015-05-14 21:20:43,527][INFO ][node [2015-05-14 21:20:43,527][INFO ][node [2015-05-14 21:20:43,605][INFO ][plugins [2015-05-14 21:20:43,605][INFO ][plugins [2015-05-14 21:20:51,293][INFO ][node [2015-05-14 21:20:51,294][INFO ][node [2015-05-14 21:20:51,323][INFO ][node [2015-05-14 21:20:51,324][INFO ][node [2015-05-14 21:20:51,704][INFO ][transport [2015-05-14 21:20:51,713][INFO ][transport [2015-05-14 21:20:51,843][INFO ][discovery [2015-05-14 21:20:51,844][INFO ][discovery [2015-05-14 21:20:54,902][INFO ][cluster.service [2015-05-14 21:20:54,908][INFO ][cluster.service [2015-05-14 21:20:54,925][INFO ][cluster.service [2015-05-14 21:20:54,929][INFO ][http [2015-05-14 21:20:54,929][INFO ][node [2015-05-14 21:20:54,951][INFO ][http [2015-05-14 21:20:54,952][INFO ][node [2015-05-14 21:20:56,635][INFO ][gateway [2015-05-14 21:22:23,128][INFO ][cluster.metadata [2015-05-14 21:22:23,419][INFO ][cluster.metadata [2015-05-14 21:22:23,952][INFO ][cluster.metadata [2015-05-14 21:22:24,483][INFO ][cluster.metadata [2015-05-14 21:23:16,356][INFO ][cluster.metadata [2015-05-14 21:23:16,994][INFO ][cluster.metadata [2015-05-14 21:23:17,455][INFO ][cluster.metadata [2015-05-14 22:57:31,374][INFO ][discovery.zen [2015-05-14 22:57:31,390][WARN ][discovery.zen [2015-05-14 22:57:31,391][INFO ][cluster.service [2015-05-14 22:57:35,286][INFO ][cluster.service ] [Shard] version[1.5.2], pid[15066], build[62ff986/2015-04-27T09:21:06Z]
] [Shard] initializing ...
] [Shard] loaded [], sites []
] [Shard] initialized
] [Shard] starting ...
] [Shard] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/198.38.86.148:9301]}
] [Shard] elasticsearch/av4OPSwvQj6OQ0t-rCGfcw
] [Shard] detected_master [Carlos Lobo][FpOGGoj6RGyFGZT2-qOpLg][vps][inet[/198.38.86.148:9300]], added {[Carlos Lobo][FpOGGoj6RGyFGZT2-qOpLg][vps][inet[/198.38.86.148:9300]],}, reason: zen-disco-receive(from master [[Carlos Lobo][FpOGGoj6RGyFGZT2-qOpLg][vps][inet[/198.38.86.148:9300]]])
] [Shard] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/198.38.86.148:9200]}
] [Shard] started
] [Scanner] version[1.5.2], pid[19203], build[62ff986/2015-04-27T09:21:06Z]
] [Aleksei Sytsevich] version[1.5.2], pid[19188], build[62ff986/2015-04-27T09:21:06Z]
] [Scanner] initializing ...
] [Aleksei Sytsevich] initializing ...
] [Scanner] loaded [], sites []
] [Aleksei Sytsevich] loaded [], sites []
] [Scanner] initialized
] [Scanner] starting ...
] [Aleksei Sytsevich] initialized
] [Aleksei Sytsevich] starting ...
] [Aleksei Sytsevich] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/198.38.86.148:9300]}
] [Scanner] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/198.38.86.148:9301]}
] [Scanner] elasticsearch/WsMtHcI9S6mvvmw68bAGgQ
] [Aleksei Sytsevich] elasticsearch/iDNXQbGaSQCbtDBgVISQCg
] [Scanner] new_master [Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]], reason: zen-disco-join (elected_as_master)
] [Scanner] added {[Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]],}, reason: zen-disco-receive(join from node[[Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]]])
] [Aleksei Sytsevich] detected_master [Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]], added {[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]],}, reason: zen-disco-receive(from master [[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]]])
] [Scanner] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/198.38.86.148:9200]}
] [Scanner] started
] [Aleksei Sytsevich] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/198.38.86.148:9201]}
] [Aleksei Sytsevich] started
] [Scanner] recovered [1] indices into cluster_state
] [Scanner] [sayit] deleting index
] [Scanner] [sayit] creating index, cause [api], templates [], shards [5]/[1], mappings []
] [Scanner] [sayit] create_mapping [modelresult]
] [Scanner] [sayit] update_mapping [modelresult] (dynamic)
] [Scanner] [sayit_write] creating index, cause [api], templates [], shards [5]/[1], mappings []
] [Scanner] [sayit_write] create_mapping [modelresult]
] [Scanner] [sayit_write] update_mapping [modelresult] (dynamic)
] [Aleksei Sytsevich] master_left [[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]]], reason [transport disconnected]
] [Aleksei Sytsevich] master left (reason = transport disconnected), current nodes: {[Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]],}
] [Aleksei Sytsevich] removed {[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]],}, reason: zen-disco-master_failed ([Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]])
] [Aleksei Sytsevich] new_master [Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]], reason: zen-disco-join (elected_as_master)

john_felipe_urrego_m · May 18, 2015, 6:23pm

felipe@vps:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

john_felipe_urrego_m · May 19, 2015, 4:09pm

Hello @tylerjl, please help me to solve this issue

tylerjl · May 19, 2015, 7:15pm

The root cause isn't immediately apparent from the logs, so some additional questions to consider:

Are you using a cloud hosting provider? If you are and you have not modified your configuration file to disable multicast, then your cluster may be communicating with other clusters, causing problems. This could be the cause of the messages about masters leaving the cluster in your logs.
What is your cluster topology? That is, how many nodes do you have, and where are they? If you are only running one machine, the logs indicate more than one is in the cluster, which is a problem (your cluster is communicating with the outside world)
If there really are no stack traces or error messages in the elasticsearch logs, you may want to look in /var/log/syslog to see if there are any messages about the kernel killing elasticsearch for some reason (OOM killer, etc.)

john_felipe_urrego_m · May 21, 2015, 3:36am

@tylerjl i do this nothing more:
apt-get install git curl nano
git clone https://gist.github.com/8265589463915837429d.git
clone this https://gist.github.com/ricardo-rossi/8265589463915837429d
cd 8265589463915837429d/
./ElasticSearch.sh 1.5.2

I have Droplets in digitalocean:
Name Status Memory Disk Region
actasfelipe Active 2 GB 40 GB nyc3

I dont disable multicast only ./ElasticSearch.sh 1.5.2

tylerjl · May 21, 2015, 4:08pm

The script you used to install elasticsearch is from a third-party and installing elasticsearch in an unsupported way (it's using scripts from a repository that is marked as deprecated.) At this point it's hard to tell what state elasticsearch is in because everything the scripts have done.

It's probably best to start fresh and follow the guide for installing with APT repos, as that's the way we distribute elasticsearch for Linux distributions. I would use new machines, as the previous installation may have done things oddly and could possibly conflict in some way.

Follow that guide to install elasticsearch, then please read the documentation on important configuration changes before proceeding. There's some important items in there, particularly disabling multicast: if you do not take that step, elasticsearch will send broadcasts advertising its presence and cluster with anyone that replies, which exposes your machine to (possibly) malicious outsiders.

john_felipe_urrego_m · May 23, 2015, 9:10pm

@tylerjl only set discovery.zen.ping.multicast.enabled: false?

tylerjl · May 26, 2015, 4:49pm

That is how to disable multicast, yes. But please consider installing elasticsearch in the supported way for APT distributions if you still encounter problems.