Service stop each 15 or 20 minutes


(John Felipe Urrego Mejia) #1

274.9 MB of 2 GB Used / 1.73 GB Free

And each 20 minutes i need do this:

sudo /etc/init.d/elasticsearch start
curl -XGET 'localhost:9200'

Please give me instructions
Tnks


(Tyler Langlois) #2

Hi there!

The first step here would be to find information regarding why elasticsearch is stopping. Look in /var/log/elasticsearch for logs, you should find logs that match your cluster name. Start at the most recent logs and you should find information regarding why elasticsearch stopped.

Also, what Linux distribution are you running, and are you running this machine locally or with a cloud provider like Amazon EC2?


(John Felipe Urrego Mejia) #3

/var/log/elasticsearch/elasticsearch.log:

[2015-05-15 13:41:59,546][INFO ][node ] [Polaris] version[1.5.2], pid[24783], build[62ff986/2015-04-27T09:21:06Z]
[2015-05-15 13:41:59,649][INFO ][node ] [Polaris] initializing ...
[2015-05-15 13:41:59,702][INFO ][plugins ] [Polaris] loaded [], sites []
[2015-05-15 13:42:13,185][INFO ][node ] [Polaris] initialized
[2015-05-15 13:42:13,186][INFO ][node ] [Polaris] starting ...
[2015-05-15 13:42:13,482][INFO ][transport ] [Polaris] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/198.38.86.148:9300]}
[2015-05-15 13:42:13,740][INFO ][discovery ] [Polaris] elasticsearch/zk4QOPRzTx-jkIAnofS0iw
[2015-05-15 13:42:17,519][INFO ][cluster.service ] [Polaris] new_master [Polaris][zk4QOPRzTx-jkIAnofS0iw][vps][inet[/198.38.86.148:9300]], reason: zen-disco-join (elected_as_master)
[2015-05-15 13:42:17,578][INFO ][http ] [Polaris] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/198.38.86.148:9200]}
[2015-05-15 13:42:17,578][INFO ][node ] [Polaris] started
[2015-05-15 13:42:18,852][INFO ][gateway ] [Polaris] recovered [2] indices into cluster_state

/var/log/elasticsearch/elasticsearch.log.2015-05-14:

[2015-05-14 14:04:47,459][INFO ][node ] [Shard] version[1.5.2], pid[15066], build[62ff986/2015-04-27T09:21:06Z]
[2015-05-14 14:04:47,460][INFO ][node ] [Shard] initializing ...
[2015-05-14 14:04:47,473][INFO ][plugins ] [Shard] loaded [], sites []
[2015-05-14 14:05:23,613][INFO ][node ] [Shard] initialized
[2015-05-14 14:05:23,614][INFO ][node ] [Shard] starting ...
[2015-05-14 14:05:24,053][INFO ][transport ] [Shard] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/198.38.86.148:9301]}
[2015-05-14 14:05:24,073][INFO ][discovery ] [Shard] elasticsearch/av4OPSwvQj6OQ0t-rCGfcw
[2015-05-14 14:05:27,162][INFO ][cluster.service ] [Shard] detected_master [Carlos Lobo][FpOGGoj6RGyFGZT2-qOpLg][vps][inet[/198.38.86.148:9300]], added {[Carlos Lobo][FpOGGoj6RGyFGZT2-qOpLg][vps][inet[/198.38.86.148:9300]],}, reason: zen-disco-receive(from master [[Carlos Lobo][FpOGGoj6RGyFGZT2-qOpLg][vps][inet[/198.38.86.148:9300]]])
[2015-05-14 14:05:27,177][INFO ][http ] [Shard] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/198.38.86.148:9200]}
[2015-05-14 14:05:27,181][INFO ][node ] [Shard] started
[2015-05-14 21:20:43,419][INFO ][node ] [Scanner] version[1.5.2], pid[19203], build[62ff986/2015-04-27T09:21:06Z]
[2015-05-14 21:20:43,436][INFO ][node ] [Aleksei Sytsevich] version[1.5.2], pid[19188], build[62ff986/2015-04-27T09:21:06Z]
[2015-05-14 21:20:43,527][INFO ][node ] [Scanner] initializing ...
[2015-05-14 21:20:43,527][INFO ][node ] [Aleksei Sytsevich] initializing ...
[2015-05-14 21:20:43,605][INFO ][plugins ] [Scanner] loaded [], sites []
[2015-05-14 21:20:43,605][INFO ][plugins ] [Aleksei Sytsevich] loaded [], sites []
[2015-05-14 21:20:51,293][INFO ][node ] [Scanner] initialized
[2015-05-14 21:20:51,294][INFO ][node ] [Scanner] starting ...
[2015-05-14 21:20:51,323][INFO ][node ] [Aleksei Sytsevich] initialized
[2015-05-14 21:20:51,324][INFO ][node ] [Aleksei Sytsevich] starting ...
[2015-05-14 21:20:51,704][INFO ][transport ] [Aleksei Sytsevich] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/198.38.86.148:9300]}
[2015-05-14 21:20:51,713][INFO ][transport ] [Scanner] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/198.38.86.148:9301]}
[2015-05-14 21:20:51,843][INFO ][discovery ] [Scanner] elasticsearch/WsMtHcI9S6mvvmw68bAGgQ
[2015-05-14 21:20:51,844][INFO ][discovery ] [Aleksei Sytsevich] elasticsearch/iDNXQbGaSQCbtDBgVISQCg
[2015-05-14 21:20:54,902][INFO ][cluster.service ] [Scanner] new_master [Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]], reason: zen-disco-join (elected_as_master)
[2015-05-14 21:20:54,908][INFO ][cluster.service ] [Scanner] added {[Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]],}, reason: zen-disco-receive(join from node[[Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]]])
[2015-05-14 21:20:54,925][INFO ][cluster.service ] [Aleksei Sytsevich] detected_master [Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]], added {[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]],}, reason: zen-disco-receive(from master [[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]]])
[2015-05-14 21:20:54,929][INFO ][http ] [Scanner] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/198.38.86.148:9200]}
[2015-05-14 21:20:54,929][INFO ][node ] [Scanner] started
[2015-05-14 21:20:54,951][INFO ][http ] [Aleksei Sytsevich] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/198.38.86.148:9201]}
[2015-05-14 21:20:54,952][INFO ][node ] [Aleksei Sytsevich] started
[2015-05-14 21:20:56,635][INFO ][gateway ] [Scanner] recovered [1] indices into cluster_state
[2015-05-14 21:22:23,128][INFO ][cluster.metadata ] [Scanner] [sayit] deleting index
[2015-05-14 21:22:23,419][INFO ][cluster.metadata ] [Scanner] [sayit] creating index, cause [api], templates [], shards [5]/[1], mappings []
[2015-05-14 21:22:23,952][INFO ][cluster.metadata ] [Scanner] [sayit] create_mapping [modelresult]
[2015-05-14 21:22:24,483][INFO ][cluster.metadata ] [Scanner] [sayit] update_mapping [modelresult] (dynamic)
[2015-05-14 21:23:16,356][INFO ][cluster.metadata ] [Scanner] [sayit_write] creating index, cause [api], templates [], shards [5]/[1], mappings []
[2015-05-14 21:23:16,994][INFO ][cluster.metadata ] [Scanner] [sayit_write] create_mapping [modelresult]
[2015-05-14 21:23:17,455][INFO ][cluster.metadata ] [Scanner] [sayit_write] update_mapping [modelresult] (dynamic)
[2015-05-14 22:57:31,374][INFO ][discovery.zen ] [Aleksei Sytsevich] master_left [[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]]], reason [transport disconnected]
[2015-05-14 22:57:31,390][WARN ][discovery.zen ] [Aleksei Sytsevich] master left (reason = transport disconnected), current nodes: {[Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]],}
[2015-05-14 22:57:31,391][INFO ][cluster.service ] [Aleksei Sytsevich] removed {[Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]],}, reason: zen-disco-master_failed ([Scanner][WsMtHcI9S6mvvmw68bAGgQ][vps][inet[/198.38.86.148:9301]])
[2015-05-14 22:57:35,286][INFO ][cluster.service ] [Aleksei Sytsevich] new_master [Aleksei Sytsevich][iDNXQbGaSQCbtDBgVISQCg][vps][inet[/198.38.86.148:9300]], reason: zen-disco-join (elected_as_master)


(John Felipe Urrego Mejia) #4

felipe@vps:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty


(John Felipe Urrego Mejia) #5

Hello @tylerjl, please help me to solve this issue


(Tyler Langlois) #6

The root cause isn't immediately apparent from the logs, so some additional questions to consider:

  • Are you using a cloud hosting provider? If you are and you have not modified your configuration file to disable multicast, then your cluster may be communicating with other clusters, causing problems. This could be the cause of the messages about masters leaving the cluster in your logs.
  • What is your cluster topology? That is, how many nodes do you have, and where are they? If you are only running one machine, the logs indicate more than one is in the cluster, which is a problem (your cluster is communicating with the outside world)
  • If there really are no stack traces or error messages in the elasticsearch logs, you may want to look in /var/log/syslog to see if there are any messages about the kernel killing elasticsearch for some reason (OOM killer, etc.)

(John Felipe Urrego Mejia) #8

@tylerjl i do this nothing more:
apt-get install git curl nano
git clone https://gist.github.com/8265589463915837429d.git
clone this https://gist.github.com/ricardo-rossi/8265589463915837429d
cd 8265589463915837429d/
./ElasticSearch.sh 1.5.2

I have Droplets in digitalocean:
Name Status Memory Disk Region
actasfelipe Active 2 GB 40 GB nyc3

I dont disable multicast only ./ElasticSearch.sh 1.5.2


(Tyler Langlois) #9

The script you used to install elasticsearch is from a third-party and installing elasticsearch in an unsupported way (it's using scripts from a repository that is marked as deprecated.) At this point it's hard to tell what state elasticsearch is in because everything the scripts have done.

It's probably best to start fresh and follow the guide for installing with APT repos, as that's the way we distribute elasticsearch for Linux distributions. I would use new machines, as the previous installation may have done things oddly and could possibly conflict in some way.

Follow that guide to install elasticsearch, then please read the documentation on important configuration changes before proceeding. There's some important items in there, particularly disabling multicast: if you do not take that step, elasticsearch will send broadcasts advertising its presence and cluster with anyone that replies, which exposes your machine to (possibly) malicious outsiders.


(John Felipe Urrego Mejia) #10

@tylerjl only set discovery.zen.ping.multicast.enabled: false?


(Tyler Langlois) #11

That is how to disable multicast, yes. But please consider installing elasticsearch in the supported way for APT distributions if you still encounter problems.


(system) #12