# ELK stops to work

**URL:** <https://discuss.elastic.co/t/elk-stops-to-work/25116>\
**Category:** Elasticsearch\
**Created:** [July 8, 2015, 6:02am UTC](https://discuss.elastic.co/t/elk-stops-to-work/25116 "2015-07-08T06:02:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tigryss](https://avatars.discourse-cdn.com/v4/letter/t/c0e974/32.png) [@Tigryss](https://discuss.elastic.co/u/Tigryss)\
**Post date:** [July 8, 2015, 6:02am UTC](https://discuss.elastic.co/t/elk-stops-to-work/25116/1 "2015-07-08T06:02:22Z")

</div>

I started a new installation ELK server(standalone ELK installed on the sameserver) and we noted that kibana doesn't show any of data since 07.06, so i checked the logs and

ls -altr /opt/elkdata/elasticsearch/elasticsearch/nodes/0/indices/  
Jul 3 09:42 logstash-2015.07.03  
Jul 3 09:54 .kibana  
Jul 4 02:00 logstash-2015.07.04  
Jul 5 02:00 logstash-2015.07.05  
Jul 6 02:00 .  
Jul 6 02:00 logstash-2015.07.06  
Jul 7 13:41 ..

but  
ls -altr /opt/elkdata/elasticsearch/logs/  
0 Jun 16 14:47 elasticsearch\_index\_indexing\_slowlog.log  
0 Jun 16 14:47 elasticsearch\_index\_search\_slowlog.log  
4096 Jun 17 10:57 ..  
3461 Jun 17 10:58 elasticsearch.log.2015-06-17  
10311 Jul 3 15:17 elasticsearch.log.2015-07-03  
5731 Jul 4 14:36 elasticsearch.log.2015-07-04  
6689 Jul 5 21:27 elasticsearch.log.2015-07-05  
21343 Jul 6 11:54 elasticsearch.log.2015-07-06  
4096 Jul 7 13:41 .  
2901 Jul 7 13:42 elasticsearch.log

last log entry in elasticsearch.log:

[2015-07-07 13:41:06,533][INFO][cluster.service] [logstash] removed {[logstash-hostname.org-16216-13418][M19k1C2cSDezf5mu1IxvcQ][hostname.org][inet[/192.168.1.2:9301]]{data=false, client=true},}, reason: zen-disco-node\_failed([logstash-hostname.org-16216-13418][M19k1C2cSDezf5mu1IxvcQ][hostname.org][inet[/192.168.1.2:9301]]{data=false, client=true}), reason transport disconnected  
[2015-07-07 13:41:16,921][INFO][node] [logstash] stopping ...  
[2015-07-07 13:41:17,334][INFO][node] [logstash] stopped  
[2015-07-07 13:41:17,335][INFO][node] [logstash] closing ...  
[2015-07-07 13:41:17,357][INFO][node] [logstash] closed  
[2015-07-07 13:41:49,954][INFO][node] [logstash] version[1.6.0], pid[9936], build[cdd3ac4/2015-06-09T13:36:34Z]  
[2015-07-07 13:41:49,955][INFO][node] [logstash] initializing ...  
[2015-07-07 13:41:49,964][INFO][plugins] [logstash] loaded [], sites []  
[2015-07-07 13:41:50,061][INFO][env] [logstash] using [1] data paths, mounts [[/opt/elkdata (/dev/mapper/ccblst01vg-elkdata)]], net usable\_space [814gb], net total\_space [944.8gb], types [ext4]  
[2015-07-07 13:41:53,864][INFO][node] [logstash] initialized  
[2015-07-07 13:41:53,865][INFO][node] [logstash] starting ...  
[2015-07-07 13:41:53,941][INFO][transport] [logstash] bound\_address {inet[/0.0.0.0:9300]}, publish\_address {inet[/192.168.1.2:9300]}  
[2015-07-07 13:41:53,955][INFO][discovery] [logstash] elasticsearch/cLwoLX7sQIyi3zcpnkmiGg  
[2015-07-07 13:41:57,748][INFO][cluster.service] [logstash] new\_master [logstash][cLwoLX7sQIyi3zcpnkmiGg][hostname.org][inet[/192.168.1.2:9300]]{master=true}, reason: zen-disco-join (elected\_as\_master)  
[2015-07-07 13:41:57,937][INFO][http] [logstash] bound\_address {inet[/0.0.0.0:9200]}, publish\_address {inet[/192.168.1.2:9200]}  
[2015-07-07 13:41:57,938][INFO][node] [logstash] started  
[2015-07-07 13:41:58,003][INFO][gateway] [logstash] recovered [5] indices into cluster\_state  
[2015-07-07 13:42:22,620][INFO][cluster.service] [logstash] added {[logstash-hostname.org-10009-13434][\_HLcxrLuRViCZq\_lvscgvQ][hostname.org][inet[/192.168.1.2:9301]]{data=false, client=true},}, reason: zen-disco-receive(join from node[[logstash-hostname.org-10009-13434][\_HLcxrLuRViCZq\_lvscgvQ][hostname.org][inet[/192.168.1.2:9301]]{data=false, client=true}])

LS:  
last entry of logstash.log  
{:timestamp=\>"2015-07-07T13:42:15.728000+0200", :message=\>"Adding pattern",

kibana last entry:  
July 6th 2015, 11:54:50.442 message: 2015-07-06T10:54:01.977;

the box is 32Gb 4CPU 1TB and here is the ls and es start param  
ES:  
/usr/bin/java -Xms16g -Xmx16g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Delasticsearch -Des.pidfile=-Des.index.store.type=memory -Des.path.home=/opt/elasticsearch -cp :/opt/elasticsearch/lib/elasticsearch-1.6.0.jar:/opt/elasticsearch/lib/_:/opt/elasticsearch/lib/sigar/_ -Des.node.name=ccb\_logstash\_tud -Des.config=/opt/elasticsearch/config/elasticsearch.yml -Des.path.home=/opt/elasticsearch -Des.path.logs=/opt/elkdata/elasticsearch/logs -Des.path.data=/opt/elkdata/elasticsearch -Des.path.work=//tmp org.elasticsearch.bootstrap.Elasticsearch

LS:  
/usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Xmx1G -Xss2048k -Djffi.boot.library.path=/opt/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Xbootclasspath/a:/opt/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/logstash/vendor/jruby -Djruby.lib=/opt/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /opt/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent --verbose -f /opt/logstash/conf --log /opt/elkdata/logstash/logstash.log

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:02am UTC](https://discuss.elastic.co/t/elk-stops-to-work/25116/2 "2017-07-06T00:02:57Z")

</div>


