Can't load kibana or curl :9200 after apt upgrade

Elastic Stack Elasticsearch
1

yesterday I did an apt upgrade on my system and it seems to have broken everything. I can't load kibana url at least. I only just installed the elastic stack on Monday (March 25, 2019).
I have today noticed that the 6.7.0 es was released on Tuesday.

Update: Just for kicks I reinstalled all latest packages from .deb, so they should all be compatible, right? Still get "Kibana server is not ready yet" in the browser - this is not helpful. Anyone come across a similar scenario?

I see a similar post that referred to a link that described this:

  1. Stop Kibana
  2. Delete the .kibana_1 and .kibana_2 indices that were create
    I have seen this all over the place, with no explanation how to delete those indices.

ok ran: curl -XDELETE http://192.168.0.92:9200/.kibana_1 and got: (localhost is the same)
curl: (7) Failed to connect to 192.168.0.92 port 9200: Connection refused

So the url is unreachable. Any ideas why?
(I can't post any logs until April 1 - which ones should I post?)

2

If you're using the default config values, port 9200 should be your Elasticsearch. It looks like it's not running.

Try running:

systemctl status elasticsearch

and see if it is running. My guess is it isn't. If it's not try running:

sudo journalctl -xe

And see what the error(s) is/are. That should point you in the right direction to resolving the issue(s)

3

Thanks Mike (@mdrisser). Will look on Monday. I'm sure I have run sudo service elasticsearch start even several times, but there may be an underlying problem. Thanks

4

Morning @mdrisser I have run the two commands below and got the following results:

root@shield-VM:~# service elasticsearch start
root@shield-VM:~# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset:  enabled)
Drop-In: /etc/systemd/system/elasticsearch.service.d
       └─override.conf
Active: active (running) since Mon 2019-04-01 08:09:59 MST; 2s ago
 Docs: http://www.elastic.co
Main PID: 232958 (java)
Tasks: 18 (limit: 9460)
CGroup: /system.slice/elasticsearch.service
       ├─232958 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch
       └─233022 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Apr 01 08:09:59 shield-VM systemd[1]: Started Elasticsearch.
Apr 01 08:09:59 shield-VM elasticsearch[232958]: warning: Falling back to java on path. This behavior is deprecated. Specify JAVA_HOME
root@shield-VM:~# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
 Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/elasticsearch.service.d
       └─override.conf
Active: failed (Result: exit-code) since Mon 2019-04-01 08:10:02 MST; 3s ago
 Docs: http://www.elastic.co
Process: 232958 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 232958 (code=exited, status=1/FAILURE)

Apr 01 08:09:59 shield-VM systemd[1]: Started Elasticsearch.
Apr 01 08:09:59 shield-VM elasticsearch[232958]: warning: Falling back to java on path. This behavior is deprecated. Specify JAVA_HOME
Apr 01 08:10:02 shield-VM systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Apr 01 08:10:02 shield-VM systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

Should I be sudoing instead of being root?

5

This shows that Elasticsearch isn't running. Running as sudo or as root doesn't really make a difference. You should probably run journalctl to see what is causing Elasticsearch to fail. Something like

journalctl | grep elasticsearch

Should do the trick.

6

Thanks Mike.
Many, many results spitting out. Here is the last three (after hitting CTRL+C):

Mar 28 18:19:44 shield-VM kibana[56522]: {"type":"log","@timestamp":"2019-03-29T01:19:44Z","tags":["warning","elasticsearch","admin"],"pid":56522,"message":"Unable to revive connection: http://192.168.0.92:9200/"}
Mar 28 18:19:44 shield-VM kibana[56522]: {"type":"log","@timestamp":"2019-03-29T01:19:44Z","tags":["warning","elasticsearch","admin"],"pid":56522,"message":"No living connections"}
Mar 28 18:19:46 shield-VM kibana[56522]: {"type":"log","@timestamp":"2019-03-29T01:19:46Z","tags":["warning","elasticsearch","admin"],"pid":56522,"message":"Unable to revive connection: http://192.168.0.92:9200/"}

I don't know how to troubleshoot these results. Help? :frowning:

7

This worked for me:
https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-6.7.html#_literal_ingest_geoip_literal_and_literal_ingest_user_agent_literal_are_no_longer_distributed_as_plugins
cd /user/share/elasticsearch
./bin/elasticsearch-plugin remove --purge ingest-geoip

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.