Kibana 4.5 status red: unable to connect to elasticsearch

elasticsearch.log file is too long. it contains information regarding indexes that i created before for testing.
[2016-05-23 22:09:55,984][INFO ][node ] [No
cturne] version[2.3.3], pid[1191], build[218bdf1/2016-05-17T15:
40:04Z]
[2016-05-23 22:09:55,985][INFO ][node ] [No
cturne] initializing ...
[2016-05-23 22:09:57,898][INFO ][plugins ] [No
cturne] modules [reindex, lang-expression, lang-groovy], plugin
s [], sites []
[2016-05-23 22:09:58,141][INFO ][env ] [No
cturne] using [1] data paths, mounts [[/ (/dev/sda1)]], net usa
ble_space [3.3gb], net total_space [8.7gb], spins? [possibly],
types [ext4]
[2016-05-23 22:09:58,141][INFO ][env ] [No
cturne] heap size [1015.6mb], compressed ordinary object pointe
rs [true]
[2016-05-23 22:09:58,141][WARN ][env ] [No
cturne] max file descriptors [65535] for elasticsearch process
likely too low, consider increasing to at least [65536]
[2016-05-23 22:10:06,468][ERROR][gateway ] [No
cturne] failed to read local state, exiting...
java.lang.IllegalStateException: unable to upgrade the mappings
for the index [data], reason: [Field name [No.] cannot contain
'.']
at org.elasticsearch.cluster.metadata.MetaDataIndexUpgr
adeService.checkMappingsCompatibility(MetaDataIndexUpgradeServi
ce.java:308)
at org.elasticsearch.cluster.metadata.MetaDataIndexUpgr
adeService.upgradeIndexMetaData(MetaDataIndexUpgradeService.jav
a:116)
at org.elasticsearch.gateway.GatewayMetaState.pre20Upgr
ade(GatewayMetaState.java:228)
at org.elasticsearch.gateway.GatewayMetaState.(Ga
tewayMetaState.java:87)
at sun.reflect.NativeConstructorAccessorImpl.newInstanc
e0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstanc
e(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newIns
tance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructo
r.java:423)
at org.elasticsearch.common.inject.DefaultConstructionP
roxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:
50)..............................................................................................................................
...............................................................................

Try following command and post the output accordingly
"tailf /var/log/elasticsearch/elasticsearch.log"

root@sab-VirtualBox:~# tailf /var/log/elasticsearch/elasticsearch.log
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at <<>>
at org.elasticsearch.node.Node.(Node.java:213)
at org.elasticsearch.node.Node.(Node.java:140)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

root@sab-VirtualBox:~# service elasticsearch start

  • Starting Elasticsearch Server [ OK ]

root@sab-VirtualBox:~# service elasticsearch status

  • elasticsearch is not running

Today when i start the system and do work.output is
root@sab-VirtualBox:~# service elasticsearch status

  • elasticsearch is not running
    root@sab-VirtualBox:~# service elasticsearch start
  • Starting Elasticsearch Server [ OK ]
    root@sab-VirtualBox:~# service elasticsearch status
  • elasticsearch is running
    root@sab-VirtualBox:~# service elasticsearch status
  • elasticsearch is not running
    root@sab-VirtualBox:~# service elasticsearch start
  • Starting Elasticsearch Server [ OK ]
    root@sab-VirtualBox:~# service elasticsearch status
  • elasticsearch is running
    root@sab-VirtualBox:~# service elasticsearch status
  • elasticsearch is not running

[2016-05-23 22:10:06,468][ERROR][gateway ] [Nocturne] failed to read local state, exiting... java.lang.IllegalStateException: unable to upgrade the mappings for the index [data], reason: [Field name [No.] cannot contain '.']

Is this the last error message in the log? You have an on-disk index named "data" that contains a field with a period in its name. That was okay prior to ES 2.0 but is no longer allowed. You have to reindex that data to rename the field before you can run ES 2.0 or later.

ok
but when i run on terminal. for eg:-
root@sab-VirtualBox:~# /opt/logstash/bin/logstash -e 'input { stdin { } } output { elasticsearch {hosts =>'localhost' } }'
hi. how r u
Settings: Default pipeline workers: 1
Connection refused {:class=>"Manticore::SocketException", :level=>:error}
Logstash startup completed
Attempted to send a bulk request to Elasticsearch configured at '["http://localhost:9200/"]', but Elasticsearch appears to be unreachable or down! {:error_message=>"Connection refused", :class=>"Manticore::SocketException", :client_config=>{:hosts=>["http://localhost:9200/"], :ssl=>nil, :transport_options=>{:socket_timeout=>0, :request_time

yes its last error message.

But if ES isn't starting up properly, why would you think Logstash would be able to connect to it?

Downgrade ES, reindex the "data" index to rename all fields with periods in their name, run the migration plugin to find any other incompatibilities, then retry the upgrade.

1 Like

ok.thanks.i try.

Is there any document regarding migration plugin?please refer.

For me, googling "elasticsearch migration plugin" has this as the first hit: https://github.com/elastic/elasticsearch-migration

thanks. nw i installed migration plugin and check it.
when i run in browser the message is

All checks completed successfully.
Elasticsearch version: 1.7.5

can i upgrade ES 2.3 version?

i install ELK by Deb .
can i use same procedure?

Yes, an upgrade should be as simple as upgrading the Debian package and restarting ES (I don't think the init script does that for you).

after upgrade there is error in elasticsearch log file regarding migration plugin.
should i again install migration plugin for upgrade version?

java.lang.IllegalStateException: Could not load plugin descriptor for existing plugin [migration]. Was the plugin built before 2.0?

You can uninstall the migration plugin after the upgrade.

ok.
thanks

i remove migration plugin but there is another error in elasticsearch log file that is:-
java.lang.IllegalStateException: unable to upgrade the mappings for the index [bad], reason: [No handler for type [murmur3] declared on field [foo]]

what should i do?

Thanks for ur support. i install mapping plugin and elasticsearch work.