Error 500 on kibana startup - Unable to connect to Elasticsearch

Hello,

I've not been able to find a similar problem or a working answer so I'm opening a new topic.
I've installed Kibana through the apt repository to see what happens in my elastic search database. I've been using ES for a while now and it's working fine on a magento 2 website.

When I try to start Kibana by sudo /etc/init.d/kibana start I have the following error in my kibana stderr log file :

FATAL [class_cast_exception] class java.lang.String cannot be cast to class java.util.Map (java.lang.String and java.util.Map are in module java.base of loader 'bootstrap') :: {"path":"/.kibana_task_manager_1","query":{},"body":"{\"mappings\":{\"dynamic\":\"strict\",\"properties\":{\"config\":{\"dynamic\":\"true\",\"properties\":{\"buildNum\":{\"type\":\"keyword\"}}},\"migrationVersion\":{\"dynamic\":\"true\",\"type\":\"object\"},\"type\":{\"type\":\"keyword\"},\"namespace\":{\"type\":\"keyword\"},\"updated_at\":{\"type\":\"date\"},\"references\":{\"type\":\"nested\",\"properties\":{\"name\":{\"type\":\"keyword\"},\"type\":{\"type\":\"keyword\"},\"id\":{\"type\":\"keyword\"}}},\"task\":{\"properties\":{\"taskType\":{\"type\":\"keyword\"},\"scheduledAt\":{\"type\":\"date\"},\"runAt\":{\"type\":\"date\"},\"startedAt\":{\"type\":\"date\"},\"retryAt\":{\"type\":\"date\"},\"interval\":{\"type\":\"text\"},\"attempts\":{\"type\":\"integer\"},\"status\":{\"type\":\"keyword\"},\"params\":{\"type\":\"text\"},\"state\":{\"type\":\"text\"},\"user\":{\"type\":\"keyword\"},\"scope\":{\"type\":\"keyword\"},\"ownerId\":{\"type\":\"keyword\"}}}},\"_meta\":{\"migrationMappingPropertyHashes\":{\"config\":\"87aca8fdb053154f11383fce3dbf3edf\",\"migrationVersion\":\"4a1746014a75ade3a714e1db5763276f\",\"type\":\"2f4316de49999235636386fe51dc06c1\",\"namespace\":\"2f4316de49999235636386fe51dc06c1\",\"updated_at\":\"00da57df13e94e9d98437d13ace4bfe0\",\"references\":\"7997cf5a56cc02bdc9c93361bde732b0\",\"task\":\"a7206eba37aaea59ba083a079fd80b65\"}}},\"settings\":{\"number_of_shards\":1,\"auto_expand_replicas\":\"0-1\"}}","statusCode":500,"response":"{\"error\":{\"root_cause\":[{\"type\":\"class_cast_exception\",\"reason\":\"class java.lang.String cannot be cast to class java.util.Map (java.lang.String and java.util.Map are in module java.base of loader 'bootstrap')\"}],\"type\":\"class_cast_exception\",\"reason\":\"class java.lang.String cannot be cast to class java.util.Map (java.lang.String and java.util.Map are in module java.base of loader 'bootstrap')\"},\"status\":500}"}

Elastic search is up as you can see :
/usr/share/kibana$ ps aux | grep -i elas sebasti+ 8545 2.6 9.9 9193528 1618584 pts/3 Sl 10:51 2:11 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-2859441201120843636 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Djava.locale.providers=COMPAT -XX:UseAVX=2 -Des.path.home=/opt/elasticsearch/elasticsearch-6.8.3 -Des.path.conf=/opt/elasticsearch/elasticsearch-6.8.3/config -Des.distribution.flavor=default -Des.distribution.type=tar -cp /opt/elasticsearch/elasticsearch-6.8.3/lib/* org.elasticsearch.bootstrap.Elasticsearch -d

By enabling the verbose log, I also have the following more detailed exception :
{"type":"log","@timestamp":"2019-12-13T11:05:21Z","tags":["debug","elasticsearch","admin","query"],"pid":17351,"message":"500\nPUT /.kibana_task_manager_1\n{\"mappings\":{\"dynamic\":\"strict\",\"properties\":{\"config\":{\"dynamic\":\"true\",\"properties\":{\"buildNum\":{\"type\":\"keyword\"}}},\"migrationVersion\":{\"dynamic\":\"true\",\"type\":\"object\"},\"type\":{\"type\":\"keyword\"},\"namespace\":{\"type\":\"keyword\"},\"updated_at\":{\"type\":\"date\"},\"references\":{\"type\":\"nested\",\"properties\":{\"name\":{\"type\":\"keyword\"},\"type\":{\"type\":\"keyword\"},\"id\":{\"type\":\"keyword\"}}},\"task\":{\"properties\":{\"taskType\":{\"type\":\"keyword\"},\"scheduledAt\":{\"type\":\"date\"},\"runAt\":{\"type\":\"date\"},\"startedAt\":{\"type\":\"date\"},\"retryAt\":{\"type\":\"date\"},\"interval\":{\"type\":\"text\"},\"attempts\":{\"type\":\"integer\"},\"status\":{\"type\":\"keyword\"},\"params\":{\"type\":\"text\"},\"state\":{\"type\":\"text\"},\"user\":{\"type\":\"keyword\"},\"scope\":{\"type\":\"keyword\"},\"ownerId\":{\"type\":\"keyword\"}}}},\"_meta\":{\"migrationMappingPropertyHashes\":{\"config\":\"87aca8fdb053154f11383fce3dbf3edf\",\"migrationVersion\":\"4a1746014a75ade3a714e1db5763276f\",\"type\":\"2f4316de49999235636386fe51dc06c1\",\"namespace\":\"2f4316de49999235636386fe51dc06c1\",\"updated_at\":\"00da57df13e94e9d98437d13ace4bfe0\",\"references\":\"7997cf5a56cc02bdc9c93361bde732b0\",\"task\":\"a7206eba37aaea59ba083a079fd80b65\"}}},\"settings\":{\"number_of_shards\":1,\"auto_expand_replicas\":\"0-1\"}}"}

Configuration file is the default one in /etc/kibana/config.yml, except for the two debug rows :
elasticsearch.logQueries: true
logging.verbose: true

A curl request on ES returns the json :
/usr/share/kibana$ curl "127.0.0.1:9200/_nodes/settings?pretty=true" { "_nodes" : { "total" : 1, "successful" : 1, "failed" : 0 }, "cluster_name" : "elastic_cluster1", "nodes" : { "hmNYtPd2S7SGBIJp7YJBEw" : { "name" : "node-2", "transport_address" : "127.0.0.1:9300", "host" : "localhost", "ip" : "127.0.0.1", "version" : "6.8.3", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "0c48c0e", "roles" : [ "master", "data", "ingest" ], "attributes" : { "ml.machine_memory" : "16578093056", "xpack.installed" : "true", "ml.max_open_jobs" : "20", "ml.enabled" : "true" }, "settings" : { "cluster" : { "name" : "elastic_cluster1" }, "node" : { "attr" : { "xpack" : { "installed" : "true" }, "ml" : { "machine_memory" : "16578093056", "max_open_jobs" : "20", "enabled" : "true" } }, "data" : "true", "name" : "node-2" }, "path" : { "logs" : "/opt/elasticsearch/elasticsearch-6.8.3/logs", "home" : "/opt/elasticsearch/elasticsearch-6.8.3" }, "client" : { "type" : "node" }, "http" : { "type" : "security4", "port" : "9200", "type.default" : "netty4" }, "transport" : { "host" : "localhost", "tcp" : { "port" : "9300" }, "features" : { "x-pack" : "true" }, "type" : "security4", "type.default" : "netty4" }, "network" : { "host" : "0.0.0.0" } } } } }

I've no password set up on my ES instance
As you can see in the curl request I'm using http and not https
I'm using debian buster up to date
java version :
/usr/share/kibana$ java --version openjdk 11.0.5 2019-10-15 OpenJDK Runtime Environment (build 11.0.5+10-post-Debian-1deb10u1) OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Debian-1deb10u1, mixed mode, sharing)

Maybe I've missed something but I'm not even able to start kibana and I have no clue of what is going on.

Have you any idea of something I could be doing wrong ??

Thanks for your help!

2 Likes

Also, I have a similar exception in my running instance of elastic search :

[2019-12-13T12:23:05,984][WARN ][r.suppressed ] [node-2] path: /.kibana_task_manager_1, params: {index=.kibana_task_manager_1} java.lang.ClassCastException: class java.lang.String cannot be cast to class java.util.Map (java.lang.String and java.util.Map are in module java.base of loader 'bootstrap') at org.elasticsearch.action.admin.indices.create.CreateIndexRequest.source(CreateIndexRequest.java:391) ~[elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.action.admin.indices.RestCreateIndexAction.prepareRequest(RestCreateIndexAction.java:71) ~[elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:91) ~[elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:87) ~[?:?] at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.8.3.jar:6.8.3]
.......
[2019-12-13T12:23:05,995][WARN ][r.suppressed ] [node-2] path: /.kibana_1, params: {index=.kibana_1} java.lang.ClassCastException: class java.lang.String cannot be cast to class java.util.Map (java.lang.String and java.util.Map are in module java.base of loader 'bootstrap') at org.elasticsearch.action.admin.indices.create.CreateIndexRequest.source(CreateIndexRequest.java:391) ~[elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.action.admin.indices.RestCreateIndexAction.prepareRequest(RestCreateIndexAction.java:71) ~[elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:91) ~[elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:87) ~[?:?] at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:336) [elasticsearch-6.8.3.jar:6.8.3] at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-6.8.3.jar:6.8.3]
......
[2019-12-13T12:23:06,016][WARN ][o.e.h.n.Netty4HttpServerTransport] [node-2] caught exception while handling client http traffic, closing connection [id: 0xd683fb47, L:/127.0.0.1:9200 - R:/127.0.0.1:44372] java.io.IOException: Connexion ré-initialisée par le correspondant at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:?] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:?] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276) ~[?:?] at sun.nio.ch.IOUtil.read(IOUtil.java:245) ~[?:?] at sun.nio.ch.IOUtil.read(IOUtil.java:223) ~[?:?] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358) ~[?:?] at io.netty.buffer.PooledHeapByteBuf.setBytes(PooledHeapByteBuf.java:261) ~[netty-buffer-4.1.32.Final.jar:4.1.32.Final]

Again, thanks for your help !

I'm experiencing same issue. I just try to newly set up elk. elastsearch is up and running. when i try to start kibana I can see same error. I'm using default elastisearch and kibana configurations.

Hi,

I uninstalled Kibana, reinstalled it from the repository and tried to update the config but nothing works.
Does somebody have any suggestion ?

Thanks!

I am struggling with same issue.

I am facing this issue on mac.
elasticsearch version: Version: 6.8.1, Build: oss/tar/1fad4e1/2019-06-18T13:16:52.517138Z, JVM: 1.8.0_212

kibana version: 7.5.1 Build: oss

I was facing the same issue. I tried with closest matching version of kibana to that of elastic search. it worked for me! e.g. for elasticssearch v6.8.1 try kibana v6.8.* instead of kibana v 7.5.1.

Hi @siddharaj.nayak and @lkravi.

you can achieve it in very simple manner

for access index of ES through Kibana you need to focus on Version compatibility with both kibana and ES.
and for more you need to map your Elasticsearch host machine with kibana.yml

if you still facing the issue please share kibana.yml file here?

Thanks
HadoopHelp

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.