Kibana discovery and timelion page are blank

xwang12345 · June 12, 2017, 6:32pm

Elasticsearch version 5.2.0, Kibana version 5.0.2

Both discovery and timelion pages are blank
I got a lot of status code 304

{"type":"response","@timestamp":"2017-06-12T18:28:44Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/bundles/commons.style.css?v=14476","method":"get","headers":{"host":"192.168.1.204:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://192.168.1.204:5601/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4","if-none-match":"\"fbbaabcdab6b8ead6e006dcf04f091eda793ca7e-gzip\"","if-modified-since":"Thu, 24 Nov 2016 10:16:41 GMT"},"remoteAddress":"192.168.1.41","userAgent":"192.168.1.41","referer":"http://192.168.1.204:5601/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET /bundles/commons.style.css?v=14476 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2017-06-12T18:28:44Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/bundles/kibana.style.css?v=14476","method":"get","headers":{"host":"192.168.1.204:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://192.168.1.204:5601/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4","if-none-match":"\"2a9c3f5add653dc3b7b156898d9176d4fb4db48c-gzip\"","if-modified-since":"Thu, 24 Nov 2016 10:16:41 GMT"},"remoteAddress":"192.168.1.41","userAgent":"192.168.1.41","referer":"http://192.168.1.204:5601/app/kibana"},"res":{"statusCode":304,"responseTime":4,"contentLength":9},"message":"GET /bundles/kibana.style.css?v=14476 304 4ms - 9.0B"}
{"type":"response","@timestamp":"2017-06-12T18:28:44Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/bundles/commons.bundle.js?v=14476","method":"get","headers":{"host":"192.168.1.204:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36","accept":"*/*","referer":"http://192.168.1.204:5601/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4","if-none-match":"\"f1212ce0e1155f69bed2a9d27124080199b1379d-gzip\"","if-modified-since":"Thu, 24 Nov 2016 10:16:41 GMT"},"remoteAddress":"192.168.1.41","userAgent":"192.168.1.41","referer":"http://192.168.1.204:5601/app/kibana"},"res":{"statusCode":304,"responseTime":4,"contentLength":9},"message":"GET /bundles/commons.bundle.js?v=14476 304 4ms - 9.0B"}

Stacey_Gammon · June 12, 2017, 6:55pm

Kibana version 5.0.X only works with elasticsearch version 5.0.x. Please upgrade your Kibana to 5.2 and see if this helps.

https://www.elastic.co/support/matrix#show_compatibility

xwang12345 · June 12, 2017, 9:03pm

I did install kibana 5.2.2 and point to the same elasticsearch server, and I got the same result.

The discover and timelion pages are blank.

weltenwort · June 13, 2017, 7:17am

Do you see any errors on the browser console?

xwang12345 · June 15, 2017, 1:18am

This might be related?

{"type":"response","@timestamp":"2017-06-15T01:10:40Z","tags":[],"pid":13051,"method":"put","statusCode":400,"req":{"url":"/es_admin/.kibana/_mapping/timelion-sheet","method":"put","headers":{"host":"localhost:5601","connection":"keep-alive","content-length":"459","accept":"application/json, text/plain, */*","origin":"http://localhost:5601","kbn-version":"5.4.2","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36","content-type":"application/json","referer":"http://localhost:5601/app/timelion","accept-encoding":"gzip, deflate, sdch, br","accept-language":"en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"http://localhost:5601/app/timelion"},"res":{"statusCode":400,"responseTime":153,"contentLength":9},"message":"PUT /es_admin/.kibana/_mapping/timelion-sheet 400 153ms - 9.0B"}

What is the timelion_sheet and why it is returning 400 status code?

This example Kibana is 5.4.2, the elasticsearch is 5.4.0, I know someone might ask me run both version the same, I did try Kibana 5.4.0, got the same result

xwang12345 · June 15, 2017, 1:54am

Some interesting log at elasticsearch side

[org.elasticsearch.deprecation.index.mapper.StringFieldMapper$TypeParser] The [string] field is deprecated, please use [text] or [keyword] instead on [description]
[2017-06-15T01:10:02,267][WARN ][org.elasticsearch.deprecation.index.mapper.StringFieldMapper$TypeParser] The [string] field is deprecated, please use [text] or [keyword] instead on [timelion_sheet]
[2017-06-15T01:10:02,270][WARN ][org.elasticsearch.deprecation.index.mapper.StringFieldMapper$TypeParser] The [string] field is deprecated, please use [text] or [keyword] instead on [timelion_interval]
[2017-06-15T01:10:02,270][WARN ][org.elasticsearch.deprecation.index.mapper.StringFieldMapper$TypeParser] The [string] field is deprecated, please use [text] or [keyword] instead on [timelion_other_interval]
[2017-06-15T01:10:02,270][WARN ][org.elasticsearch.deprecation.index.mapper.StringFieldMapper$TypeParser] The [string] field is deprecated, please use [text] or [keyword] instead on [searchSourceJSON]
[2017-06-15T01:10:02,271][DEBUG][org.elasticsearch.action.admin.indices.mapping.put.TransportPutMappingAction] failed to put mappings on indices [[[.kibana/OV3d61a1SK-xWF3vBa4AFQ]]], type [timelion-sheet]
java.lang.IllegalArgumentException: mapper [hits] cannot be changed from type [long] to [integer]
        at org.elasticsearch.index.mapper.MappedFieldType.checkTypeName(MappedFieldType.java:147) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MappedFieldType.checkCompatibility(MappedFieldType.java:159) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:128) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.FieldTypeLookup.copyAndAddAll(FieldTypeLookup.java:94) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:398) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:334) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:266) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:311) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:633) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:611) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:570) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:262) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) [elasticsearch-5.4.0.jar:5.4.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
[2017-06-15T01:10:02,274][DEBUG][rest.suppressed          ] path: /.kibana/_mapping/timelion-sheet, params: {index=.kibana, type=timelion-sheet}
java.lang.IllegalArgumentException: mapper [hits] cannot be changed from type [long] to [integer]
        at org.elasticsearch.index.mapper.MappedFieldType.checkTypeName(MappedFieldType.java:147) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MappedFieldType.checkCompatibility(MappedFieldType.java:159) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:128) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.FieldTypeLookup.copyAndAddAll(FieldTypeLookup.java:94) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:398) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:334) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:266) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:311) ~[elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-5.4.0.jar:5.4.0]
:

I am running kibana 5.4.2 here, how can it still used obsoleted datatype?

xwang12345 · June 15, 2017, 1:56am

This is very frastrating, very very frastrating.
The elasticsearch is the official container version 5.4.0

I did install elasticsearch host version which seems to be fine, very strange

xwang12345 · June 15, 2017, 2:02am

Can someone tell me which line of code at kibana side generate this HTTP request? I can put a break point and investigate

{"type":"response","@timestamp":"2017-06-15T01:10:40Z","tags":[],"pid":13051,"method":"put","statusCode":400,"req":{"url":"/es_admin/.kibana/_mapping/timelion-sheet","method":"put","headers":{"host":"localhost:5601","connection":"keep-alive","content-length":"459","accept":"application/json, text/plain, */*","origin":"http://localhost:5601","kbn-version":"5.4.2","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36","content-type":"application/json","referer":"http://localhost:5601/app/timelion","accept-encoding":"gzip, deflate, sdch, br","accept-language":"en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"http://localhost:5601/app/timelion"},"res":{"statusCode":400,"responseTime":153,"contentLength":9},"message":"PUT /es_admin/.kibana/_mapping/timelion-sheet 400 153ms - 9.0B"}

weltenwort · June 15, 2017, 9:45am

It looks like the mappings for the timelion-sheet type in your .kibana index are incorrect. Did you manually import any documents into .kibana? To ensure the mapping is properly created I would recommend to

stop Kibana
dump all documents from the .kibana index
delete it
start Kibana
see if you can access the discover and timelion pages
import your dumped documents into the .kibana index to restore your saved objects

xwang12345 · June 15, 2017, 6:43pm

Hi weltenwort,

I tried a few fresh installed kibana and all got the same result, I have got log from both kibana and elastic search side, and I also nailed down the Request sent from kibana to elastic search

Here is the payload:

timelion-sheet
:
{properties: {title: {type: "string"}, hits: {type: "integer"}, description: {type: "string"},…}}
properties
:
{title: {type: "string"}, hits: {type: "integer"}, description: {type: "string"},…}
description
:
{type: "string"}
hits
:
{type: "integer"}
kibanaSavedObjectMeta
:
{properties: {searchSourceJSON: {type: "string"}}}
timelion_chart_height
:
{type: "integer"}
timelion_columns
:
{type: "integer"}
timelion_interval
:
{type: "string"}
timelion_other_interval
:
{type: "string"}
timelion_rows
:
{type: "integer"}
timelion_sheet
:
{type: "string"}
title
:
{type: "string"}
version
:
{type: "integer"}

It seems that elastic search side does not like this request, and from the log it throws exception at elastic search side

[2017-06-15T18:03:39,938][DEBUG][org.elasticsearch.action.admin.indices.mapping.put.TransportPutMappingAction] failed to put mappings on indices [[[.kibana/eHJaGWvgQbq_0f76M2IfNw]]], type [timelion-sheet]
java.lang.IllegalArgumentException: mapper [hits] cannot be changed from type [long] to [integer]

The request is saying Hits is Integer, while at elastic search side it is Long?

I did deleted the .kibana index at elastic search side but I got the same result

I tried to run this in kibana dev tools to query what is on elastic search
GET .kibana/_mapping/timelion-sheet
I got nothing

xwang12345 · June 15, 2017, 6:51pm

I googled the error log, something to do with dynamic mapping, seems to be a bug to be fixed in 5.5? I am using both kibana and es 5.4 here.

github.com/elastic/kibana

mapper [hits] cannot be changed from type [long] to [integer]

opened 10:02PM - 16 Jan 17 UTC

closed 03:27AM - 17 Jul 18 UTC

Hronom

bug Team:Operations

Please solve this issue, the full description is here https://discuss.elastic.co…/t/discover-tab-wont-load-anymore/38549/25 This happens when you put `dashboards` and `visualizations` to `.kibana` index directly by using ES, not from Kibana UI. Please let Kibana creates `.kibana` index from the scratch with correct mapping for fields `hits` and `version`, so it will looks like this: ``` { ".kibana": { "aliases": {}, "mappings": { "server": { "properties": { "uuid": { "type": "keyword" } } }, "dashboard": { "properties": { "description": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "hits": { "type": "integer" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "optionsJSON": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "panelsJSON": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "timeFrom": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "timeRestore": { "type": "boolean" }, "timeTo": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "title": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "uiStateJSON": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "version": { "type": "integer" } } }, "search": { "properties": { "columns": { "type": "text" }, "description": { "type": "text" }, "hits": { "type": "integer" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "sort": { "type": "text" }, "title": { "type": "text" }, "version": { "type": "integer" } } }, "timelion-sheet": { "properties": { "description": { "type": "text" }, "hits": { "type": "integer" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "timelion_chart_height": { "type": "integer" }, "timelion_columns": { "type": "integer" }, "timelion_interval": { "type": "text" }, "timelion_other_interval": { "type": "text" }, "timelion_rows": { "type": "integer" }, "timelion_sheet": { "type": "text" }, "title": { "type": "text" }, "version": { "type": "integer" } } }, "config": { "properties": { "buildNum": { "type": "keyword" }, "defaultIndex": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "index-pattern": { "properties": { "fieldFormatMap": { "type": "text" }, "fields": { "type": "text" }, "intervalName": { "type": "text" }, "notExpandable": { "type": "boolean" }, "sourceFilters": { "type": "text" }, "timeFieldName": { "type": "text" }, "title": { "type": "text" } } }, "visualization": { "properties": { "description": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }, "savedSearchId": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "title": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "uiStateJSON": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "version": { "type": "integer" }, "visState": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } }, "settings": { "index": { "creation_date": "1484597580045", "number_of_shards": "1", "number_of_replicas": "1", "uuid": "R5XXkVnsQsWZwQewrxsJvg", "version": { "created": "5010199" }, "provided_name": ".kibana" } } } } ``` This is very annoying bug, with this bug the `Discover tab` cannot be loaded. **Summary:** When Kibana starts and creates `.kibana` index in ES it didn't put mapping for this index, this is bad for all, who want to put dashboards and visualizations directly in ES, not from Kibana UI. Because when ES sees new type of document without previously setted mapping it try to recognize type of field in this document and here the problem begins. ES recognize type of fields `hits` and `version` like `long` but Kibana needs `integer` type.

xwang12345 · June 15, 2017, 7:12pm

Here is my understanding based on the research, correct me if I am wrong

I have fresh install of kibana. I also removed the .kibana index to start from fresh. In discovery page of kibana, I did create field mapping, which generated the index-pattern mapping. So far so good.

Then I clicked on the "Timelion" menu item, here the kibana tried to put a "timelion-sheet" mapping to elasticsearch. This put mapping request failed. I am sure at this moment ES side does not have this mapping, and somehow it used something called dynamic mapping:
[org.elasticsearch.index.mapper.MapperService] using dynamic[true]

And I am guessing it has some expectation about what the mapping could be, and what kibana sent to ES make it really not happy, it complains the "String" type should no be use, and eventually throw exception when it saw "Hit" type is "Integer" from kibana, but it is already assume the "Hit" is "long".

I am using both kibana and ES same version 5.4.

weltenwort · June 19, 2017, 8:04am

This mapping conflict usually occurs if something writes to the .kibana index before Kibana itself was able to create the mappings. I am unable to reproduce your problem when performing the following steps:

create a new Elasticsearch 5.4.0 cluster
ingest some log data
start Kibana 5.4.0
create an index pattern for these log data
save a timelion sheet

Does this match the steps you took?

xwang12345 · June 20, 2017, 5:13pm

You are right, I have figured it out and it is working now for me.
Thanks,

system · July 18, 2017, 5:13pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.