Elasticsearch is still initializing the kibana index

Elastic Stack Elasticsearch
1

I'm getting the same error message but its not resolved even I deleting .kibana index and restarted the service.

Elasticsearch cluster state : Red

Versions:
Elasticsearch : 2.0
Kibana : 4.2.0

Cluster health Log:

curl http://localhost:9200/_cluster/health?pretty

{
"cluster_name" : "sscn",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 2,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 0.0
}

Thanks in advance !

2

Have you looked in the Elasticsearch logs to see if there are any clues about why Elasticsearch is unable to allocate any indexes?

3

Issue only got fixed once I re-installed elasticsearch. Not sure what was the reason for this issue.

Its weird :worried: , Log file is not there. Again not sure, log files were removed during re-installation process of elasticsearch.

But I got the error message from my browser history.
Failed to execute phase [query], all shards failed]

Do you have any idea , in what occasion this error message this pop-up ?.

4

I have no idea what you mean by browser history, but again, read the server side logs. If you don't get any logs, fixing that is your first priority. Do not attempt to run Elasticsearch without proper logs.

5

I'm receiving the same error again in Kibana dashboard. Here is the server log. Please check and let me know , is there any way to resolve it.

[2015-11-17 08:32:54,635][INFO ][cluster.metadata ] [node1-ovh-de] [.kibana] creating index, cause [api], templates [], shards [1]/[1], mappings [config]
[2015-11-17 08:32:54,650][WARN ][indices.cluster ] [node1-ovh-de] [[.kibana][0]] marking and sending shard failed due to [failed to create shard]
[.kibana][[.kibana][0]] ElasticsearchException[failed to create shard]; nested: ElasticsearchParseException[Failed to parse setting [index.refresh_interval] with value [30] as a time value: unit is missing or unrecognized];
at org.elasticsearch.index.IndexService.createShard(IndexService.java:371)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyInitializingShard(IndicesClusterStateService.java:641)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewOrUpdatedShards(IndicesClusterStateService.java:541)
at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:170)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:494)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:225)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: ElasticsearchParseException[Failed to parse setting [index.refresh_interval] with value [30] as a time value: unit is missing or unrecognized]
at org.elasticsearch.common.unit.TimeValue.parseTimeValue(TimeValue.java:262)
at org.elasticsearch.common.settings.Settings.getAsTime(Settings.java:418)
at org.elasticsearch.index.shard.IndexShard.(IndexShard.java:236)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)

7

Is ES in a red state?

8

Yes, Cluster state in RED Color

els20.png827×552 29.1 KB

9

What indices do you have in the cluster? Check with _cat/indices.

10

This looks like the root cause:

ElasticsearchParseException[Failed to parse setting [index.refresh_interval] with value [30] as a time value: unit is missing or unrecognized];

It's weird if this comes from Kibana.

11

oh !. But still have same problem.

Anyway, I will try to downgrade the ELK version and parse the same logs. Will let you know whats happened.

12

It'd be really helpful if you answered our questions, otherwise we're not going to be able to help.

13

I installed timelion and then shortly after loading it, I saw this in the elasticsearch logs causing the cluster to go red:

[2015-12-09 23:49:04,758][INFO ][cluster.metadata         ] [node-2-master] [.kibana] create_mapping [timelion-sheet]
[2015-12-09 23:49:04,780][WARN ][cluster.action.shard     ] [node-2-master] [.kibana][0] received shard failed for [.kibana][0], node[mqZtmksNQwWZ_NS2v0X0pA], [P], v[23], s[STARTED], a[id=j5M16ot6Qm2bliPLqExOxA], indexUUID [Mw_IEucTSwSWTBXjpjO87Q], message [failed to update mappings], failure [IllegalArgumentException[Mapper for [hits] conflicts with existing mapping in other types:
[mapper [hits] cannot be changed from type [long] to [int]]]]
java.lang.IllegalArgumentException: Mapper for [hits] conflicts with existing mapping in other types:
[mapper [hits] cannot be changed from type [long] to [int]]
	at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)
	at org.elasticsearch.index.mapper.MapperService.checkNewMappersCompatibility(MapperService.java:364)
	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:315)
	at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:261)
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.processMapping(IndicesClusterStateService.java:418)
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyMappings(IndicesClusterStateService.java:372)
	at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:177)
	at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:494)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[2015-12-09 23:49:04,783][WARN ][cluster.action.shard     ] [node-2-master] [.kibana][0] received shard failed for [.kibana][0], node[mqZtmksNQwWZ_NS2v0X0pA], [P], v[23], s[STARTED], a[id=j5M16ot6Qm2bliPLqExOxA], indexUUID [Mw_IEucTSwSWTBXjpjO87Q], message [master {node-2-master}{hXSo4xWMTbeSy99Qpf38rg}{10.128.212.73}{10.128.212.73:9300}{temp=none, data=false, master=true} marked shard as started, but shard has previous failed. resending shard failure.], failure [Unknown]
14

Please start your own thread.

15

I replied here because it sounds like the same thing.

16

the same error after installed timelion. solved it? :slight_smile:

17

I did not. I resorted to deleting the .kibana index and re-creating from backups.

18

Hi there

Is this issue solved? I have this error message. Can you help me?

Greetings