# Kibana is not able to fetch data from kibana collector **URL:** \ **Category:** Elasticsearch\ **Created:** [October 1, 2020, 2:13pm UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694 "2020-10-01T14:13:09Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![sudhakar1](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@sudhakar1](https://discuss.elastic.co/u/sudhakar1)\ **Post date:** [October 1, 2020, 2:13pm UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/1 "2020-10-01T14:13:09Z")
Kibana is not able to fetch data from kibana collector error is showing as screen shot below. Also Elastic Search is crashing , I am running ES 6.5.4 [http://localhost:9200/\_cluster/stats](http://localhost:9200/_cluster/stats) {"\_nodes":{"total":1,"successful":1,"failed":0},"cluster\_name":"elasticsearch","cluster\_uuid":"JlQW2XTaS6WNB3Av9qvjow","timestamp":1601560937455,"status":"red","indices":{"count":714,"shards":{"total":3545,"primaries":3545,"replication":0.0,"index":{"shards":{"min":1,"max":5,"avg":4.964985994397759},"primaries":{"min":1,"max":5,"avg":4.964985994397759},"replication":{"min":0.0,"max":0.0,"avg":0.0}}},"docs":{"count":26181250,"deleted":505087},"store":{"size\_in\_bytes":7757660809},"fielddata":{"memory\_size\_in\_bytes":0,"evictions":0},"query\_cache":{"memory\_size\_in\_bytes":0,"total\_count":0,"hit\_count":0,"miss\_count":0,"cache\_size":0,"cache\_count":0,"evictions":0},"completion":{"size\_in\_bytes":0},"segments":{"count":826,"memory\_in\_bytes":19268575,"terms\_memory\_in\_bytes":10639340,"stored\_fields\_memory\_in\_bytes":1961960,"term\_vectors\_memory\_in\_bytes":0,"norms\_memory\_in\_bytes":536192,"points\_memory\_in\_bytes":4195587,"doc\_values\_memory\_in\_bytes":1935496,"index\_writer\_memory\_in\_bytes":0,"version\_map\_memory\_in\_bytes":0,"fixed\_bit\_set\_memory\_in\_bytes":3341584,"max\_unsafe\_auto\_id\_timestamp":1601560378899,"file\_sizes":{}}},"nodes":{"count":{"total":1,"data":1,"coordinating\_only":0,"master":1,"ingest":1},"versions":["6.5.4"],"os":{"available\_processors":64,"allocated\_processors":64,"names":[{"name":"Windows Server 2016","count":1}],"mem":{"total\_in\_bytes":68163448832,"free\_in\_bytes":42088034304,"used\_in\_bytes":26075414528,"free\_percent":62,"used\_percent":38}},"process":{"cpu":{"percent":2},"open\_file\_descriptors":{"min":-1,"max":-1,"avg":0}},"jvm":{"max\_uptime\_in\_millis":625664,"versions":[{"version":"1.8.0\_144","vm\_name":"Java HotSpot(TM) 64-Bit Server VM","vm\_version":"25.144-b01","vm\_vendor":"Oracle Corporation","count":1}],"mem":{"heap\_used\_in\_bytes":856835264,"heap\_max\_in\_bytes":1037959168},"threads":324},"fs":{"total\_in\_bytes":2898566049792,"free\_in\_bytes":2873972973568,"available\_in\_bytes":2873972973568},"plugins":[{"name":"ingest-attachment","version":"6.5.4","elasticsearch\_version":"6.5.4","java\_version":"1.8","description":"Ingest processor that uses Apache Tika to extract contents","classname":"org.elasticsearch.ingest.attachment.IngestAttachmentPlugin","extended\_plugins":,"has\_native\_controller":false}],"network\_types":{"transport\_types":{"security4":1},"http\_types":{"security4":1}}}} Error log ES : [2020-10-01T19:02:17,421][ERROR][o.e.ExceptionsHelper] [uUQH632] fatal error at org.elasticsearch.ExceptionsHelper.lambda$maybeDieOnAnotherThread$2(ExceptionsHelper.java:264) at java.util.Optional.ifPresent(Optional.java:159) at org.elasticsearch.ExceptionsHelper.maybeDieOnAnotherThread(ExceptionsHelper.java:254) at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.exceptionCaught(Netty4HttpRequestHandler.java:176) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:285) at ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 1, 2020, 9:11pm UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/2 "2020-10-01T21:11:36Z")
You have 3545 shards on a single node, that is _ **way** _ too much. You need to reduce that to a few hundred. ---
**Author:** ![sudhakar1](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@sudhakar1](https://discuss.elastic.co/u/sudhakar1)\ **Post date:** [October 1, 2020, 10:55pm UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/3 "2020-10-01T22:55:23Z")
Thanks , but how do I do it , is there any configuration for this , I m doing it on a windows server I am doing it through elastic client c# dot net using below code where docindex is being created dynamically for each file. Pls suggest Code : string docIndex = "index\_" + dm.Id; var settings = new ConnectionSettings(new Uri(node)) ``` .InferMappingFor(m => m.IndexName(docIndex)); ``` var client = new ElasticClient(settings); var indexResponse = client.CreateIndex(docIndex, c =\> c ``` .Settings(s => s .Analysis(a => a .Analyzers(ad => ad .Custom("windows_path_hierarchy_analyzer", ca => ca .Tokenizer("windows_path_hierarchy_tokenizer") ) ) .Tokenizers(t => t .PathHierarchy("windows_path_hierarchy_tokenizer", ph => ph .Delimiter('\\') ) ) ) ) .Mappings(m => m .Map(mp => mp .AutoMap() .AllField(all => all .Enabled(false) ) .Properties(ps => ps .Text(s => s .Name(n => n.Path) .Analyzer("windows_path_hierarchy_analyzer") ) .Object(a => a .Name(n => n.Attachments) .AutoMap() ) ) ) ) ); ``` ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 1, 2020, 10:55pm UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/4 "2020-10-01T22:55:46Z")
Look at the `_shrink` API to start. ---
**Author:** ![sudhakar1](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@sudhakar1](https://discuss.elastic.co/u/sudhakar1)\ **Post date:** [October 2, 2020, 8:03am UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/5 "2020-10-02T08:03:29Z")
I tried doing this but it gave an error, Please help me at what stage i can fix this , what are the other options. curl -XPUT '[http://localhost:9200/\_all/\_settings?preserve\_existing=true](http://localhost:9200/_all/_settings?preserve_existing=true)' -d '{"index.number\_of\_shards" : "2","index.number\_of\_replicas" : "1"}' error { "error": { "root\_cause": [ { "type": "illegal\_argument\_exception", "reason": "Can't update non dynamic settings [[index.number\_of\_shards]] for open indices [[mykase\_af0c2213-2d3f-4c02-8646-b5b358029ebb/fG-XT4omTLijm4VeRe0\_hA], [mykase\_9b47d389-4298-4d68-972b-3d4b90d4d0a0/bFeuLmjDRwiTibM6lXVPVg], [mykase\_7aa622e8-7824-4529-af7e- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 2, 2020, 10:32am UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/6 "2020-10-02T10:32:15Z")
You can not modify the number of primary shards of an existing index. Instead, as @warkolm wrote, you can use the Shrink API. In case you did not find the documentation, here it is: > **[Shrink index API | Elasticsearch Reference \[7.9\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/indices-shrink-index.html)** ---
**Author:** ![sudhakar1](https://avatars.discourse-cdn.com/v4/letter/s/54ee81/32.png) [@sudhakar1](https://discuss.elastic.co/u/sudhakar1)\ **Post date:** [October 2, 2020, 10:45am UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/7 "2020-10-02T10:45:32Z")
i wish to do reindex , now how do i can control the shards while creating new index , can i do something while creating it .so that i can take care it initially , alos i have many indices so do i need to shrink each index one by one, PLs suggest what should i do now i am trying to do this way where mykase is prefix of all indices [http://localhost:9200/mykase\_\*/\_shrink/archive\_mykase\_2oct2020](http://localhost:9200/mykase_*/_shrink/archive_mykase_2oct2020) can something be done in code so that i can define shards at index creation level. string docIndex = "mykase\_" + dm.Id; ``` var settings = new ConnectionSettings(new Uri(node)) .InferMappingFor(m => m.IndexName(docIndex)); var client = new ElasticClient(settings); ``` var indexResponse = client.CreateIndex(docIndex, c =\> c .Settings(s =\> s .Analysis(a =\> a .Analyzers(ad =\> ad .Custom("windows\_path\_hierarchy\_analyzer", ca =\> ca .Tokenizer("windows\_path\_hierarchy\_tokenizer") ) ) .Tokenizers(t =\> t .PathHierarchy("windows\_path\_hierarchy\_tokenizer", ph =\> ph .Delimiter('\') ) ) ) ) ``` .Mappings(m => m .Map(mp => mp .AutoMap() .AllField(all => all .Enabled(false) ) .Properties(ps => ps .Text(s => s .Name(n => n.Path) .Analyzer("windows_path_hierarchy_analyzer") ) .Object(a => a .Name(n => n.Attachments) .AutoMap() ) ) ) ) ); ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 2, 2020, 11:55am UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/8 "2020-10-02T11:55:39Z")
> [@sudhakar1](#): > > can something be done in code so that i can define shards at index creation level. Yes. You can define the index setting: `index.number_of_shards`. See > **[Index modules | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#_static_index_settings)** And > **[Create index API | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#create-index-settings)** Note that it's probably better and easier to define an index template than changing your code. As you probably want to be able to change this configuration without having to ship your code again. See > **[Create or update index template API | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-template.html)** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 30, 2020, 11:55am UTC](https://discuss.elastic.co/t/kibana-is-not-able-to-fetch-data-from-kibana-collector/250694/9 "2020-10-30T11:55:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.