# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=882 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 883 --- ## [Question about es shards](https://discuss.elastic.co/t/question-about-es-shards/238421)
**Author:** [@jerry-chen-315](https://discuss.elastic.co/u/jerry-chen-315)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 9:43am UTC](https://discuss.elastic.co/t/question-about-es-shards/238421 "2020-06-24T09:43:40Z")
Hello: Q1.Do the shards auto relocate to other nodes when a node failure or shutdown? Q2.How can I monitor the unassigned shards progress of allocation? Thanks for all your help --- ## [Failed to validate incoming join request from node](https://discuss.elastic.co/t/failed-to-validate-incoming-join-request-from-node/238420)
**Author:** [@elk6](https://discuss.elastic.co/u/elk6)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 9:21am UTC](https://discuss.elastic.co/t/failed-to-validate-incoming-join-request-from-node/238420 "2020-06-24T09:21:09Z")
I'm trying to set up a simple master-1 node with data-1 and data-2 but when I look at the log I get this error. All nodes are fresh CentOS 7 machine that can ping each other. For all the cluster I've set: cluster.na… --- ## [Index with additional field taking less space than the same index without it](https://discuss.elastic.co/t/index-with-additional-field-taking-less-space-than-the-same-index-without-it/237815)
**Author:** [@jnioche](https://discuss.elastic.co/u/jnioche)\ **Replies:** 6\ **Last updated:** [June 24, 2020, 8:36am UTC](https://discuss.elastic.co/t/index-with-additional-field-taking-less-space-than-the-same-index-without-it/237815 "2020-06-24T08:36:14Z")
Hi, We have been using a field in a similar way to what the old \_all field used to do. We have a bunch of copy\_to from most of our other fields to copy to that field. I have been trying to measure the impact on disk sp… --- ## [How to split an Index, number\_of\_routing\_shards issue](https://discuss.elastic.co/t/how-to-split-an-index-number-of-routing-shards-issue/238024)
**Author:** [@defalt](https://discuss.elastic.co/u/defalt)\ **Replies:** 3\ **Last updated:** [June 24, 2020, 8:04am UTC](https://discuss.elastic.co/t/how-to-split-an-index-number-of-routing-shards-issue/238024 "2020-06-24T08:04:44Z")
Hi, I want to split my index with 5 shards into 10 or 30 shards. At first I tried it with the \_splitAPI but i get the following result the number of routing shards \[5\] must be a multiple of the target shards \[10\] I tr… --- ## [Elastic Search Indexing](https://discuss.elastic.co/t/elastic-search-indexing/238212)
**Author:** [@vijaya](https://discuss.elastic.co/u/vijaya)\ **Replies:** 8\ **Last updated:** [June 24, 2020, 7:51am UTC](https://discuss.elastic.co/t/elastic-search-indexing/238212 "2020-06-24T07:51:06Z")
Dear Team, How to include aggregations at the time of creating indexing in elastic search. Please help me. Best Regards, Vijaya. --- ## [Reindex or sciling index](https://discuss.elastic.co/t/reindex-or-sciling-index/238393)
**Author:** [@umay\_fb](https://discuss.elastic.co/u/umay_fb)\ **Replies:** 3\ **Last updated:** [June 24, 2020, 7:11am UTC](https://discuss.elastic.co/t/reindex-or-sciling-index/238393 "2020-06-24T07:11:32Z")
i have one index with size 400GB. its too big so i want slicing it,i do it with reindex to monthly. but i get duplicate hit in index pattern. how to solve it? can i reindex from existing index not copy the data but cut … --- ## [How does must work's for multiple query with in array and in filter if there is null value(for query) then automatically remove that query](https://discuss.elastic.co/t/how-does-must-works-for-multiple-query-with-in-array-and-in-filter-if-there-is-null-value-for-query-then-automatically-remove-that-query/238394)
**Author:** [@akshaybhuradia](https://discuss.elastic.co/u/akshaybhuradia)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 7:06am UTC](https://discuss.elastic.co/t/how-does-must-works-for-multiple-query-with-in-array-and-in-filter-if-there-is-null-value-for-query-then-automatically-remove-that-query/238394 "2020-06-24T07:06:16Z")
{ "query": { "bool":{ "must": \[ {"match" : { "userid" : "001" } }, {"query\_string" : {"fields":\["searchbar\_content"\],"query" : "\*"} } \], "filter":\[ { "terms" : { "tag" : \["tag yesterday"\] } }, … --- ## [Service Elasticsearch failed](https://discuss.elastic.co/t/service-elasticsearch-failed/238286)
**Author:** [@Weathmious](https://discuss.elastic.co/u/Weathmious)\ **Replies:** 4\ **Last updated:** [June 24, 2020, 7:08am UTC](https://discuss.elastic.co/t/service-elasticsearch-failed/238286 "2020-06-24T07:08:17Z")
Hi, Hi want to create a Graylog server. He had an error due to elasticsearch so I removed elasticsearch and then install again. The problem is that elasticsearch.service failed after a few seconds when I start it. I wan… --- ## [Elasticsearch DB backup for last one hour for a particular index](https://discuss.elastic.co/t/elasticsearch-db-backup-for-last-one-hour-for-a-particular-index/238377)
**Author:** [@ad1982](https://discuss.elastic.co/u/ad1982)\ **Replies:** 6\ **Last updated:** [June 24, 2020, 7:04am UTC](https://discuss.elastic.co/t/elasticsearch-db-backup-for-last-one-hour-for-a-particular-index/238377 "2020-06-24T07:04:51Z")
Is there a way to take a backup a particular index for just the last 1 hour ? snapshot doesnt provide the time period option? Any ideas? --- ## [How to recover when the disk is fault](https://discuss.elastic.co/t/how-to-recover-when-the-disk-is-fault/238357)
**Author:** [@jinjinbari](https://discuss.elastic.co/u/jinjinbari)\ **Replies:** 7\ **Last updated:** [June 24, 2020, 6:48am UTC](https://discuss.elastic.co/t/how-to-recover-when-the-disk-is-fault/238357 "2020-06-24T06:48:30Z")
Hello. My server disk is broken yesterday. I couldn\`t get snapshot, only I have index files. I will make new cluster -\> store old files -\> configure path -\> start. Could I recover if I make new cluster only with inde… --- ## [I can't add a new node to my cluster](https://discuss.elastic.co/t/i-cant-add-a-new-node-to-my-cluster/238250)
**Author:** [@dferez](https://discuss.elastic.co/u/dferez)\ **Replies:** 5\ **Last updated:** [June 24, 2020, 6:08am UTC](https://discuss.elastic.co/t/i-cant-add-a-new-node-to-my-cluster/238250 "2020-06-24T06:08:31Z")
Hi. I currently have a two node cluster, each node is separated on a different host. I want to add a third node to the cluster, but it does not start, it has never started. Next I leave the configuration of the nodes: N… --- ## [Querying limited nested data (However nested type is deprecated in 7.x)](https://discuss.elastic.co/t/querying-limited-nested-data-however-nested-type-is-deprecated-in-7-x/238329)
**Author:** [@emmm-dee](https://discuss.elastic.co/u/emmm-dee)\ **Replies:** 6\ **Last updated:** [June 24, 2020, 5:54am UTC](https://discuss.elastic.co/t/querying-limited-nested-data-however-nested-type-is-deprecated-in-7-x/238329 "2020-06-24T05:54:35Z")
Elasticsearch 7.7 and I'm using the official php client to interact with the server. My issue was somewhat solved here: Need to return part of a doc from a search query/filter; is parent-child the way to go? - #2 by xa… --- ## [How to take snapshot for four different indices through curator](https://discuss.elastic.co/t/how-to-take-snapshot-for-four-different-indices-through-curator/238262)
**Author:** [@gokulnath112](https://discuss.elastic.co/u/gokulnath112)\ **Replies:** 2\ **Last updated:** [June 24, 2020, 5:38am UTC](https://discuss.elastic.co/t/how-to-take-snapshot-for-four-different-indices-through-curator/238262 "2020-06-24T05:38:01Z")
Hi Everyone.. Greetings...! I want to write a curator job to take 4 indices backup. But the all indices name are not starts with same word.(all are having different names) . In other case i have another set of indices … --- ## [Master not discovered or elected yet, an election requires 2 nodes with ids](https://discuss.elastic.co/t/master-not-discovered-or-elected-yet-an-election-requires-2-nodes-with-ids/237986)
**Author:** [@poetL](https://discuss.elastic.co/u/poetL)\ **Replies:** 2\ **Last updated:** [June 24, 2020, 5:27am UTC](https://discuss.elastic.co/t/master-not-discovered-or-elected-yet-an-election-requires-2-nodes-with-ids/237986 "2020-06-24T05:27:09Z")
I have three node clusters, one of which is a master node. The master node has been lost and there are only two slave nodes at present. However, after I rejoin the new master node, it cannot work properly \[node33\] maste… --- ## [Get count of top-level deleted documents in Elasticsearch](https://discuss.elastic.co/t/get-count-of-top-level-deleted-documents-in-elasticsearch/238371)
**Author:** [@kailoofi](https://discuss.elastic.co/u/kailoofi)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 5:24am UTC](https://discuss.elastic.co/t/get-count-of-top-level-deleted-documents-in-elasticsearch/238371 "2020-06-24T05:24:35Z")
I am looking to get an accurate count of deleted Elasticsearch documents excluding the nested documents at Lucene level. I am aware of the following APIs to get document and deleted document counts but none of them retur… --- ## [io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Session has no PSK](https://discuss.elastic.co/t/io-netty-handler-codec-decoderexception-javax-net-ssl-sslexception-session-has-no-psk/233722)
**Author:** [@rahul\_s](https://discuss.elastic.co/u/rahul_s)\ **Replies:** 5\ **Last updated:** [June 24, 2020, 5:22am UTC](https://discuss.elastic.co/t/io-netty-handler-codec-decoderexception-javax-net-ssl-sslexception-session-has-no-psk/233722 "2020-06-24T05:22:05Z")
Using Elasticsearch 7.6.1 with SSL enabled we noticed the following error being logged in Elasticsearch logs \[WARN \]\[o.e.h.AbstractHttpServerTransport\] \[host\] caught exception while handling client http traffic, closing… --- ## [How to connect API with Search UI](https://discuss.elastic.co/t/how-to-connect-api-with-search-ui/237970)
**Author:** [@spadel](https://discuss.elastic.co/u/spadel)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 5:07am UTC](https://discuss.elastic.co/t/how-to-connect-api-with-search-ui/237970 "2020-06-24T05:07:50Z")
I have an API with Flask, that runs on my localhost. Basically, I have a search box, where I can enter any sentence. Furthermore I have a database with sentences (in my case an excel sheet). Through some neural network, … --- ## [Elasticsearch cluster performance tuning](https://discuss.elastic.co/t/elasticsearch-cluster-performance-tuning/236812)
**Author:** [@AClerk](https://discuss.elastic.co/u/AClerk)\ **Replies:** 11\ **Last updated:** [June 24, 2020, 3:54am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-performance-tuning/236812 "2020-06-24T03:54:14Z")
Hi, Following an error I got in Kibana and the following posts How can I optimise and tune the elastic cluster? Elasticsearch/Kibana version are 7.6.2 I have 5 nodes in the cluster, all run on a CentOS 7 VMs 28G … --- ## [Shard allocation - primaries allocated on same data nodes](https://discuss.elastic.co/t/shard-allocation-primaries-allocated-on-same-data-nodes/238338)
**Author:** [@andrezenun](https://discuss.elastic.co/u/andrezenun)\ **Replies:** 4\ **Last updated:** [June 24, 2020, 3:42am UTC](https://discuss.elastic.co/t/shard-allocation-primaries-allocated-on-same-data-nodes/238338 "2020-06-24T03:42:23Z")
Hello everyone! I have a cluster, that I'm on the process of setup everything to go to production. I have six data nodes and 3 masters. I don't know why but for some reason that I cannot identify the shards are allocate… --- ## [Spark rdd save to elasticsearch with http signing](https://discuss.elastic.co/t/spark-rdd-save-to-elasticsearch-with-http-signing/238367)
**Author:** [@kdhccb](https://discuss.elastic.co/u/kdhccb)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 3:12am UTC](https://discuss.elastic.co/t/spark-rdd-save-to-elasticsearch-with-http-signing/238367 "2020-06-24T03:12:26Z")
I would like to add a document to elasticsearch with the spark application. However, because http signature authentication cannot be added in the spark application, the document cannot be added due to permission issues. … --- ## [ES 7.8 ILM shrink action does not allocate primaries](https://discuss.elastic.co/t/es-7-8-ilm-shrink-action-does-not-allocate-primaries/238155)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 3:06am UTC](https://discuss.elastic.co/t/es-7-8-ilm-shrink-action-does-not-allocate-primaries/238155 "2020-06-24T03:06:30Z")
The ILM shrink action seems to be unreliable when trying to allocate primary shards to the same node because it stuck at this step check-shrink-allocation step with the message: Waiting for node \[XYZ\] to contain \[3\] shar… --- ## [How to create a Watcher for no ingested logs for X minutes](https://discuss.elastic.co/t/how-to-create-a-watcher-for-no-ingested-logs-for-x-minutes/237842)
**Author:** [@ckough](https://discuss.elastic.co/u/ckough)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 11:00pm UTC](https://discuss.elastic.co/t/how-to-create-a-watcher-for-no-ingested-logs-for-x-minutes/237842 "2020-06-23T23:00:10Z")
Hi There, I'm using ELK 7.6. I have a question regarding watchers. Is it possible to create a watcher that sends an alert when it detects that there are no new documents or indexes being created for the past, say 30, m… --- ## [Flow Search Data MDM via Elasticsearch](https://discuss.elastic.co/t/flow-search-data-mdm-via-elasticsearch/238224)
**Author:** [@alf1an](https://discuss.elastic.co/u/alf1an)\ **Replies:** 3\ **Last updated:** [June 23, 2020, 10:57pm UTC](https://discuss.elastic.co/t/flow-search-data-mdm-via-elasticsearch/238224 "2020-06-23T22:57:26Z")
Hi All, I have a project to integrate Web Portal, Elasticsearch and IBM MDM. I already create the flow about it, but is it correct flow? Big appreciate for your thought and recommendation Thanks --- ## [How to Set up a cluster for high availability](https://discuss.elastic.co/t/how-to-set-up-a-cluster-for-high-availability/237978)
**Author:** [@emreorucoglu](https://discuss.elastic.co/u/emreorucoglu)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 10:48pm UTC](https://discuss.elastic.co/t/how-to-set-up-a-cluster-for-high-availability/237978 "2020-06-23T22:48:13Z")
Hello there, I am trying to establish a remote cluster connection for high usability between two remote machines. However, although I can access the Elastic Search port, I cannot connect to the remote cluster. Can you h… --- ## [Resources usage](https://discuss.elastic.co/t/resources-usage/238333)
**Author:** [@Elastan](https://discuss.elastic.co/u/Elastan)\ **Replies:** 6\ **Last updated:** [June 23, 2020, 10:39pm UTC](https://discuss.elastic.co/t/resources-usage/238333 "2020-06-23T22:39:59Z")
Hello, I'm newbie to Elastic. Assuming index size will have 500GB, let's say 100 users each 5GB of index data. What hardware is needed for that? If querying/searching will be only performed always on about 1/100 of wh… --- ## [Failed to execute index. number of documents in the index cannot exceed 2147483519](https://discuss.elastic.co/t/failed-to-execute-index-number-of-documents-in-the-index-cannot-exceed-2147483519/238124)
**Author:** [@DucTran](https://discuss.elastic.co/u/DucTran)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 10:43pm UTC](https://discuss.elastic.co/t/failed-to-execute-index-number-of-documents-in-the-index-cannot-exceed-2147483519/238124 "2020-06-23T22:43:31Z")
Hi Guys, I am having problems with the capacity of ElasticSearch as below. I'm understanding the number od docs per shard reached to maximum number - 2147483519. I deleted many docs but the error continues, Elasticsear… --- ## [Moving from tribe to cross cluster search](https://discuss.elastic.co/t/moving-from-tribe-to-cross-cluster-search/238238)
**Author:** [@dextermorgan](https://discuss.elastic.co/u/dextermorgan)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 10:27pm UTC](https://discuss.elastic.co/t/moving-from-tribe-to-cross-cluster-search/238238 "2020-06-23T22:27:09Z")
Hi All, We have an infra where we have around 10 big clusters behind tribe nodes and all of them are on ES 5.4 and data is time sharded and ever growing. We need to move our infra to 7.8 (priority is to have live data c… --- ## [Does utf8json used in elastic v7.x support user defined custom formatters?](https://discuss.elastic.co/t/does-utf8json-used-in-elastic-v7-x-support-user-defined-custom-formatters/237283)
**Author:** [@stoj](https://discuss.elastic.co/u/stoj)\ **Replies:** 5\ **Last updated:** [June 23, 2020, 9:43pm UTC](https://discuss.elastic.co/t/does-utf8json-used-in-elastic-v7-x-support-user-defined-custom-formatters/237283 "2020-06-23T21:43:29Z")
I've got an interesting situation whereby elastic v7 default utf8json has managed to serialize my object, but is unable to de-serialize it correctly. public class MyClass : FlagsSet { \[JsonIgnore\] … --- ## [Writing to Elasticsearch from Spark failing](https://discuss.elastic.co/t/writing-to-elasticsearch-from-spark-failing/238344)
**Author:** [@Biswajit.kundu](https://discuss.elastic.co/u/Biswajit.kundu)\ **Replies:** 0\ **Last updated:** [June 23, 2020, 9:33pm UTC](https://discuss.elastic.co/t/writing-to-elasticsearch-from-spark-failing/238344 "2020-06-23T21:33:36Z")
Hi, Am trying to write to elastic search from Spark running on my local. I have a data frame df and am doing this df.write.format("org.elasticsearch.spark.sql").option("es.nodes","abc-es.xyz.com").option("es.port","920… --- ## [Query Hit limit](https://discuss.elastic.co/t/query-hit-limit/238336)
**Author:** [@th3Hopper](https://discuss.elastic.co/u/th3Hopper)\ **Replies:** 1\ **Last updated:** [June 23, 2020, 9:05pm UTC](https://discuss.elastic.co/t/query-hit-limit/238336 "2020-06-23T21:05:37Z")
I have index 2M document. If I query GET indexName/\_search?q=email:info@yahoo.com It's giving me this result:- "took" : 12, "timed\_out" : false, "\_shards" : { "total" : 1, "successful" : 1, "skipped" … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=881) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=883)