# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=927 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 928 --- ## [Implementing a "set" type in Elasticsearch](https://discuss.elastic.co/t/implementing-a-set-type-in-elasticsearch/231628)
**Author:** [@pushshift](https://discuss.elastic.co/u/pushshift)\ **Replies:** 4\ **Last updated:** [May 10, 2020, 11:04pm UTC](https://discuss.elastic.co/t/implementing-a-set-type-in-elasticsearch/231628 "2020-05-10T23:04:09Z")
As a test, I have a very basic mapping that consists of an id field and a "colors" text field. I want the colors field to only hold one of each type of color. If a color already exists in the colors field, I don't want t… --- ## [Find and delete duplicate documents in Elasticsearch](https://discuss.elastic.co/t/find-and-delete-duplicate-documents-in-elasticsearch/231533)
**Author:** [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\ **Replies:** 4\ **Last updated:** [May 10, 2020, 11:03pm UTC](https://discuss.elastic.co/t/find-and-delete-duplicate-documents-in-elasticsearch/231533 "2020-05-10T23:03:11Z")
I use python for find and delete duplicate documents in Elasticsearch I use this script but i have an error #!/usr/local/bin/python3 import hashlib from elasticsearch import Elasticsearch es = Elasticsearch(\["localhost… --- ## [I am receiving master not discovered or elected yet](https://discuss.elastic.co/t/i-am-receiving-master-not-discovered-or-elected-yet/231712)
**Author:** [@nadimmansour90](https://discuss.elastic.co/u/nadimmansour90)\ **Replies:** 7\ **Last updated:** [May 10, 2020, 10:01pm UTC](https://discuss.elastic.co/t/i-am-receiving-master-not-discovered-or-elected-yet/231712 "2020-05-10T22:01:59Z")
I was trying to renew the node\_pem certificate for the nodes in the elasticsearch cluster and the cluster became unavailable in the logs i can see : \[master\_10.118.12.10\] master not discovered or elected yet, an electi… --- ## [Limit amount of dynamic fields to a 1000](https://discuss.elastic.co/t/limit-amount-of-dynamic-fields-to-a-1000/231908)
**Author:** [@galbash](https://discuss.elastic.co/u/galbash)\ **Replies:** 0\ **Last updated:** [May 10, 2020, 8:48pm UTC](https://discuss.elastic.co/t/limit-amount-of-dynamic-fields-to-a-1000/231908 "2020-05-10T20:48:43Z")
Hi all, My use case is that I am indexing documents over which I have little control, and would like to use dynamic mappings. However, I would like to limit the number of fields in each index to a 1,000 (the default val… --- ## [Ways to speed up a date range query with an aggregation of a parent child field](https://discuss.elastic.co/t/ways-to-speed-up-a-date-range-query-with-an-aggregation-of-a-parent-child-field/231899)
**Author:** [@shocking\_black](https://discuss.elastic.co/u/shocking_black)\ **Replies:** 0\ **Last updated:** [May 10, 2020, 6:00pm UTC](https://discuss.elastic.co/t/ways-to-speed-up-a-date-range-query-with-an-aggregation-of-a-parent-child-field/231899 "2020-05-10T18:00:36Z")
Am trying to implement a "trending" feature for our video site based on date-tagged user favorites of videos. Our schema consists of Video documents with Favorites implemented as a child document of Videos, and each Fav… --- ## [Email alerts from machine learning, bucket and record anomalies](https://discuss.elastic.co/t/email-alerts-from-machine-learning-bucket-and-record-anomalies/231588)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 3\ **Last updated:** [May 10, 2020, 5:35pm UTC](https://discuss.elastic.co/t/email-alerts-from-machine-learning-bucket-and-record-anomalies/231588 "2020-05-10T17:35:20Z")
Hi, I have created a mail watch alert for machine learning and choose the lower Severity threshold for it In the execution history of the watch I have several entries, but I dont get email alerts I enter in the edi… --- ## [One node have less shards then the others](https://discuss.elastic.co/t/one-node-have-less-shards-then-the-others/231001)
**Author:** [@Burga](https://discuss.elastic.co/u/Burga)\ **Replies:** 2\ **Last updated:** [May 10, 2020, 2:32pm UTC](https://discuss.elastic.co/t/one-node-have-less-shards-then-the-others/231001 "2020-05-10T14:32:42Z")
Hi , we have a 9 node cluster where one node has almost half shards then the others but using more capacity then others. what could be the reason of this behavour ? and how can it be solved , rebalance ? by the way… --- ## [Redirection to log message from graphs/charts](https://discuss.elastic.co/t/redirection-to-log-message-from-graphs-charts/231867)
**Author:** [@nikhilesh](https://discuss.elastic.co/u/nikhilesh)\ **Replies:** 1\ **Last updated:** [May 10, 2020, 10:40am UTC](https://discuss.elastic.co/t/redirection-to-log-message-from-graphs-charts/231867 "2020-05-10T10:40:30Z")
Hi All, Can any one please help on the below points ? When we click on the graphical charts( line, bar, pie) it should re-direct to log file of the error. is it possible? Already i have applied grok filter for my … --- ## [Transform index is not updating the records in destination index in batch process](https://discuss.elastic.co/t/transform-index-is-not-updating-the-records-in-destination-index-in-batch-process/230626)
**Author:** [@anjilinga](https://discuss.elastic.co/u/anjilinga)\ **Replies:** 11\ **Last updated:** [May 10, 2020, 9:53am UTC](https://discuss.elastic.co/t/transform-index-is-not-updating-the-records-in-destination-index-in-batch-process/230626 "2020-05-10T09:53:53Z")
I have created transorm index in kibana with aggregations. The records are updated to destination index. I have updated the frequesncy as 10m and started the transform as below. POST \_transform/test-transform/\_update … --- ## [Shards initializing randomly](https://discuss.elastic.co/t/shards-initializing-randomly/229395)
**Author:** [@Lior\_Yakobov](https://discuss.elastic.co/u/Lior_Yakobov)\ **Replies:** 2\ **Last updated:** [May 10, 2020, 8:06am UTC](https://discuss.elastic.co/t/shards-initializing-randomly/229395 "2020-05-10T08:06:15Z")
Hello, I have upgraded my cluster to 7.5.2 around 2 weeks ago, and I'm seeing strange behavior related to shards ever since. (cluster running on EC2 Centos 7, RPM based Elasticsearch 7.5.2). For some reason, without a… --- ## [Fail to configure Xpack AD Realms](https://discuss.elastic.co/t/fail-to-configure-xpack-ad-realms/229818)
**Author:** [@ckough](https://discuss.elastic.co/u/ckough)\ **Replies:** 15\ **Last updated:** [May 10, 2020, 5:35am UTC](https://discuss.elastic.co/t/fail-to-configure-xpack-ad-realms/229818 "2020-05-10T05:35:42Z")
Hi there, I'm using ELK 7.6 with xpack security. The error I'm getting is Authentication to realm foo\_ad failed - authenticate failed (Caused by LDAPException(resultCode=49 (invalid credentials), diagnosticMessage='800… --- ## [Search\_as\_you\_type usage](https://discuss.elastic.co/t/search-as-you-type-usage/231506)
**Author:** [@CSharpBender](https://discuss.elastic.co/u/CSharpBender)\ **Replies:** 3\ **Last updated:** [May 9, 2020, 11:40am UTC](https://discuss.elastic.co/t/search-as-you-type-usage/231506 "2020-05-09T11:40:36Z")
Hello, I'm trying to make a search that orders the results by: -exact match -starts with -phrase match -exact words -fuzzy I'm using Elasticsearch 7.6.1 and after trying to make my own query I noticed that there i… --- ## [Override discovery.type=single-node in docker-compose.yml](https://discuss.elastic.co/t/override-discovery-type-single-node-in-docker-compose-yml/231622)
**Author:** [@braddre](https://discuss.elastic.co/u/braddre)\ **Replies:** 2\ **Last updated:** [May 9, 2020, 5:47pm UTC](https://discuss.elastic.co/t/override-discovery-type-single-node-in-docker-compose-yml/231622 "2020-05-09T17:47:20Z")
I'm using two -c yaml files on the docker-compose command line to provide override of Elasticsearch settings. I have the base settings in docker-compose.yml in the services.elasticsearch.environment, and the override in… --- ## [Possibilities of increasing data security in elasticsearch - power failures \[?\]](https://discuss.elastic.co/t/possibilities-of-increasing-data-security-in-elasticsearch-power-failures/231839)
**Author:** [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Replies:** 5\ **Last updated:** [May 9, 2020, 5:36pm UTC](https://discuss.elastic.co/t/possibilities-of-increasing-data-security-in-elasticsearch-power-failures/231839 "2020-05-09T17:36:53Z")
Hi, Relatively simple question about the configuration of elasticsearch with respect to specific data security. Are there any possibilities to configure an elasticsearch cluster so that in case of a power failure there… --- ## [Elasticsearch Watcher Rate function query](https://discuss.elastic.co/t/elasticsearch-watcher-rate-function-query/231570)
**Author:** [@rj23495](https://discuss.elastic.co/u/rj23495)\ **Replies:** 3\ **Last updated:** [May 9, 2020, 3:30pm UTC](https://discuss.elastic.co/t/elasticsearch-watcher-rate-function-query/231570 "2020-05-09T15:30:53Z")
Hi All, I wish to create a watch in elasticsearch based on rate function of logstash emitted events as seen in monitoring overview of logstash. How should i go about in creating a watch for the purpose as in if rate of… --- ## [Shards failure - recovery possible?](https://discuss.elastic.co/t/shards-failure-recovery-possible/231834)
**Author:** [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Replies:** 6\ **Last updated:** [May 9, 2020, 3:10pm UTC](https://discuss.elastic.co/t/shards-failure-recovery-possible/231834 "2020-05-09T15:10:19Z")
Hi, Unfortunately, there was a problem on my cluster when the power was switched off. Cluster has RED status - a few shards do not want to get up. I've tried using reroute: POST /\_cluster/reroute?retry\_failed=true b… --- ## [Not sure how to enable painless regex](https://discuss.elastic.co/t/not-sure-how-to-enable-painless-regex/231806)
**Author:** [@skazan](https://discuss.elastic.co/u/skazan)\ **Replies:** 2\ **Last updated:** [May 9, 2020, 12:00pm UTC](https://discuss.elastic.co/t/not-sure-how-to-enable-painless-regex/231806 "2020-05-09T12:00:33Z")
Hi, I want to enable regex for painless script as it is not enabled by default and hence my scripted field is failing to create. I added "script.painless.regex.enabled" under env section of elasticsearch.yaml file (plea… --- ## [Can be elasticsearch use to storage permanet data?](https://discuss.elastic.co/t/can-be-elasticsearch-use-to-storage-permanet-data/231821)
**Author:** [@nino](https://discuss.elastic.co/u/nino)\ **Replies:** 2\ **Last updated:** [May 9, 2020, 11:30am UTC](https://discuss.elastic.co/t/can-be-elasticsearch-use-to-storage-permanet-data/231821 "2020-05-09T11:30:51Z")
Hello all, i am wondering if elasticsearch can be use to register data for ever (basic version), i mean i have several python processes that get data from different API and generate a custom document that then is indexe… --- ## [Unable to run ElasticMR job](https://discuss.elastic.co/t/unable-to-run-elasticmr-job/231830)
**Author:** [@vkg0312](https://discuss.elastic.co/u/vkg0312)\ **Replies:** 0\ **Last updated:** [May 9, 2020, 10:31am UTC](https://discuss.elastic.co/t/unable-to-run-elasticmr-job/231830 "2020-05-09T10:31:57Z")
While running a elastic mr job facing HttpConnectionManger not found: Got the below error Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpConnectionManager at org.elasticsea… --- ## [Curl: (6) Could not resolve host: quickstart-es-http” error](https://discuss.elastic.co/t/curl-6-could-not-resolve-host-quickstart-es-http-error/229328)
**Author:** [@ubiquitousdude](https://discuss.elastic.co/u/ubiquitousdude)\ **Replies:** 1\ **Last updated:** [May 9, 2020, 8:47am UTC](https://discuss.elastic.co/t/curl-6-could-not-resolve-host-quickstart-es-http-error/229328 "2020-05-09T08:47:39Z")
Hi, I'm trying to follow the article here for deploying Elastic Cloud on Kubernetes: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html But I get an error when requesting ElasticSearch endpoint i… --- ## [I've setup transport and http layer security but my two nodes can't detect eachother?](https://discuss.elastic.co/t/ive-setup-transport-and-http-layer-security-but-my-two-nodes-cant-detect-eachother/231808)
**Author:** [@syost](https://discuss.elastic.co/u/syost)\ **Replies:** 5\ **Last updated:** [May 9, 2020, 8:40am UTC](https://discuss.elastic.co/t/ive-setup-transport-and-http-layer-security-but-my-two-nodes-cant-detect-eachother/231808 "2020-05-09T08:40:16Z")
I'm trying to secure my cluster through the tutorial in the docs. I have two EC2 instances running. I have installed elasticsearch on both of them. I've setup transport layer and http layer security. I have started the e… --- ## [Unique index for each user](https://discuss.elastic.co/t/unique-index-for-each-user/231820)
**Author:** [@JavaDeveloper](https://discuss.elastic.co/u/JavaDeveloper)\ **Replies:** 4\ **Last updated:** [May 9, 2020, 8:12am UTC](https://discuss.elastic.co/t/unique-index-for-each-user/231820 "2020-05-09T08:12:29Z")
Hi everyone. I'm sending my application's logs to logstash as json and I want to save them in elasticsearch by different indices. In fact, I want to save logs of each user in a unique index that is equal to user key in… --- ## [How to set the property on POCO for ElasticSearch NEST API to serialize it correctly](https://discuss.elastic.co/t/how-to-set-the-property-on-poco-for-elasticsearch-nest-api-to-serialize-it-correctly/231745)
**Author:** [@itsbalur](https://discuss.elastic.co/u/itsbalur)\ **Replies:** 2\ **Last updated:** [May 9, 2020, 5:32am UTC](https://discuss.elastic.co/t/how-to-set-the-property-on-poco-for-elasticsearch-nest-api-to-serialize-it-correctly/231745 "2020-05-09T05:32:40Z")
I have the below POCO for ElasticSearch response public PublicationInfo PublicationInfo { get; } where the PublicationInfo is defined as below public class PublicationInfo { public string Issn { get; } … --- ## [Index restoration from snapshot is taking a lot of time in 7.2.0 ES](https://discuss.elastic.co/t/index-restoration-from-snapshot-is-taking-a-lot-of-time-in-7-2-0-es/227184)
**Author:** [@Faiz\_Ahmed\_Mushtak\_H](https://discuss.elastic.co/u/Faiz_Ahmed_Mushtak_H)\ **Replies:** 19\ **Last updated:** [May 8, 2020, 8:32am UTC](https://discuss.elastic.co/t/index-restoration-from-snapshot-is-taking-a-lot-of-time-in-7-2-0-es/227184 "2020-05-08T08:32:34Z")
Elasticsearch version ( bin/elasticsearch --version ): 7.2.0 Plugins installed : JVM version ( java -version ): JDK 12 (bundled) OS version ( uname -a if on a Unix-like system): Ubuntu 14.04.5 LTS This is in ref… --- ## [FieldData or Nested optimization - Discuss](https://discuss.elastic.co/t/fielddata-or-nested-optimization-discuss/231768)
**Author:** [@EliuFlorez](https://discuss.elastic.co/u/EliuFlorez)\ **Replies:** 1\ **Last updated:** [May 9, 2020, 3:44am UTC](https://discuss.elastic.co/t/fielddata-or-nested-optimization-discuss/231768 "2020-05-09T03:44:06Z")
Hi guys, I find myself making some changes to an old scrits and I have the doubt that it would be more optimal if I continue using ' fielddata ' or ' nested ' that is to say the scripts do the searches or aggregations fr… --- ## [When indexing a repository of files (using fsCrawler or genearlly) what happens under the hood?](https://discuss.elastic.co/t/when-indexing-a-repository-of-files-using-fscrawler-or-genearlly-what-happens-under-the-hood/230885)
**Author:** [@Lisahtwy](https://discuss.elastic.co/u/Lisahtwy)\ **Replies:** 3\ **Last updated:** [May 9, 2020, 3:35am UTC](https://discuss.elastic.co/t/when-indexing-a-repository-of-files-using-fscrawler-or-genearlly-what-happens-under-the-hood/230885 "2020-05-09T03:35:32Z")
This might seem silly, but What happens when I index files in a repository? just want to elaborate my question here.. I have around 20GB of files, when I indexed them using fsCrawler it created an index of size 56mb. … --- ## [Fuzziness with best\_fields query not working as expected](https://discuss.elastic.co/t/fuzziness-with-best-fields-query-not-working-as-expected/231803)
**Author:** [@gideongrossman](https://discuss.elastic.co/u/gideongrossman)\ **Replies:** 0\ **Last updated:** [May 8, 2020, 9:25pm UTC](https://discuss.elastic.co/t/fuzziness-with-best-fields-query-not-working-as-expected/231803 "2020-05-08T21:25:20Z")
I am running a multimatch best\_fields query and the fuzziness parameter is not operating as I expect. I am using a very small simple index to test. The index contains two documents. { "\_index": "contacts\_4", "\_… --- ## [How to get top distinct users that reply to a specific user?](https://discuss.elastic.co/t/how-to-get-top-distinct-users-that-reply-to-a-specific-user/229726)
**Author:** [@pushshift](https://discuss.elastic.co/u/pushshift)\ **Replies:** 3\ **Last updated:** [May 8, 2020, 8:52pm UTC](https://discuss.elastic.co/t/how-to-get-top-distinct-users-that-reply-to-a-specific-user/229726 "2020-05-08T20:52:57Z")
I'm curious what combination of aggregations would be best for the following use-case. I would like to see the top users (percentage) that have replied to a collection of original tweets made by a user. For example, the… --- ## [Was the icu\_tokenizer renamed?](https://discuss.elastic.co/t/was-the-icu-tokenizer-renamed/231798)
**Author:** [@pushshift](https://discuss.elastic.co/u/pushshift)\ **Replies:** 1\ **Last updated:** [May 8, 2020, 8:40pm UTC](https://discuss.elastic.co/t/was-the-icu-tokenizer-renamed/231798 "2020-05-08T20:40:25Z")
I'm getting the following error after performing: (I have installed the analyzer-icu plugin under ES 7.6.x) curl -H "content-type: application/json" -XPUT localhost:9200/icu\_sample -d @icu\_tokenizer icu\_tokenizer file: … --- ## [How to specify mapping with both dynamic template and \_size enabled](https://discuss.elastic.co/t/how-to-specify-mapping-with-both-dynamic-template-and-size-enabled/231443)
**Author:** [@plm](https://discuss.elastic.co/u/plm)\ **Replies:** 2\ **Last updated:** [May 8, 2020, 8:20pm UTC](https://discuss.elastic.co/t/how-to-specify-mapping-with-both-dynamic-template-and-size-enabled/231443 "2020-05-08T20:20:09Z")
Hi, I am trying to create an index template that includes both dynamic templates as well as enabling the \_size field. Here is what I am passing to the High Level Rest API PutIndexTemplateRequest templateIndexRequest =… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=926) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=928)