# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=858 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 859 --- ## [How to set default mapping to “keyword” for dynamically added properties in Elasticsearch](https://discuss.elastic.co/t/how-to-set-default-mapping-to-keyword-for-dynamically-added-properties-in-elasticsearch/241485)
**Author:** [@gitterhh](https://discuss.elastic.co/u/gitterhh)\ **Replies:** 1\ **Last updated:** [July 17, 2020, 3:26am UTC](https://discuss.elastic.co/t/how-to-set-default-mapping-to-keyword-for-dynamically-added-properties-in-elasticsearch/241485 "2020-07-17T03:26:13Z")
In our application we add new documents which can have new unknown properties. Is it possible to set default mapping to "keyword" for dynamically added new properties in Elasticsearch? --- ## [Shards unbalanced distribution](https://discuss.elastic.co/t/shards-unbalanced-distribution/241569)
**Author:** [@LuckyNemo](https://discuss.elastic.co/u/LuckyNemo)\ **Replies:** 1\ **Last updated:** [July 17, 2020, 2:18am UTC](https://discuss.elastic.co/t/shards-unbalanced-distribution/241569 "2020-07-17T02:18:11Z")
ES version 6.0.0 8 nodes(5 datanode 3 ingests,all are 8c16g config) I have a index (5 primary 1 replica) mapping like this { "myindex": { "mappings": { "order": { "properties": { "addTime… --- ## [ES 7.4 Field \[start\_time\] of type \[keyword\] does not support custom formats](https://discuss.elastic.co/t/es-7-4-field-start-time-of-type-keyword-does-not-support-custom-formats/241554)
**Author:** [@ghirsch](https://discuss.elastic.co/u/ghirsch)\ **Replies:** 5\ **Last updated:** [July 17, 2020, 1:55am UTC](https://discuss.elastic.co/t/es-7-4-field-start-time-of-type-keyword-does-not-support-custom-formats/241554 "2020-07-17T01:55:10Z")
Hi, I have an elasticsearch index with dynamic field mapping. I am inserting a log into the index with a field start\_time in the RFC3339 format (e.g. "2020-07-15T23:45:16Z") and until today this field was getting discove… --- ## [Ml detector with exclude frequent enabled](https://discuss.elastic.co/t/ml-detector-with-exclude-frequent-enabled/241387)
**Author:** [@saraKM](https://discuss.elastic.co/u/saraKM)\ **Replies:** 5\ **Last updated:** [July 16, 2020, 11:40pm UTC](https://discuss.elastic.co/t/ml-detector-with-exclude-frequent-enabled/241387 "2020-07-16T23:40:08Z")
Hi, I have two jobs (Each job has two detectors with max and high\_mean functions) working on same data and exclude frequent enabled for one. I have seen some high severity anomalies raised when exclude frequent is none,… --- ## [Reindex 350K data from RestHighLevelClient without timeout & shard fail](https://discuss.elastic.co/t/reindex-350k-data-from-resthighlevelclient-without-timeout-shard-fail/241467)
**Author:** [@sandeep\_rana](https://discuss.elastic.co/u/sandeep_rana)\ **Replies:** 1\ **Last updated:** [July 16, 2020, 11:17pm UTC](https://discuss.elastic.co/t/reindex-350k-data-from-resthighlevelclient-without-timeout-shard-fail/241467 "2020-07-16T23:17:37Z")
Why the timeout or fail shared while reindex 350k data with same mapping and settings --- ## [Elasticsearch 7.x with Puppet 3.x](https://discuss.elastic.co/t/elasticsearch-7-x-with-puppet-3-x/241529)
**Author:** [@maatthias](https://discuss.elastic.co/u/maatthias)\ **Replies:** 0\ **Last updated:** [July 16, 2020, 6:40pm UTC](https://discuss.elastic.co/t/elasticsearch-7-x-with-puppet-3-x/241529 "2020-07-16T18:40:10Z")
We are interested in using puppet to instrument our ELK stack. The ELK cluster is currently running 7.6.2. Puppet is older version 3.8.7. Is anyone aware of any successful puppet configurations in this arrangement?… --- ## [Elastic Cluster on Azure Cloud - Linux - one server fails to startup while the other is successful](https://discuss.elastic.co/t/elastic-cluster-on-azure-cloud-linux-one-server-fails-to-startup-while-the-other-is-successful/241021)
**Author:** [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\ **Replies:** 3\ **Last updated:** [July 16, 2020, 10:39pm UTC](https://discuss.elastic.co/t/elastic-cluster-on-azure-cloud-linux-one-server-fails-to-startup-while-the-other-is-successful/241021 "2020-07-16T22:39:58Z")
Here is a configuration of a 2 node elastic cluster: node.max\_local\_storage\_nodes: 2 cluster.name: elasticsearch node.name: server2 path.data: /etc/elasticsearch/data path.logs: /etc/elasticsearch/logs discovery.se… --- ## [How apply a security of data and services into Elastcisearch](https://discuss.elastic.co/t/how-apply-a-security-of-data-and-services-into-elastcisearch/241428)
**Author:** [@vero](https://discuss.elastic.co/u/vero)\ **Replies:** 3\ **Last updated:** [July 16, 2020, 10:39pm UTC](https://discuss.elastic.co/t/how-apply-a-security-of-data-and-services-into-elastcisearch/241428 "2020-07-16T22:39:29Z")
Hi, Do you have please some ideas on how we securise our data into Elasticsearch ? As my information we don't have a DataLake. On the other hand, how we securise also the data shifting between services, could we apply… --- ## [Alias name duplication](https://discuss.elastic.co/t/alias-name-duplication/241364)
**Author:** [@Ugon](https://discuss.elastic.co/u/Ugon)\ **Replies:** 2\ **Last updated:** [July 16, 2020, 9:17pm UTC](https://discuss.elastic.co/t/alias-name-duplication/241364 "2020-07-16T21:17:12Z")
Hello :slight\_smile: After restart of one of my clusters, data nodes fail to start because of alias name duplication. On my other clusters I have similar situations (alias name pointing to multiple indices) and i can re… --- ## [Need help using a bucket\_script with single-value or filters filters](https://discuss.elastic.co/t/need-help-using-a-bucket-script-with-single-value-or-filters-filters/241544)
**Author:** [@teebu](https://discuss.elastic.co/u/teebu)\ **Replies:** 0\ **Last updated:** [July 16, 2020, 8:56pm UTC](https://discuss.elastic.co/t/need-help-using-a-bucket-script-with-single-value-or-filters-filters/241544 "2020-07-16T20:56:41Z")
I'm trying to compute count of a nested field where active and inactive is not 0 for both values. But I can't get my aggs working inside the bucket\_script. Initially I tried using the active\_entities" : { "filter" but I … --- ## [Implications of using serial ids for documents](https://discuss.elastic.co/t/implications-of-using-serial-ids-for-documents/241355)
**Author:** [@terell](https://discuss.elastic.co/u/terell)\ **Replies:** 3\ **Last updated:** [July 16, 2020, 7:27pm UTC](https://discuss.elastic.co/t/implications-of-using-serial-ids-for-documents/241355 "2020-07-16T19:27:02Z")
I'm planning on linking documents I write to ES to records in my relational database by simply taking the record id and writing it to the document id. This means the IDs for my documents will be serial (ie. 1,2,3,4,5) in… --- ## [\[parent\] Data too large (for agg or reused\_arrays)](https://discuss.elastic.co/t/parent-data-too-large-for-agg-or-reused-arrays/241517)
**Author:** [@duttad](https://discuss.elastic.co/u/duttad)\ **Replies:** 0\ **Last updated:** [July 16, 2020, 5:44pm UTC](https://discuss.elastic.co/t/parent-data-too-large-for-agg-or-reused-arrays/241517 "2020-07-16T17:44:14Z")
I am doing nested aggregation over fields with large cardinality - so, probably the large number of buckets is tripping the circuit breakers. Trying to understand the following circuit breaker exception thoroughly. shar… --- ## [Boosting specific documents to be at the top of the results](https://discuss.elastic.co/t/boosting-specific-documents-to-be-at-the-top-of-the-results/240970)
**Author:** [@droplet](https://discuss.elastic.co/u/droplet)\ **Replies:** 6\ **Last updated:** [July 16, 2020, 3:24pm UTC](https://discuss.elastic.co/t/boosting-specific-documents-to-be-at-the-top-of-the-results/240970 "2020-07-16T15:24:50Z")
Hi there, I don't know how to best describe the topic, excuse the vague title. I'm having some issue with the search results coming from an index. The index contains about 1M+ documents (various products), but doing a… --- ## [Script\_score with multiline code](https://discuss.elastic.co/t/script-score-with-multiline-code/241468)
**Author:** [@ClubberLang](https://discuss.elastic.co/u/ClubberLang)\ **Replies:** 1\ **Last updated:** [July 16, 2020, 2:42pm UTC](https://discuss.elastic.co/t/script-score-with-multiline-code/241468 "2020-07-16T14:42:41Z")
Hi, I have a query, this one compute a score with painless. I try to use multiline painless, using """, but the parser seems to unlike this. From what I understand, it should work, right ? Here is my query for review. … --- ## [Elasticsearch for correlation](https://discuss.elastic.co/t/elasticsearch-for-correlation/241479)
**Author:** [@Suat\_Bey](https://discuss.elastic.co/u/Suat_Bey)\ **Replies:** 5\ **Last updated:** [July 16, 2020, 2:40pm UTC](https://discuss.elastic.co/t/elasticsearch-for-correlation/241479 "2020-07-16T14:40:26Z")
Is it possible to write a query for elacticsearch something like this: If same IP and same username generates 3 different rule.id within 15 min. --- ## [Elasticsearch unassigned shards](https://discuss.elastic.co/t/elasticsearch-unassigned-shards/241296)
**Author:** [@apintilie](https://discuss.elastic.co/u/apintilie)\ **Replies:** 13\ **Last updated:** [July 16, 2020, 2:30pm UTC](https://discuss.elastic.co/t/elasticsearch-unassigned-shards/241296 "2020-07-16T14:30:16Z")
Hi all, I have an ES cluster, made from 3 nodes, all are master/ingest/data. The cluster is configured with 5 shards and 1 replica, so 5 primary shards and 5 replica shards. After a restart of the cluster, I have a lot… --- ## [All shards failed even after tuning](https://discuss.elastic.co/t/all-shards-failed-even-after-tuning/241080)
**Author:** [@gzucca](https://discuss.elastic.co/u/gzucca)\ **Replies:** 9\ **Last updated:** [July 16, 2020, 2:14pm UTC](https://discuss.elastic.co/t/all-shards-failed-even-after-tuning/241080 "2020-07-16T14:14:31Z")
Hi all, I took a look into lots of open thread related to the error I'm getting. In my company we are using ELK 7.5.1 for a development enviroment. Both ELK and Kibana stay on the same machine with 8GB of RAM. The JVM… --- ## [Elasticsearch-7.8.0 user authentication implementation](https://discuss.elastic.co/t/elasticsearch-7-8-0-user-authentication-implementation/241456)
**Author:** [@Ashok\_Reddy](https://discuss.elastic.co/u/Ashok_Reddy)\ **Replies:** 4\ **Last updated:** [July 16, 2020, 2:11pm UTC](https://discuss.elastic.co/t/elasticsearch-7-8-0-user-authentication-implementation/241456 "2020-07-16T14:11:40Z")
Hello There, I am using elasticsearch 7.8.0, I want to implement the user authentication . is it possible to implement only user authentication without SSL enabled ( I don't want to enable ssl transport layer) ? --- ## [Nested geo\_shape's only stores the first shape](https://discuss.elastic.co/t/nested-geo-shapes-only-stores-the-first-shape/241448)
**Author:** [@aes](https://discuss.elastic.co/u/aes)\ **Replies:** 1\ **Last updated:** [July 16, 2020, 1:46pm UTC](https://discuss.elastic.co/t/nested-geo-shapes-only-stores-the-first-shape/241448 "2020-07-16T13:46:57Z")
Given a mapping of: ... "polygons": { - "type": "nested", "properties": { - "polygon": { - "type": "geo\_shape" }, "name": { - "type… --- ## [Get text similarity by Levenstein distance in range 0-1](https://discuss.elastic.co/t/get-text-similarity-by-levenstein-distance-in-range-0-1/241399)
**Author:** [@Alex\_D](https://discuss.elastic.co/u/Alex_D)\ **Replies:** 1\ **Last updated:** [July 16, 2020, 11:50am UTC](https://discuss.elastic.co/t/get-text-similarity-by-levenstein-distance-in-range-0-1/241399 "2020-07-16T11:50:12Z")
Hi all ! I search by a text field and want to get a similarity score for the entire phrase. Algorithm - Levenshtein distance. The result should be normalized in the range 0 - 1 Example: text in query: "big hat" ex… --- ## [Maximum heap reached and GC not working](https://discuss.elastic.co/t/maximum-heap-reached-and-gc-not-working/241436)
**Author:** [@Barak](https://discuss.elastic.co/u/Barak)\ **Replies:** 2\ **Last updated:** [July 16, 2020, 11:23am UTC](https://discuss.elastic.co/t/maximum-heap-reached-and-gc-not-working/241436 "2020-07-16T11:23:08Z")
Hi, As can be seen in the attached screenshot, at around 06:45:20 the heap scrapes the limit, when the GC count is 0 for the next minute. This causes this node to be unresponsive for this time period. It happens quite… --- ## [AWS EC2 based cluster best practices](https://discuss.elastic.co/t/aws-ec2-based-cluster-best-practices/235354)
**Author:** [@Lior\_Yakobov](https://discuss.elastic.co/u/Lior_Yakobov)\ **Replies:** 17\ **Last updated:** [July 16, 2020, 11:22am UTC](https://discuss.elastic.co/t/aws-ec2-based-cluster-best-practices/235354 "2020-07-16T11:22:23Z")
Hello, We are currently running a 20 data nodes cluster on AWS EC2 instances. Since we have only 1 replica per shard, and we cannot tolerate failure of more than 1 node at the time, we are not using ephermal storage bu… --- ## [Annotating a main class with enable elasticsearch repositories in Spring Data Elasticsearch](https://discuss.elastic.co/t/annotating-a-main-class-with-enable-elasticsearch-repositories-in-spring-data-elasticsearch/241434)
**Author:** [@imacpherson](https://discuss.elastic.co/u/imacpherson)\ **Replies:** 0\ **Last updated:** [July 16, 2020, 10:33am UTC](https://discuss.elastic.co/t/annotating-a-main-class-with-enable-elasticsearch-repositories-in-spring-data-elasticsearch/241434 "2020-07-16T10:33:25Z")
I've got a question regarding Spring Data Elasticsearch best practices using the EnableElasticsearchRepositories annotation. We've been working on writing a common Elasticsearch library to avoid repeating the same boiler… --- ## [How Attachments or file storage and searching is handled in Elasticsearch](https://discuss.elastic.co/t/how-attachments-or-file-storage-and-searching-is-handled-in-elasticsearch/241246)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 6\ **Last updated:** [July 16, 2020, 10:32am UTC](https://discuss.elastic.co/t/how-attachments-or-file-storage-and-searching-is-handled-in-elasticsearch/241246 "2020-07-16T10:32:41Z")
Hello Team, Could you please help me understand How Attachments or files like (pdf/text files/doc) are stored in elasticsearch and how the content and inverted index managed for searching. tried searching the document… --- ## [ILM hot-warm with beats and component templates](https://discuss.elastic.co/t/ilm-hot-warm-with-beats-and-component-templates/241272)
**Author:** [@niklaskurvinen](https://discuss.elastic.co/u/niklaskurvinen)\ **Replies:** 1\ **Last updated:** [July 16, 2020, 10:13am UTC](https://discuss.elastic.co/t/ilm-hot-warm-with-beats-and-component-templates/241272 "2020-07-16T10:13:22Z")
I've been looking at implementing hot-warm architecture in the most default way possible, but with 7.8 the component templates were introduced which changes some behavior previously documented in guides for this topic. I… --- ## [Intersect documents with same id in two indices (5.6)](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321)
**Author:** [@AsafHirschfeld](https://discuss.elastic.co/u/AsafHirschfeld)\ **Replies:** 3\ **Last updated:** [July 16, 2020, 9:55am UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321 "2020-07-16T09:55:37Z")
I have two indices with documents, each document assigned with a unique id. My goal is to find duplicate documents among the indices. Is there any sort of an intersection aggregation that i can make in order to determi… --- ## [Elasticsearch 7.7 crashing for Term Query if term text 200 - 300 char](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395)
**Author:** [@pappukhode](https://discuss.elastic.co/u/pappukhode)\ **Replies:** 9\ **Last updated:** [July 16, 2020, 9:29am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395 "2020-07-16T09:29:26Z")
Hi, I have upgraded Elasticsearch 5.5 to 7.7 recently. I have only 1 index of 30 fields and 6K data. Dataset is very simple in nature. \[provided 1g memory in jvm options\] In the dataset i have a field description whi… --- ## [Benchmarking and stress/load testing scripted fields](https://discuss.elastic.co/t/benchmarking-and-stress-load-testing-scripted-fields/241252)
**Author:** [@vivekkarne](https://discuss.elastic.co/u/vivekkarne)\ **Replies:** 0\ **Last updated:** [July 15, 2020, 8:46am UTC](https://discuss.elastic.co/t/benchmarking-and-stress-load-testing-scripted-fields/241252 "2020-07-15T08:46:07Z")
I would like to know if there is a set standard to test the performance of an elastic cluster when scripted fields are sent in a request? I am trying to extract some sort of parity between the cluster performance without… --- ## [How Data scientist can use data stored in elasticsearch](https://discuss.elastic.co/t/how-data-scientist-can-use-data-stored-in-elasticsearch/241244)
**Author:** [@vero](https://discuss.elastic.co/u/vero)\ **Replies:** 3\ **Last updated:** [July 16, 2020, 8:32am UTC](https://discuss.elastic.co/t/how-data-scientist-can-use-data-stored-in-elasticsearch/241244 "2020-07-16T08:32:06Z")
Hi, I have a new project to do an IA project ( predictive maintenace). I'm looking for an architecture to store the data for the Data scientist. We hesitate between PostgreSQL or Elasticsearch. My proposititon currentl… --- ## [How to improve my GeoPoint ElasticSearch Index for better response time?](https://discuss.elastic.co/t/how-to-improve-my-geopoint-elasticsearch-index-for-better-response-time/241353)
**Author:** [@zoomash](https://discuss.elastic.co/u/zoomash)\ **Replies:** 7\ **Last updated:** [July 16, 2020, 8:29am UTC](https://discuss.elastic.co/t/how-to-improve-my-geopoint-elasticsearch-index-for-better-response-time/241353 "2020-07-16T08:29:50Z")
Below is my Index (tz\_geo) and mapping details: Index setting: { "index.blocks.read\_only\_allow\_delete": "false", "index.priority": "1", "index.query.default\_field": \[ "\*" \], "index.write.wait\_for\_active\_s… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=857) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=859)