# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=790 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 791 --- ## [ELK SAML with Gsuite](https://discuss.elastic.co/t/elk-saml-with-gsuite/251462)
**Author:** [@mhr](https://discuss.elastic.co/u/mhr)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 5:12pm UTC](https://discuss.elastic.co/t/elk-saml-with-gsuite/251462 "2020-10-08T17:12:35Z")
HI Support, Do we require license version of ELK stack to use SAML with Google Gsuite. --- ## [ES master CPU spike suddenly 100% and fails to apply cluster state](https://discuss.elastic.co/t/es-master-cpu-spike-suddenly-100-and-fails-to-apply-cluster-state/251488)
**Author:** [@UDixit](https://discuss.elastic.co/u/UDixit)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 4:55pm UTC](https://discuss.elastic.co/t/es-master-cpu-spike-suddenly-100-and-fails-to-apply-cluster-state/251488 "2020-10-08T16:55:50Z")
We are currently operating with ES5.6. All nodes in the ES cluster are master and data eligible. The min\_master\_nodes is set to n/2+1. We are recently seeing issues that the matser's CPU usage spikes up very high ~90% … --- ## [BM25 normalizing per field](https://discuss.elastic.co/t/bm25-normalizing-per-field/251485)
**Author:** [@tp-c-c](https://discuss.elastic.co/u/tp-c-c)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 4:29pm UTC](https://discuss.elastic.co/t/bm25-normalizing-per-field/251485 "2020-10-08T16:29:40Z")
Hi, I have a question about the BM25 ranking algorithm, specifically about the dividing by the field length. I understand the general descriptions of the algorithm where they divide by the document length as in this for… --- ## [Email Troubleshooting](https://discuss.elastic.co/t/email-troubleshooting/251480)
**Author:** [@teej](https://discuss.elastic.co/u/teej)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 4:05pm UTC](https://discuss.elastic.co/t/email-troubleshooting/251480 "2020-10-08T16:05:26Z")
Hi, I am trying to setup alerts using email as the notification vehicle. Unfortunately, the alerts are not showing up in the inbox. So I need to trace where the failure is occurring as it may not even be an elastic issu… --- ## [Esrally performance benchmark](https://discuss.elastic.co/t/esrally-performance-benchmark/251477)
**Author:** [@kagrahar](https://discuss.elastic.co/u/kagrahar)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 3:41pm UTC](https://discuss.elastic.co/t/esrally-performance-benchmark/251477 "2020-10-08T15:41:08Z")
i am running esrally for the first time. What are the key metrics in the output that determines performance of the elastic search cluster? Any best practices to stress test? Is there a best practice for setting the numbe… --- ## [How to preserve timestamp while re-indexing?](https://discuss.elastic.co/t/how-to-preserve-timestamp-while-re-indexing/251461)
**Author:** [@srbhklkrn](https://discuss.elastic.co/u/srbhklkrn)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 2:51pm UTC](https://discuss.elastic.co/t/how-to-preserve-timestamp-while-re-indexing/251461 "2020-10-08T14:51:03Z")
I'm reindexing from 6.4 to 7.9 and while I got all documents as it is one thing I noticed timestamp is different, how do I preserve old timestamp in the document? --- ## [Can I run elasticsearch completely in memory?](https://discuss.elastic.co/t/can-i-run-elasticsearch-completely-in-memory/251063)
**Author:** [@kagrahar](https://discuss.elastic.co/u/kagrahar)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 2:38pm UTC](https://discuss.elastic.co/t/can-i-run-elasticsearch-completely-in-memory/251063 "2020-10-08T14:38:17Z")
I have about 15TB of DRAM. Can I create RAMdisk of 1TB size and completely run elastic search in memory? My data is about 300GB. Thanks in advance --- ## [7.9: "....creating index, cause \[auto(bulk api)\],..." ignores null\_values set in legacy index template](https://discuss.elastic.co/t/7-9-creating-index-cause-auto-bulk-api-ignores-null-values-set-in-legacy-index-template/251463)
**Author:** [@jze](https://discuss.elastic.co/u/jze)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 2:22pm UTC](https://discuss.elastic.co/t/7-9-creating-index-cause-auto-bulk-api-ignores-null-values-set-in-legacy-index-template/251463 "2020-10-08T14:22:29Z")
Excerpt from template: (Ruby hash) index\_patterns: \[ "ldb-dvm-\*", \], order: 200, ... ... aliases: { 'ldb-dvm' =\> {}, }, mappings: { \_source: { enabled: true, }, dynamic\_templates: \[{ string… --- ## [Ingest Pipeline parsing the wrong week](https://discuss.elastic.co/t/ingest-pipeline-parsing-the-wrong-week/251456)
**Author:** [@Antonis\_Michael](https://discuss.elastic.co/u/Antonis_Michael)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 2:06pm UTC](https://discuss.elastic.co/t/ingest-pipeline-parsing-the-wrong-week/251456 "2020-10-08T14:06:08Z")
Hello, I am trying to parse weekly indices by utilizing the ingest pipelines in elasticsearch but I keep getting a week before the actual week of the year. Would you know what the issue is? { "date\_index\_name" : { "d… --- ## [Change Primary shards number](https://discuss.elastic.co/t/change-primary-shards-number/250963)
**Author:** [@vodka\_pivo](https://discuss.elastic.co/u/vodka_pivo)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 2:00pm UTC](https://discuss.elastic.co/t/change-primary-shards-number/250963 "2020-10-08T14:00:25Z")
Hello! i use elasticsearch 7.4.0 and i want to change the primary shards number on existing indices how can i do this? i want to increase some primary shards, and decrease --- ## [Elasticsearch Index life cycle policy timing](https://discuss.elastic.co/t/elasticsearch-index-life-cycle-policy-timing/251180)
**Author:** [@jijo.john](https://discuss.elastic.co/u/jijo.john)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 1:58pm UTC](https://discuss.elastic.co/t/elasticsearch-index-life-cycle-policy-timing/251180 "2020-10-08T13:58:18Z")
Hi Team, Can you please let me know what exactly "Timing for Warm phase" means . My Hot Phase Time is 50GB or 30 days , and my index reaches 50 GB in 5 days , but i would like to keep the index in Hot phase for another… --- ## [How to get size disk closed index - Mapper size plugin](https://discuss.elastic.co/t/how-to-get-size-disk-closed-index-mapper-size-plugin/250685)
**Author:** [@Thomas74](https://discuss.elastic.co/u/Thomas74)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 1:55pm UTC](https://discuss.elastic.co/t/how-to-get-size-disk-closed-index-mapper-size-plugin/250685 "2020-10-08T13:55:17Z")
Hi, Is it possible to retrieve disk size information of closed index with mapper size plugin ? I need to create index disk usage statistics but I don't really know how to get closed index information.. Kind regards, … --- ## [Index lifecycle error, unauthorized for user](https://discuss.elastic.co/t/index-lifecycle-error-unauthorized-for-user/251442)
**Author:** [@whysmer](https://discuss.elastic.co/u/whysmer)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 12:58pm UTC](https://discuss.elastic.co/t/index-lifecycle-error-unauthorized-for-user/251442 "2020-10-08T12:58:52Z")
Hello, I have the following error for rollover & index winlogbeat: Index lifecycle error security\_exception: action \[indices:admin/rollover\] is unauthorized for user \[beats\_admin\] This user have exactly the same pe… --- ## [Nested Object Cardinality Aggregation not Working](https://discuss.elastic.co/t/nested-object-cardinality-aggregation-not-working/251400)
**Author:** [@hetvakil](https://discuss.elastic.co/u/hetvakil)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 1:49pm UTC](https://discuss.elastic.co/t/nested-object-cardinality-aggregation-not-working/251400 "2020-10-08T13:49:11Z")
I want to get Total Unique email count in a Nested object. Below is the example of the document. When we keep Nested object as property in mapping then the count for cardinality is always showing zero while if that objec… --- ## [Retrieve metricbeat parameters from Elasticsearch using java Rest high level client](https://discuss.elastic.co/t/retrieve-metricbeat-parameters-from-elasticsearch-using-java-rest-high-level-client/251416)
**Author:** [@pathfinder225](https://discuss.elastic.co/u/pathfinder225)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 10:48am UTC](https://discuss.elastic.co/t/retrieve-metricbeat-parameters-from-elasticsearch-using-java-rest-high-level-client/251416 "2020-10-08T10:48:53Z")
Hi Team, We have to retrieve below parameters from our metric beat index from elasticsearch, index pattern would look like something like metricbeat-appId All these parameters are not present in single document they ar… --- ## [Changing node.attr.data from hot to warm](https://discuss.elastic.co/t/changing-node-attr-data-from-hot-to-warm/251447)
**Author:** [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 1:27pm UTC](https://discuss.elastic.co/t/changing-node-attr-data-from-hot-to-warm/251447 "2020-10-08T13:27:21Z")
Hello, Just wondering what's the expected behaviour when we change the node.attr.data attribute from hot to warm in elasticsearch.yml and restart elasticsearch. As I'm preparing to add some new warm nodes to our cluster… --- ## [How to get the consolidated aggregartion on specfic part of String](https://discuss.elastic.co/t/how-to-get-the-consolidated-aggregartion-on-specfic-part-of-string/251432)
**Author:** [@rajan01](https://discuss.elastic.co/u/rajan01)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 12:44pm UTC](https://discuss.elastic.co/t/how-to-get-the-consolidated-aggregartion-on-specfic-part-of-string/251432 "2020-10-08T12:44:44Z")
In the elastic search we have logs with fi\_id as BA-844, BAX-844, BA-SYSTEM-844 and BA-090, BAX-090, BA-SYSTEM-WER-090 and many more with same numeric value as last part of the string. I want to write the elastic query … --- ## [Upgrade cluster to 7.9.2, master\_not\_discovered\_exception](https://discuss.elastic.co/t/upgrade-cluster-to-7-9-2-master-not-discovered-exception/251287)
**Author:** [@JSkier](https://discuss.elastic.co/u/JSkier)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 12:33pm UTC](https://discuss.elastic.co/t/upgrade-cluster-to-7-9-2-master-not-discovered-exception/251287 "2020-10-08T12:33:33Z")
Hi, This started out with Kibana not working, now seeing the root issue may be with my 3 node cluster no longer working. I get this error (telnet to 9300 works on all three search nodes): {"error":{"root\_cause":\[{"typ… --- ## [Group and keep only first result of that group](https://discuss.elastic.co/t/group-and-keep-only-first-result-of-that-group/250624)
**Author:** [@lanwin](https://discuss.elastic.co/u/lanwin)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 11:18am UTC](https://discuss.elastic.co/t/group-and-keep-only-first-result-of-that-group/250624 "2020-10-08T11:18:44Z")
Is there any way to do a search, do a group (and discard all but one result of that group)? We have a simple parent-\>children relationship (building also a group). Our problem is now that we have a complex filter and it… --- ## [Query by max date](https://discuss.elastic.co/t/query-by-max-date/251424)
**Author:** [@Kennethtruyers](https://discuss.elastic.co/u/Kennethtruyers)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 11:10am UTC](https://discuss.elastic.co/t/query-by-max-date/251424 "2020-10-08T11:10:55Z")
I have an index that is append only. Every change gets saved as a new document, together with a date property. In order to get the current active data I need to group all entities by their ID and get the one with the lat… --- ## [How to sort with case insensitive in elasticsearch](https://discuss.elastic.co/t/how-to-sort-with-case-insensitive-in-elasticsearch/251420)
**Author:** [@Mak\_Mak](https://discuss.elastic.co/u/Mak_Mak)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 10:59am UTC](https://discuss.elastic.co/t/how-to-sort-with-case-insensitive-in-elasticsearch/251420 "2020-10-08T10:59:15Z")
My index name is data\_new Below is the code to insert into index test = \[ {'id':1,'name': 'Cost Accounting 400', 'professor': \['Bill Cage', 'accounting'\]}, { 'id':2, 'name': 'Computer Internals 250', 'professor'… --- ## [Master can not be discovered](https://discuss.elastic.co/t/master-can-not-be-discovered/251237)
**Author:** [@dorcas](https://discuss.elastic.co/u/dorcas)\ **Replies:** 6\ **Last updated:** [October 8, 2020, 10:02am UTC](https://discuss.elastic.co/t/master-can-not-be-discovered/251237 "2020-10-08T10:02:14Z")
After i enabled xpack.security.enabled: true xpack.security.transport.ssl.enabled: true and i restart my two nodes i get this error master not discovered yet, this node has not previously joined a bootstrapped (v7+) … --- ## [Loosing node connection after update to 7.9.2](https://discuss.elastic.co/t/loosing-node-connection-after-update-to-7-9-2/251405)
**Author:** [@whysmer](https://discuss.elastic.co/u/whysmer)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 9:28am UTC](https://discuss.elastic.co/t/loosing-node-connection-after-update-to-7-9-2/251405 "2020-10-08T09:28:18Z")
Hello, I have one node which is not reconecting after updating from 7.7.1 to 7.9.2 my cluster of three nodes. Here is the error, I don't understand why it's a connection problem as nothing changed ? \[2020-10-08T11:22… --- ## [Get the top document from each group with sorting and paging](https://discuss.elastic.co/t/get-the-top-document-from-each-group-with-sorting-and-paging/251003)
**Author:** [@Sergii](https://discuss.elastic.co/u/Sergii)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 8:34am UTC](https://discuss.elastic.co/t/get-the-top-document-from-each-group-with-sorting-and-paging/251003 "2020-10-08T08:34:36Z")
Hi! I have a collection of entities and I need to group them by RelationId and select entity with biggest Revision in each group. Result should be ordered by "title" and support paging. Is it possible to do this with El… --- ## [Limit script\_score rescoring to the top query matches](https://discuss.elastic.co/t/limit-script-score-rescoring-to-the-top-query-matches/251247)
**Author:** [@fpug](https://discuss.elastic.co/u/fpug)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 8:35am UTC](https://discuss.elastic.co/t/limit-script-score-rescoring-to-the-top-query-matches/251247 "2020-10-08T08:35:15Z")
Hi! I'm trying to score some documents via their a dense\_vector field, via a script\_score query. The cosine computation is relatively expensive, so I'd like to limit it to the best scoring results of the "query" clause… --- ## [Prettify Watcher JSON Response](https://discuss.elastic.co/t/prettify-watcher-json-response/250947)
**Author:** [@Porton](https://discuss.elastic.co/u/Porton)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 8:31am UTC](https://discuss.elastic.co/t/prettify-watcher-json-response/250947 "2020-10-08T08:31:38Z")
What is the best way to pretty JSON response from Elasticsearch Watch ? Currently doing webhook to MS Teams, body": "{ title: 'Server Metrics Alert', text: '{{#toJson}}ctx.payload.hits{{/toJson}}' }", Everything works… --- ## [How to get around the problem with Incompatible types found in multi-mapping during reading data by Spark from Elasticsearch?](https://discuss.elastic.co/t/how-to-get-around-the-problem-with-incompatible-types-found-in-multi-mapping-during-reading-data-by-spark-from-elasticsearch/251389)
**Author:** [@dlubom](https://discuss.elastic.co/u/dlubom)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 7:48am UTC](https://discuss.elastic.co/t/how-to-get-around-the-problem-with-incompatible-types-found-in-multi-mapping-during-reading-data-by-spark-from-elasticsearch/251389 "2020-10-08T07:48:03Z")
How to get around this problem (without reindexing): Py4JJavaError: An error occurred while calling o2649.load. : org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Incompatible types found in multi-mapping: Fie… --- ## [Pointing to different Nodes](https://discuss.elastic.co/t/pointing-to-different-nodes/251198)
**Author:** [@Jtomage](https://discuss.elastic.co/u/Jtomage)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 7:36am UTC](https://discuss.elastic.co/t/pointing-to-different-nodes/251198 "2020-10-08T07:36:17Z")
Hi all, New to elasticsearch I am currently for clarification on pointing to nodes with different roles. The current set up is a 3 Master-data node cluster and looking to possibly scale out. Currently there are 2 applic… --- ## [\[Elastic Cloud\] Shipped JSON logs sometimes not parsed](https://discuss.elastic.co/t/elastic-cloud-shipped-json-logs-sometimes-not-parsed/251382)
**Author:** [@val](https://discuss.elastic.co/u/val)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 7:06am UTC](https://discuss.elastic.co/t/elastic-cloud-shipped-json-logs-sometimes-not-parsed/251382 "2020-10-08T07:06:51Z")
Recently, Elastic Cloud has added the ability to also ship logs to a remote monitoring cluster. Enabling that feature worked great, however, I'm seeing parsing issues in the elasticsearch.server dataset. Two subsequent … --- ## [Queries get slow while indexing documents](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243)
**Author:** [@taichi](https://discuss.elastic.co/u/taichi)\ **Replies:** 8\ **Last updated:** [October 8, 2020, 6:52am UTC](https://discuss.elastic.co/t/queries-get-slow-while-indexing-documents/251243 "2020-10-08T06:52:01Z")
I have an index with approximately 2.3M documents. It usually returns results of query within 1~2 seconds, but it gets very slow (sometimes as long as 20~30s) while I'm inserting data into the index. I insert data in R… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=789) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=791)