# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=613 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 614 --- ## [Creating watcher for index rate to alert in case of excessive logging](https://discuss.elastic.co/t/creating-watcher-for-index-rate-to-alert-in-case-of-excessive-logging/278125)
**Author:** [@MarcelVarallo](https://discuss.elastic.co/u/MarcelVarallo)\ **Replies:** 0\ **Last updated:** [July 8, 2021, 2:50am UTC](https://discuss.elastic.co/t/creating-watcher-for-index-rate-to-alert-in-case-of-excessive-logging/278125 "2021-07-08T02:50:49Z")
How would you create a watcher to email alert you when the index rate for an index exceeds something stupid. Use Case: When everything is fine we don't log too much to this index but if one of our apps has a problem it… --- ## [Unknown time-zone ID: strict\_date\_hour\_minute\_second](https://discuss.elastic.co/t/unknown-time-zone-id-strict-date-hour-minute-second/277933)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 2\ **Last updated:** [July 7, 2021, 11:56pm UTC](https://discuss.elastic.co/t/unknown-time-zone-id-strict-date-hour-minute-second/277933 "2021-07-07T23:56:51Z")
I read the document and wrote a DSL query, but there was an error. What is a time zone ID? It's not mentioned in the document. quote doc date\_hour\_minute\_second or strict\_date\_hour\_minute\_second A formatter that com… --- ## [Why elasticsearch cannot show manually inserted data?](https://discuss.elastic.co/t/why-elasticsearch-cannot-show-manually-inserted-data/277946)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 2\ **Last updated:** [July 7, 2021, 11:54pm UTC](https://discuss.elastic.co/t/why-elasticsearch-cannot-show-manually-inserted-data/277946 "2021-07-07T23:54:44Z")
I mean, I copied a doc data and then inserted it back into elastic search. The data did not appear. I try to select the largest time range of query,but still no results. What changes do I need to make to successfull… --- ## [New documents are not getting added to an already existing index](https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026)
**Author:** [@Krishna\_Sathya\_A\_C](https://discuss.elastic.co/u/Krishna_Sathya_A_C)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 11:37pm UTC](https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026 "2021-07-07T23:37:38Z")
Hi, I am new to the ELK stack workflow, currently I am doing a project that pulls live feed from twitter every time a user requests and I want to display the results using Kibana. In my project instead of using Logstash… --- ## [\[CASE\] how to join a physical cluster with elastic cloud](https://discuss.elastic.co/t/case-how-to-join-a-physical-cluster-with-elastic-cloud/278109)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 3\ **Last updated:** [July 7, 2021, 10:03pm UTC](https://discuss.elastic.co/t/case-how-to-join-a-physical-cluster-with-elastic-cloud/278109 "2021-07-07T22:03:59Z")
hello everyone, i need your help to know how i can join a physical cluster, i mean, i have ELK installed in a VM (local) and i want to know if i can join it to my elastic cloud in the cloud where i have my indexes of com… --- ## [Script Field - version change](https://discuss.elastic.co/t/script-field-version-change/278098)
**Author:** [@angelogabeira](https://discuss.elastic.co/u/angelogabeira)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 5:06pm UTC](https://discuss.elastic.co/t/script-field-version-change/278098 "2021-07-07T17:06:25Z")
guys, I'm changing from version 6.5 to version 7.9 and I need to redo the fields scripts. They're not working the same way, I've done some research but I couldn't. this example is what i did in 6.5 and it's working, i … --- ## [Conditional Transfer of updated data between Clusters](https://discuss.elastic.co/t/conditional-transfer-of-updated-data-between-clusters/278105)
**Author:** [@Wondwosen\_Eshetu](https://discuss.elastic.co/u/Wondwosen_Eshetu)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 6:20pm UTC](https://discuss.elastic.co/t/conditional-transfer-of-updated-data-between-clusters/278105 "2021-07-07T18:20:27Z")
Hello, I want to transfer updated data between two clusters on a daily basis (Periodically). The process fetches modified data from first cluster and update the second (if the document exists and its modified time stamp… --- ## [Painless updates are not searchable](https://discuss.elastic.co/t/painless-updates-are-not-searchable/277518)
**Author:** [@dude88](https://discuss.elastic.co/u/dude88)\ **Replies:** 12\ **Last updated:** [July 7, 2021, 5:54pm UTC](https://discuss.elastic.co/t/painless-updates-are-not-searchable/277518 "2021-07-07T17:54:17Z")
I am trying to do an updateByQuery with the javascript api, and am using painless. Before running updateByQuery, I first update the index's mapping by adding in the following: await putMapping(elastic, index, { pr… --- ## [Migration From Windows Cluster to Linux Cluster](https://discuss.elastic.co/t/migration-from-windows-cluster-to-linux-cluster/278097)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 4:48pm UTC](https://discuss.elastic.co/t/migration-from-windows-cluster-to-linux-cluster/278097 "2021-07-07T16:48:18Z")
I am looking at migrating our single node Windows Elasticsearch instance to a 3-node Linux cluster. We currently have about 10TB of data that is indexed as both traditional indices and also data streams. How would I g… --- ## [Wait\_for not working as expected](https://discuss.elastic.co/t/wait-for-not-working-as-expected/278001)
**Author:** [@jbalaguero](https://discuss.elastic.co/u/jbalaguero)\ **Replies:** 2\ **Last updated:** [July 7, 2021, 4:48pm UTC](https://discuss.elastic.co/t/wait-for-not-working-as-expected/278001 "2021-07-07T16:48:14Z")
Hello, I'm performing a test indexing documents with 'wait\_for' refresh policy using a bulk processor. This bulk processor flushes documents every 500ms. The index refresh interval is 60s for this example. When I perfo… --- ## [Hard time setting up monitoring cluster using metricbeat](https://discuss.elastic.co/t/hard-time-setting-up-monitoring-cluster-using-metricbeat/278088)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 3:38pm UTC](https://discuss.elastic.co/t/hard-time-setting-up-monitoring-cluster-using-metricbeat/278088 "2021-07-07T15:38:39Z")
I have follow few document of elk. it making me more confuse what am I missing? this is my simple test that I am trying to run. elkprod01 single node cluster elkmonprod01 is single node cluster ( which I would like t… --- ## [Enable \_size in index template](https://discuss.elastic.co/t/enable-size-in-index-template/278055)
**Author:** [@radovan](https://discuss.elastic.co/u/radovan)\ **Replies:** 7\ **Last updated:** [July 7, 2021, 3:25pm UTC](https://discuss.elastic.co/t/enable-size-in-index-template/278055 "2021-07-07T15:25:28Z")
Hi, I've installed mapper size plugin on all of our elastic nodes, I can see/use the size field, when I enable it on an index: PUT index-xxx/\_mapping { "\_size": { "enabled": true } } and create a scripted fiel… --- ## [Intermittent Problem with nssm and Kibana / Logstash](https://discuss.elastic.co/t/intermittent-problem-with-nssm-and-kibana-logstash/278086)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 3:12pm UTC](https://discuss.elastic.co/t/intermittent-problem-with-nssm-and-kibana-logstash/278086 "2021-07-07T15:12:42Z")
Good Morning, I'm having an intermittent issue with Logstash / Kibana services created using nssm and am wondering if anybody else has experienced something similar and if so were they able to resolve the issue. My iss… --- ## [How to sort an elasticsearch query based on the size of nested objects that match a condition?](https://discuss.elastic.co/t/how-to-sort-an-elasticsearch-query-based-on-the-size-of-nested-objects-that-match-a-condition/277810)
**Author:** [@EmilieM](https://discuss.elastic.co/u/EmilieM)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 2:55pm UTC](https://discuss.elastic.co/t/how-to-sort-an-elasticsearch-query-based-on-the-size-of-nested-objects-that-match-a-condition/277810 "2021-07-07T14:55:55Z")
I try to sort my elastic search query based on the size of nested objects that match a condition (here: size of boosters with a specific status) Here is my mapping : { "companies" : { "mappings" : { "dynami… --- ## [ES queries became very slow after upgrading from 7.11 to 7.13.2](https://discuss.elastic.co/t/es-queries-became-very-slow-after-upgrading-from-7-11-to-7-13-2/277932)
**Author:** [@OmarDacca](https://discuss.elastic.co/u/OmarDacca)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 2:52pm UTC](https://discuss.elastic.co/t/es-queries-became-very-slow-after-upgrading-from-7-11-to-7-13-2/277932 "2021-07-07T14:52:50Z")
Hi, ES queries became very slow after upgrading from 7.11 to 7.13.2. what can I do ? what changes made in 7.12 and 7.13 that may affect the performance ? Thanks --- ## [Query containing special character is not working](https://discuss.elastic.co/t/query-containing-special-character-is-not-working/277080)
**Author:** [@ElektroKinetik](https://discuss.elastic.co/u/ElektroKinetik)\ **Replies:** 2\ **Last updated:** [July 7, 2021, 2:35pm UTC](https://discuss.elastic.co/t/query-containing-special-character-is-not-working/277080 "2021-07-07T14:35:13Z")
This works! { "match\_phrase": { "fields.customer": { "query": “\\"MFG OPS AC DEV\\"" } } }, This doe's … --- ## [Elasticsearch-keystore list command could not find java](https://discuss.elastic.co/t/elasticsearch-keystore-list-command-could-not-find-java/278069)
**Author:** [@Prabaharan\_Kathiresa](https://discuss.elastic.co/u/Prabaharan_Kathiresa)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 12:27pm UTC](https://discuss.elastic.co/t/elasticsearch-keystore-list-command-could-not-find-java/278069 "2021-07-07T12:27:02Z")
Hi Team we are migrating ES 6.8.5 Java environment from Java8 to Java11. we are using oraclejdk11 as our Java vendor. we set Java\_home in bin/elasticsearch file like below usr/java/jdk11 In ansbile below task is fa… --- ## [I am trying to change the values of the fields while querying the data to Vega](https://discuss.elastic.co/t/i-am-trying-to-change-the-values-of-the-fields-while-querying-the-data-to-vega/277995)
**Author:** [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 12:47pm UTC](https://discuss.elastic.co/t/i-am-trying-to-change-the-values-of-the-fields-while-querying-the-data-to-vega/277995 "2021-07-07T12:47:01Z")
So, I have data something like this And I want the Data to be like : Except for Closed and Submitted states it should change the value to In Progress I know how this can be achieved through the scripted fields, but… --- ## [What is considered a "large arbitrary set of keys", for the nested mapping?](https://discuss.elastic.co/t/what-is-considered-a-large-arbitrary-set-of-keys-for-the-nested-mapping/278064)
**Author:** [@Lockhead](https://discuss.elastic.co/u/Lockhead)\ **Replies:** 2\ **Last updated:** [July 7, 2021, 12:42pm UTC](https://discuss.elastic.co/t/what-is-considered-a-large-arbitrary-set-of-keys-for-the-nested-mapping/278064 "2021-07-07T12:42:46Z")
On Nested field type | Elasticsearch Guide \[7.13\] | Elastic you can find this statement: "When ingesting key-value pairs with a large, arbitrary set of keys, you might consider modeling each key-value pair as its own nes… --- ## [Entity-centric indexing with Transforms](https://discuss.elastic.co/t/entity-centric-indexing-with-transforms/277798)
**Author:** [@katja1](https://discuss.elastic.co/u/katja1)\ **Replies:** 4\ **Last updated:** [July 7, 2021, 12:34pm UTC](https://discuss.elastic.co/t/entity-centric-indexing-with-transforms/277798 "2021-07-07T12:34:51Z")
Hello, we're working with Elasticsearch for the first time and we are currently deciding on what would be the best solution for our problem at hand. We are receiving Event based logs (in JSON form) from our applications… --- ## [Get rid of message: ... waiting for elected master node... to setup local exporter... (does it have x-pack installed?)](https://discuss.elastic.co/t/get-rid-of-message-waiting-for-elected-master-node-to-setup-local-exporter-does-it-have-x-pack-installed/277999)
**Author:** [@Dyanne\_Ostlind](https://discuss.elastic.co/u/Dyanne_Ostlind)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 12:28pm UTC](https://discuss.elastic.co/t/get-rid-of-message-waiting-for-elected-master-node-to-setup-local-exporter-does-it-have-x-pack-installed/277999 "2021-07-07T12:28:36Z")
How do I shut off this message? I'm using the fully open-source version of ES 7.5.2: \[2020-02-20T00:22:18,086\]\[INFO \]\[o.e.x.m.e.l.LocalExporter\] \[coordinator6\] waiting for elected master node \[{master1}{68sPQmYrRdW60YXS… --- ## [ES is not coming up after changing JAVA\_HOME](https://discuss.elastic.co/t/es-is-not-coming-up-after-changing-java-home/277835)
**Author:** [@Prabaharan\_Kathiresa](https://discuss.elastic.co/u/Prabaharan_Kathiresa)\ **Replies:** 3\ **Last updated:** [July 7, 2021, 12:05pm UTC](https://discuss.elastic.co/t/es-is-not-coming-up-after-changing-java-home/277835 "2021-07-07T12:05:15Z")
Hi Team, We are running ES 6.8.5 in unix machine and its running on 1.8. We decided to promote java 1.8 to java11 oracle jdk. Can someone advise me what should i do now ? Steps we did Stopped ES 6.8.5 from Unix Mac… --- ## [As best practice as there any limit on number of indices in elastic search](https://discuss.elastic.co/t/as-best-practice-as-there-any-limit-on-number-of-indices-in-elastic-search/278049)
**Author:** [@mba](https://discuss.elastic.co/u/mba)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 11:28am UTC](https://discuss.elastic.co/t/as-best-practice-as-there-any-limit-on-number-of-indices-in-elastic-search/278049 "2021-07-07T11:28:13Z")
I want to do logging of 100+ microservices. So is it good practice to have separate index for every microservice (index=MS name) ?? --- ## [Activate Xpack + Watcher for existing Elasticsearch installation](https://discuss.elastic.co/t/activate-xpack-watcher-for-existing-elasticsearch-installation/278057)
**Author:** [@jdbcplusnet](https://discuss.elastic.co/u/jdbcplusnet)\ **Replies:** 1\ **Last updated:** [July 7, 2021, 11:22am UTC](https://discuss.elastic.co/t/activate-xpack-watcher-for-existing-elasticsearch-installation/278057 "2021-07-07T11:22:12Z")
Hi all My company has already installed Elasticsearch + Kibana 6.8.7 with Basic License. We need to use Watcher API from Xpack plugin, so I guess we should buy a License for it. Can someone describe the process in case… --- ## [ElasticSearch Goes Down (Ram issue)](https://discuss.elastic.co/t/elasticsearch-goes-down-ram-issue/277696)
**Author:** [@bhamidpour](https://discuss.elastic.co/u/bhamidpour)\ **Replies:** 6\ **Last updated:** [July 7, 2021, 10:42am UTC](https://discuss.elastic.co/t/elasticsearch-goes-down-ram-issue/277696 "2021-07-07T10:42:00Z")
Hello, I assigned 3GB ram to elasticSearch and also I used one cluster with 3 nodes for it. 3 JDBC pipelines (log data with logstash) run on it and I want to extend it in the future too. and also I used Docker to run ELK… --- ## [Endless Exceptions loop fixed or not?](https://discuss.elastic.co/t/endless-exceptions-loop-fixed-or-not/278047)
**Author:** [@pascalku](https://discuss.elastic.co/u/pascalku)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 8:59am UTC](https://discuss.elastic.co/t/endless-exceptions-loop-fixed-or-not/278047 "2021-07-07T08:59:46Z")
Hi there, I am running into an endless loop of exceptions (see: Fix Infinite Loops in ExceptionsHelper#unwrap by original-brownbear · Pull Request #42716 · elastic/elasticsearch · GitHub). According to your release docu… --- ## [Can the split “query” and “aggregations” achieve higher performance？](https://discuss.elastic.co/t/can-the-split-query-and-aggregations-achieve-higher-performance/278034)
**Author:** [@Ryze](https://discuss.elastic.co/u/Ryze)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 7:32am UTC](https://discuss.elastic.co/t/can-the-split-query-and-aggregations-achieve-higher-performance/278034 "2021-07-07T07:32:48Z")
Imagine I have an index "article" with 100 million pieces of data. Its template data and mapping file are as follows. What I want to ask is, does splitting query and aggs (The following query1) and then using \_bulk api … --- ## [Fields removed from Filebeat Zeek Module/Pipeline](https://discuss.elastic.co/t/fields-removed-from-filebeat-zeek-module-pipeline/278029)
**Author:** [@valhalla](https://discuss.elastic.co/u/valhalla)\ **Replies:** 0\ **Last updated:** [July 7, 2021, 6:57am UTC](https://discuss.elastic.co/t/fields-removed-from-filebeat-zeek-module-pipeline/278029 "2021-07-07T06:57:19Z")
Hi team, I need some help regarding some fields that are being removed from one version to another. I am using ES 7.13.2. Some fields from this Zeek module/pipeline were available under older ES/ ECS versions, that are … --- ## [Why is only one node processing the query?](https://discuss.elastic.co/t/why-is-only-one-node-processing-the-query/275070)
**Author:** [@Aurel\_Drejta](https://discuss.elastic.co/u/Aurel_Drejta)\ **Replies:** 11\ **Last updated:** [July 6, 2021, 11:23pm UTC](https://discuss.elastic.co/t/why-is-only-one-node-processing-the-query/275070 "2021-07-06T23:23:35Z")
Hi everyone. I had an index that had 5 primary shards and around 250GB of data in total (primaries only). I decided to reindex it with more primary shards (30) to see if I would have performance improvements. The index … --- ## [Query randomly fail](https://discuss.elastic.co/t/query-randomly-fail/277880)
**Author:** [@ClubberLang](https://discuss.elastic.co/u/ClubberLang)\ **Replies:** 2\ **Last updated:** [July 6, 2021, 8:58pm UTC](https://discuss.elastic.co/t/query-randomly-fail/277880 "2021-07-06T20:58:51Z")
Hi, I have a website installed since month now, working perfectly. Nothing has changed, no deploy etc... But now, I test a specific query, sometime it work and sometimes it failed! It is completely random. So, I have … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=612) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=614)