# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=957 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 958 --- ## [IML for daily indices - how?](https://discuss.elastic.co/t/iml-for-daily-indices-how/227311)
**Author:** [@Rohlik](https://discuss.elastic.co/u/Rohlik)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 10:13pm UTC](https://discuss.elastic.co/t/iml-for-daily-indices-how/227311 "2020-04-13T22:13:14Z")
Hello gyus, how properly setup index lifecycle management in Elastic stack 7.6 if using daily indices? I found this docs - https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle… --- ## [Lifecycle Policies current action blank](https://discuss.elastic.co/t/lifecycle-policies-current-action-blank/226718)
**Author:** [@pitcher2](https://discuss.elastic.co/u/pitcher2)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 10:11pm UTC](https://discuss.elastic.co/t/lifecycle-policies-current-action-blank/226718 "2020-04-13T22:11:45Z")
I had an Elasticsearch cluster running on an aging setup that had lifecycle policies working. I setup a brand new cluster and want to do different lifecycle policies based on the new resources. I did a snapshot of my d… --- ## [How is an 'avg' aggregation updated in transforms when old records are deleted?](https://discuss.elastic.co/t/how-is-an-avg-aggregation-updated-in-transforms-when-old-records-are-deleted/227013)
**Author:** [@duttad](https://discuss.elastic.co/u/duttad)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 9:23pm UTC](https://discuss.elastic.co/t/how-is-an-avg-aggregation-updated-in-transforms-when-old-records-are-deleted/227013 "2020-04-13T21:23:05Z")
I am creating a transformed index from a source index. (https://www.elastic.co/guide/en/elasticsearch/reference/master/transform-overview.html) Logs older than 5 days are deleted from the source index. When computing an… --- ## [Limit of total fields \[1000\], but there are not 1000 fields indexed](https://discuss.elastic.co/t/limit-of-total-fields-1000-but-there-are-not-1000-fields-indexed/227802)
**Author:** [@deepak643](https://discuss.elastic.co/u/deepak643)\ **Replies:** 4\ **Last updated:** [April 13, 2020, 8:17pm UTC](https://discuss.elastic.co/t/limit-of-total-fields-1000-but-there-are-not-1000-fields-indexed/227802 "2020-04-13T20:17:19Z")
Team, I am facing the issue with ES on the error "Limit of total fields \[1000\]". I have three different environments and have never faced such issue and its always same kind of data gets indexed. We spinned up new envir… --- ## [Initializing a SearchResponse object to null with the high level REST client](https://discuss.elastic.co/t/initializing-a-searchresponse-object-to-null-with-the-high-level-rest-client/227705)
**Author:** [@gideongrossman](https://discuss.elastic.co/u/gideongrossman)\ **Replies:** 5\ **Last updated:** [April 13, 2020, 8:08pm UTC](https://discuss.elastic.co/t/initializing-a-searchresponse-object-to-null-with-the-high-level-rest-client/227705 "2020-04-13T20:08:10Z")
Can anyone shed light on how to achieve this initialization or a workaround? I am trying to initialize the SearchResponse object before a conditional loop and populate the object inside the loop. The motivation is to mak… --- ## [To collect logs from remote server](https://discuss.elastic.co/t/to-collect-logs-from-remote-server/227798)
**Author:** [@rakeshraagu2](https://discuss.elastic.co/u/rakeshraagu2)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 6:32pm UTC](https://discuss.elastic.co/t/to-collect-logs-from-remote-server/227798 "2020-04-13T18:32:09Z")
Hi all, I have a elk stack on my server which consists of elasticsearch,beats and kibana. Now I need to collect logs from a remote server using filebeat. What all configurations are need to done on the elk node and remo… --- ## [100 percent cpu utilization on single node elasticsearch](https://discuss.elastic.co/t/100-percent-cpu-utilization-on-single-node-elasticsearch/226178)
**Author:** [@Shubham\_Mittal](https://discuss.elastic.co/u/Shubham_Mittal)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 5:38pm UTC](https://discuss.elastic.co/t/100-percent-cpu-utilization-on-single-node-elasticsearch/226178 "2020-04-13T17:38:26Z")
Hi Team, Facing 100 cpu utilization when running multiple search requests with only 10 users on an AWS Windows server, with 2 core cpu and 16 GB RAM, with 2 GB allocated as HEAP in elasticsearch. Can anyone suggest ho… --- ## [Elasticsearch Cross Cluster Replication connect\_timeout](https://discuss.elastic.co/t/elasticsearch-cross-cluster-replication-connect-timeout/227799)
**Author:** [@Matt\_G](https://discuss.elastic.co/u/Matt_G)\ **Replies:** 3\ **Last updated:** [April 13, 2020, 4:37pm UTC](https://discuss.elastic.co/t/elasticsearch-cross-cluster-replication-connect-timeout/227799 "2020-04-13T16:37:22Z")
We are running 2 instances of elasticsearch running version 6.8.1. The first instance in our production environment and the second one is in our development environment. We're attempting to replicate the data from the pr… --- ## [Composite aggregation ordered by time over entire data in database](https://discuss.elastic.co/t/composite-aggregation-ordered-by-time-over-entire-data-in-database/227106)
**Author:** [@oridool](https://discuss.elastic.co/u/oridool)\ **Replies:** 9\ **Last updated:** [April 13, 2020, 4:24pm UTC](https://discuss.elastic.co/t/composite-aggregation-ordered-by-time-over-entire-data-in-database/227106 "2020-04-13T16:24:32Z")
Hello, I have a requirement to return aggregations over multiple fields, and present in UI the returned aggregations order by time descending. I wrote the query below which seems to work. The only problem I noticed is… --- ## [Esrally exception](https://discuss.elastic.co/t/esrally-exception/226217)
**Author:** [@kinlee](https://discuss.elastic.co/u/kinlee)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 4:19pm UTC](https://discuss.elastic.co/t/esrally-exception/226217 "2020-04-13T16:19:14Z")
Hi while benchmarking my cluster on m5l with 2core and 8g memory I get all indexing time, merge time = 0. in the log I get lots of exception '''\* During handling of the above exception, another exception occurred: '… --- ## [Temporarily prevent Elastic from accepting requests](https://discuss.elastic.co/t/temporarily-prevent-elastic-from-accepting-requests/227521)
**Author:** [@jasonperrone](https://discuss.elastic.co/u/jasonperrone)\ **Replies:** 4\ **Last updated:** [April 13, 2020, 2:40pm UTC](https://discuss.elastic.co/t/temporarily-prevent-elastic-from-accepting-requests/227521 "2020-04-13T14:40:36Z")
I am practicing doing an upgrade of Elastic 7.0.0 -\> 7.6.2 in a UAT instance. Here's what keeps happening. On the old instance I create a snapshot. I shut it down. I turn on the new instance. I try to restore the snapsho… --- ## [Elasticsearch not starting as a service in RHEL 8](https://discuss.elastic.co/t/elasticsearch-not-starting-as-a-service-in-rhel-8/227758)
**Author:** [@gefela](https://discuss.elastic.co/u/gefela)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 2:22pm UTC](https://discuss.elastic.co/t/elasticsearch-not-starting-as-a-service-in-rhel-8/227758 "2020-04-13T14:22:58Z")
Hi When I try to start elasticsearch using the following command, I get the following error message \[root@gefela ansible\_junos\]# systemctl restart logstash elasticsearch kibana Job for elasticsearch.service failed bec… --- ## [Tshark output support](https://discuss.elastic.co/t/tshark-output-support/226372)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 7\ **Last updated:** [April 13, 2020, 2:15pm UTC](https://discuss.elastic.co/t/tshark-output-support/226372 "2020-04-13T14:15:12Z")
Which versions of Tshark json for ES are supported? Looks like old version (v2) is working fine when in new (v3) output didn't get parsed in latest ES (accually why, because it have no sense because this output is dedica… --- ## [Not able join two nodes elastic search clustering](https://discuss.elastic.co/t/not-able-join-two-nodes-elastic-search-clustering/227771)
**Author:** [@abhishek1](https://discuss.elastic.co/u/abhishek1)\ **Replies:** 4\ **Last updated:** [April 13, 2020, 2:08pm UTC](https://discuss.elastic.co/t/not-able-join-two-nodes-elastic-search-clustering/227771 "2020-04-13T14:08:13Z")
I am not able to join two nodes while configuring cluster setup for elasticsearch 7.4.2 version. I am getting below error org.elasticsearch.transport.RemoteTransportException: \[node-1\]\[10.112.2.93:9300\]\[internal:cluste… --- ## [Exists query with highlighting](https://discuss.elastic.co/t/exists-query-with-highlighting/223526)
**Author:** [@Jogi\_Michal](https://discuss.elastic.co/u/Jogi_Michal)\ **Replies:** 3\ **Last updated:** [March 16, 2020, 1:41pm UTC](https://discuss.elastic.co/t/exists-query-with-highlighting/223526 "2020-03-16T13:41:14Z")
Is there a possibility to use elastic search exists query with highlights? What I need achieve is to find all documents with some field and I'm using the exists query for this (it works fine) but additionally, I wan… --- ## [30 million documents in one index best hardware to use](https://discuss.elastic.co/t/30-million-documents-in-one-index-best-hardware-to-use/227757)
**Author:** [@Bharatwaj\_Sriram](https://discuss.elastic.co/u/Bharatwaj_Sriram)\ **Replies:** 5\ **Last updated:** [April 13, 2020, 1:25pm UTC](https://discuss.elastic.co/t/30-million-documents-in-one-index-best-hardware-to-use/227757 "2020-04-13T13:25:59Z")
Hi elasticsearch lovers, I really appreciate the fact that these types of questions get asked a lot in the community and a lot of these get answered. I have been through a lot of similar threads and i learn something ne… --- ## [Elasticsearch operator deletes es pods which are scaled by Horizontal autoscaler](https://discuss.elastic.co/t/elasticsearch-operator-deletes-es-pods-which-are-scaled-by-horizontal-autoscaler/227775)
**Author:** [@kumar\_kung](https://discuss.elastic.co/u/kumar_kung)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 1:11pm UTC](https://discuss.elastic.co/t/elasticsearch-operator-deletes-es-pods-which-are-scaled-by-horizontal-autoscaler/227775 "2020-04-13T13:11:34Z")
Kibana version : 7.5.2 Elasticsearch version : 7.5.2 ECK-1.0.0 GA I am using elasticsearch quickstart from eck guide i autoscale when average mem hits 60%, when ever a new pod is created through autoscaling, elasti… --- ## [Configuring elk 7.6.2](https://discuss.elastic.co/t/configuring-elk-7-6-2/227773)
**Author:** [@amanehsan](https://discuss.elastic.co/u/amanehsan)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 12:32pm UTC](https://discuss.elastic.co/t/configuring-elk-7-6-2/227773 "2020-04-13T12:32:35Z")
Hi I am upgrading elk to latest version. I am unable to understand the master notes settings in discovery. Previously I was not giving any such information. Please help me understand which IPs are to be given here. What … --- ## [Create Horizontal pod autoscaling for elasticsearch in kubernetes (eck operator)](https://discuss.elastic.co/t/create-horizontal-pod-autoscaling-for-elasticsearch-in-kubernetes-eck-operator/227753)
**Author:** [@kumar\_kung](https://discuss.elastic.co/u/kumar_kung)\ **Replies:** 5\ **Last updated:** [April 13, 2020, 12:03pm UTC](https://discuss.elastic.co/t/create-horizontal-pod-autoscaling-for-elasticsearch-in-kubernetes-eck-operator/227753 "2020-04-13T12:03:31Z")
ECK = 1.0.0 ga Cloud= EKS Kibana version : 7.5.2 Elasticsearch version : 7.5.2 EKS: 1.15 Please provide details for Horizontal pod autoscaling for elasticsearch in kubernetes (eck operator) --- ## [How to debug Elasticsearch source code in Idea with Security disabled](https://discuss.elastic.co/t/how-to-debug-elasticsearch-source-code-in-idea-with-security-disabled/227763)
**Author:** [@unnisworld](https://discuss.elastic.co/u/unnisworld)\ **Replies:** 0\ **Last updated:** [April 13, 2020, 10:59am UTC](https://discuss.elastic.co/t/how-to-debug-elasticsearch-source-code-in-idea-with-security-disabled/227763 "2020-04-13T10:59:33Z")
I have setup the Elasticsearch debug environment (for 6.8.1) using the instructions given here - https://www.elastic.co/blog/how-to-debug-elasticsearch-source-code-in-intellij-idea When I run Elasticsearch using ./gradl… --- ## [How many nodes should an ElasticSearch cluster have?](https://discuss.elastic.co/t/how-many-nodes-should-an-elasticsearch-cluster-have/227674)
**Author:** [@Robert\_Ga](https://discuss.elastic.co/u/Robert_Ga)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 9:00am UTC](https://discuss.elastic.co/t/how-many-nodes-should-an-elasticsearch-cluster-have/227674 "2020-04-13T09:00:47Z")
Hi everyone! I am trying to understand how to better configure my ELS Cluster. From what I understand, a node is an Elasticsearch instance (please correct me if I\`m wrong). What I want to know is: Should each node … --- ## [Max number of shards for a single node](https://discuss.elastic.co/t/max-number-of-shards-for-a-single-node/226952)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 8:49am UTC](https://discuss.elastic.co/t/max-number-of-shards-for-a-single-node/226952 "2020-04-13T08:49:25Z")
I have an elasticsearch node with different hw performance compared to others. I would like to set a lower limit of shards for this particular node. It's possible to do? --- ## [The cluster often reports an error \[service\_unavailable / 2 / no master\]](https://discuss.elastic.co/t/the-cluster-often-reports-an-error-service-unavailable-2-no-master/227739)
**Author:** [@xujingrui](https://discuss.elastic.co/u/xujingrui)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 8:47am UTC](https://discuss.elastic.co/t/the-cluster-often-reports-an-error-service-unavailable-2-no-master/227739 "2020-04-13T08:47:12Z")
HI! The \[service ﹣ unavailable / 2 / no master\] error message often occurs in the cluster master node. Have you ever encountered it? It will appear after running for a period of time. After an error is reported, you can… --- ## [Spreading files across data paths](https://discuss.elastic.co/t/spreading-files-across-data-paths/227026)
**Author:** [@Flatline2005](https://discuss.elastic.co/u/Flatline2005)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 8:41am UTC](https://discuss.elastic.co/t/spreading-files-across-data-paths/227026 "2020-04-13T08:41:58Z")
There are lots of docs talking about the distribution of shards across nodes in a cluster, but very little about the actual usage of multiple data paths within nodes. It is clear that a shard cannot be split over differ… --- ## [How elasticsearch uses memory](https://discuss.elastic.co/t/how-elasticsearch-uses-memory/227462)
**Author:** [@ahmed\_charafouddine](https://discuss.elastic.co/u/ahmed_charafouddine)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 8:21am UTC](https://discuss.elastic.co/t/how-elasticsearch-uses-memory/227462 "2020-04-13T08:21:28Z")
Hello, how elasticsearch uses memory ? Question which could appear naive but I cannot find a satisfactory solution. If anyone could help me. thank you --- ## [Transforming multiple documents to a single document for a Kibana visualization](https://discuss.elastic.co/t/transforming-multiple-documents-to-a-single-document-for-a-kibana-visualization/227642)
**Author:** [@Rowan](https://discuss.elastic.co/u/Rowan)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 8:12am UTC](https://discuss.elastic.co/t/transforming-multiple-documents-to-a-single-document-for-a-kibana-visualization/227642 "2020-04-13T08:12:13Z")
Hi, I have the following documents in the same Index timestamp | sessionID=123 | IPAddress="1.1.1.1" timestamp | sessionID=123 | username=harry timestamp | sessionID=123 | error=1002 timestamp | sessionID=123 | reconne… --- ## [Indices are not showing for Fluentd on kubernetes](https://discuss.elastic.co/t/indices-are-not-showing-for-fluentd-on-kubernetes/227558)
**Author:** [@odirionyeo](https://discuss.elastic.co/u/odirionyeo)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 8:00am UTC](https://discuss.elastic.co/t/indices-are-not-showing-for-fluentd-on-kubernetes/227558 "2020-04-13T08:00:15Z")
I deployed the EFK Stack(Elasticsearch, Fluentd, Kibana) on kuberntes using helm charts from elastic. I am trying to load the indices from kibana yet I don't see anything. when i did curl localhost:9200/\_cat/indices, i … --- ## [How to improve Scroll runtime for 5 billion record retrieval?](https://discuss.elastic.co/t/how-to-improve-scroll-runtime-for-5-billion-record-retrieval/227734)
**Author:** [@Sheraz\_Tariq](https://discuss.elastic.co/u/Sheraz_Tariq)\ **Replies:** 2\ **Last updated:** [April 13, 2020, 7:51am UTC](https://discuss.elastic.co/t/how-to-improve-scroll-runtime-for-5-billion-record-retrieval/227734 "2020-04-13T07:51:56Z")
I want to retrieve 5 billion \_ids from an index in Elasticsearch - the index itself is around 3TB and I'm using the scroll feature to do this. However, I'm getting pretty poor runtime. I'm seeing it take around 11 second… --- ## [Elasticsearch data dependency](https://discuss.elastic.co/t/elasticsearch-data-dependency/227713)
**Author:** [@xodn0812](https://discuss.elastic.co/u/xodn0812)\ **Replies:** 1\ **Last updated:** [April 13, 2020, 7:32am UTC](https://discuss.elastic.co/t/elasticsearch-data-dependency/227713 "2020-04-13T07:32:09Z")
Hello. in the case of capturing the elastic volume as the host volume in the Kubernetes environment, there is a problem that the node does not work properly when the node position changes when all Elasticsearch PODs are… --- ## [Which certificate should I prepare when enabling mutual tls authentication?](https://discuss.elastic.co/t/which-certificate-should-i-prepare-when-enabling-mutual-tls-authentication/227593)
**Author:** [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\ **Replies:** 3\ **Last updated:** [April 13, 2020, 7:29am UTC](https://discuss.elastic.co/t/which-certificate-should-i-prepare-when-enabling-mutual-tls-authentication/227593 "2020-04-13T07:29:28Z")
Hello . I am trying to setup mutual authenticatation by setting xpack.security.http.ssl.client\_authentication to true . However, I am getting below error on elasticsearch (v7.6.2) side . Caused by: javax.net.ssl.… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=956) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=958)