# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=761 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 762 --- ## [Continuously copy specific documents to another index?](https://discuss.elastic.co/t/continuously-copy-specific-documents-to-another-index/255765)
**Author:** [@Thomas\_D1](https://discuss.elastic.co/u/Thomas_D1)\ **Replies:** 8\ **Last updated:** [November 18, 2020, 9:28pm UTC](https://discuss.elastic.co/t/continuously-copy-specific-documents-to-another-index/255765 "2020-11-18T21:28:16Z")
Hi All, we're using Elasticsearch for storing our application logs using daily indexes, which we delete after two weeks. Now we would like to keep all logs with the loglevel "error" for another 2 months. We thought abo… --- ## [Node.processors setting](https://discuss.elastic.co/t/node-processors-setting/255879)
**Author:** [@wazzu62](https://discuss.elastic.co/u/wazzu62)\ **Replies:** 1\ **Last updated:** [November 18, 2020, 9:27pm UTC](https://discuss.elastic.co/t/node-processors-setting/255879 "2020-11-18T21:27:12Z")
I have an ES cluster running 7.4.2. 14 physical servers, 37 ES nodes. Some of the servers run 2 ES nodes, some run 3 ES nodes. Would like to use the node-processor setting. The servers have 48 processors in them. T… --- ## [Watcher to notify if only last snapshot failed](https://discuss.elastic.co/t/watcher-to-notify-if-only-last-snapshot-failed/255891)
**Author:** [@mwise29](https://discuss.elastic.co/u/mwise29)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 9:18pm UTC](https://discuss.elastic.co/t/watcher-to-notify-if-only-last-snapshot-failed/255891 "2020-11-18T21:18:16Z")
I'm new to ElasticSearch and trying to setup a monitoring watch that would email me if the status of the last snapshot has failed. I've seen examples in the forums where they looked at the output for a /\_snapshot/repo/\_… --- ## [Date aggregation query based out of a range](https://discuss.elastic.co/t/date-aggregation-query-based-out-of-a-range/255895)
**Author:** [@Alok\_Ranjan](https://discuss.elastic.co/u/Alok_Ranjan)\ **Replies:** 0\ **Last updated:** [November 18, 2020, 8:15pm UTC](https://discuss.elastic.co/t/date-aggregation-query-based-out-of-a-range/255895 "2020-11-18T20:15:13Z")
Hi All, I have a simple requirement to get the aggregated result as per the posted date like documents posted any day, today, less than 7 days, less than 30 days, or greater than 30 days. I wrote a simple date range qu… --- ## [Java High Level Client Nodes Info API](https://discuss.elastic.co/t/java-high-level-client-nodes-info-api/255885)
**Author:** [@tberne](https://discuss.elastic.co/u/tberne)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 7:43pm UTC](https://discuss.elastic.co/t/java-high-level-client-nodes-info-api/255885 "2020-11-18T19:43:57Z")
Hi! I'm in a process of migration between Java Transport Client and Java High Level Client. Transport client had a nodesInfo API (we used client.admin().cluster().prepareNodesInfo().execute().get().getNodes() to be mor… --- ## [ElasticSearch Optimizing questions](https://discuss.elastic.co/t/elasticsearch-optimizing-questions/253233)
**Author:** [@Joran](https://discuss.elastic.co/u/Joran)\ **Replies:** 4\ **Last updated:** [November 18, 2020, 6:30pm UTC](https://discuss.elastic.co/t/elasticsearch-optimizing-questions/253233 "2020-11-18T18:30:13Z")
Just a heads-up, this is the first time I post something on this forum. I'm a data scientist working at a start-up and we use elasticsearch to store, and query our data. We have used ES with much success over the last y… --- ## [Invalid index name \[\_all\], must not start with '\_'\]](https://discuss.elastic.co/t/invalid-index-name-all-must-not-start-with/255808)
**Author:** [@lankin1](https://discuss.elastic.co/u/lankin1)\ **Replies:** 1\ **Last updated:** [November 18, 2020, 4:35pm UTC](https://discuss.elastic.co/t/invalid-index-name-all-must-not-start-with/255808 "2020-11-18T16:35:00Z")
Hi all, When I try to delete all documents by id: POST /\_all/\_delete\_by\_query {"query": {"match": {"id":"xxxxxxxx"} }} I get the following error: 400 Bad Request: \[{"error":"RemoteTransportException\[\[Launch\_001\]\[inet… --- ## [ElasticSearch: RestHighLevelClient is getting timeout exception if client API is idle for some time](https://discuss.elastic.co/t/elasticsearch-resthighlevelclient-is-getting-timeout-exception-if-client-api-is-idle-for-some-time/255442)
**Author:** [@john\_jayaraj](https://discuss.elastic.co/u/john_jayaraj)\ **Replies:** 7\ **Last updated:** [November 18, 2020, 3:32pm UTC](https://discuss.elastic.co/t/elasticsearch-resthighlevelclient-is-getting-timeout-exception-if-client-api-is-idle-for-some-time/255442 "2020-11-18T15:32:45Z")
I have an REST API with spring boot and RestHighLevelClient to retrieve data from ELasticSearch(AWS-ElasticSearch) and this client API is deployed in OpenShift Container. normally this client API will respond back with… --- ## [Best ways to get into the code of elasticsearch?](https://discuss.elastic.co/t/best-ways-to-get-into-the-code-of-elasticsearch/255844)
**Author:** [@qohelet](https://discuss.elastic.co/u/qohelet)\ **Replies:** 1\ **Last updated:** [November 18, 2020, 3:31pm UTC](https://discuss.elastic.co/t/best-ways-to-get-into-the-code-of-elasticsearch/255844 "2020-11-18T15:31:55Z")
I'm a FOSS-enthusiast and started to fiddle around with elasticsearch. I like the implementation and the possibilities on how to set up the queries. Despite developing Java for quite a while I haven't worked with Gradl… --- ## [ELasticsearch - ILM Policy](https://discuss.elastic.co/t/elasticsearch-ilm-policy/255856)
**Author:** [@shadu88](https://discuss.elastic.co/u/shadu88)\ **Replies:** 1\ **Last updated:** [November 18, 2020, 2:58pm UTC](https://discuss.elastic.co/t/elasticsearch-ilm-policy/255856 "2020-11-18T14:58:47Z")
Hi There, I have a scenario to implement in Elastic cluster. I have 12 node elastic cluster 2 - coordinating Nodes 4 - Hot Data Nodes 3 - Warm data Nodes 3 - Master Nodes Im trying to implement ILM policy based on … --- ## [Machine Learning - Index migration in progress](https://discuss.elastic.co/t/machine-learning-index-migration-in-progress/255837)
**Author:** [@Jorge7](https://discuss.elastic.co/u/Jorge7)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 2:09pm UTC](https://discuss.elastic.co/t/machine-learning-index-migration-in-progress/255837 "2020-11-18T14:09:16Z")
Hi all, I upgraded my elk license to Trial in order to work with the Machine Learning feature. But since 2 days ago, it only appears this message: "Index migration in progress Indices related to Machine Learning are c… --- ## [What is the username and password to be entered](https://discuss.elastic.co/t/what-is-the-username-and-password-to-be-entered/255825)
**Author:** [@Lokesh\_Mayan](https://discuss.elastic.co/u/Lokesh_Mayan)\ **Replies:** 0\ **Last updated:** [November 18, 2020, 11:09am UTC](https://discuss.elastic.co/t/what-is-the-username-and-password-to-be-entered/255825 "2020-11-18T11:09:00Z")
xpack.notification.email.account: gmail\_account: profile: gmail smtp: host: smtp.gmail.com port: 587 user: \*\*\*\*\* password: \*\*\*\*\* auth: true #starttls.required: true in the above , what is the username and passw… --- ## [Unable to setup elastic cluster](https://discuss.elastic.co/t/unable-to-setup-elastic-cluster/255794)
**Author:** [@praveencdcx](https://discuss.elastic.co/u/praveencdcx)\ **Replies:** 1\ **Last updated:** [November 18, 2020, 10:44am UTC](https://discuss.elastic.co/t/unable-to-setup-elastic-cluster/255794 "2020-11-18T10:44:58Z")
Hello, Need help !! I am trying to setup a 3 Master,Data nodes and 1 ingest node. I have done the below configuration. elasticsearch service is starting but i dont see any logs in elasticsearch.log file. \[INFO \]\[o.e.… --- ## [Mapping](https://discuss.elastic.co/t/mapping/255689)
**Author:** [@Elitlogik](https://discuss.elastic.co/u/Elitlogik)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 9:36am UTC](https://discuss.elastic.co/t/mapping/255689 "2020-11-18T09:36:58Z")
Logstash creates datestamped indexes, syslog-2020.11.17 etc, so I have an index for each day. I use kv filter to parse data. The syslog source sends different parameters depending on event type, so the index pattern Fi… --- ## [Can't start Elasticsearch service (code=exited, status=1/FAILURE) affter add x.pack security](https://discuss.elastic.co/t/cant-start-elasticsearch-service-code-exited-status-1-failure-affter-add-x-pack-security/254346)
**Author:** [@aloalo2242](https://discuss.elastic.co/u/aloalo2242)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 8:34am UTC](https://discuss.elastic.co/t/cant-start-elasticsearch-service-code-exited-status-1-failure-affter-add-x-pack-security/254346 "2020-11-18T08:34:26Z")
Hi, I try to add Enable Elasticsearch security features by add this line in elasticsearch.yml: xpack.security.enabled: true and this is full conf: # ======================== Elasticsearch Configuration ===========… --- ## [Cannot insert data at random times. \[OptionalDataException\]](https://discuss.elastic.co/t/cannot-insert-data-at-random-times-optionaldataexception/255476)
**Author:** [@nopma](https://discuss.elastic.co/u/nopma)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 8:27am UTC](https://discuss.elastic.co/t/cannot-insert-data-at-random-times-optionaldataexception/255476 "2020-11-18T08:27:47Z")
I have 3 data nodes. I use fluentbit to send logs to elastic to only one node. Random times I see that it's failed to send data to elastic. It takes 1h exactly to resolve the issue. Its happen one per day. Some logs f… --- ## [Index 350MB of file content using ingest attachment pipiline Elastic.Net throws System.OutOfMemory Exception](https://discuss.elastic.co/t/index-350mb-of-file-content-using-ingest-attachment-pipiline-elastic-net-throws-system-outofmemory-exception/255620)
**Author:** [@kiet.tran](https://discuss.elastic.co/u/kiet.tran)\ **Replies:** 3\ **Last updated:** [November 18, 2020, 6:26am UTC](https://discuss.elastic.co/t/index-350mb-of-file-content-using-ingest-attachment-pipiline-elastic-net-throws-system-outofmemory-exception/255620 "2020-11-18T06:26:34Z")
Hi There, hope you all well. Our setup: our program on dotnet full framework using NEST 7.9 and elastic search 7.9. Changed elasticsearch.yml http.max\_content\_length: 1000mb and using plugin ingest attachment for parsi… --- ## [Need to search for messages containing angle bracket (\<)](https://discuss.elastic.co/t/need-to-search-for-messages-containing-angle-bracket/255706)
**Author:** [@naveen\_wilson](https://discuss.elastic.co/u/naveen_wilson)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 4:32am UTC](https://discuss.elastic.co/t/need-to-search-for-messages-containing-angle-bracket/255706 "2020-11-18T04:32:44Z")
In Kibana logs, the xml feeds are printing line by line which makes it hard to check the logs. So i would want to search for the log entries having angle bracket to get the xml lines together. I tried with the below que… --- ## [Max requests per Keep-Alive Connection](https://discuss.elastic.co/t/max-requests-per-keep-alive-connection/255586)
**Author:** [@Vish\_M](https://discuss.elastic.co/u/Vish_M)\ **Replies:** 2\ **Last updated:** [November 18, 2020, 3:41am UTC](https://discuss.elastic.co/t/max-requests-per-keep-alive-connection/255586 "2020-11-18T03:41:03Z")
Hello, How can I set the max number of requests that a Keep-Alive Connection can serve before it is closed. I am looking to set the max parameter here. Any pointers in this regard is much appreciated. --- ## [Disable specific roles for a node in 7.X+](https://discuss.elastic.co/t/disable-specific-roles-for-a-node-in-7-x/255779)
**Author:** [@Dennis1](https://discuss.elastic.co/u/Dennis1)\ **Replies:** 0\ **Last updated:** [November 18, 2020, 3:15am UTC](https://discuss.elastic.co/t/disable-specific-roles-for-a-node-in-7-x/255779 "2020-11-18T03:15:51Z")
Hello, I noticed that the manner for enabling specific roles for nodes changed between Elasticsearch versions 6.X and 7.X, and am wondering what is the new way to disable specific roles for nodes? In particular, the doc… --- ## [ElasticSearch master and slaves](https://discuss.elastic.co/t/elasticsearch-master-and-slaves/255735)
**Author:** [@Dzyann](https://discuss.elastic.co/u/Dzyann)\ **Replies:** 6\ **Last updated:** [November 18, 2020, 12:58am UTC](https://discuss.elastic.co/t/elasticsearch-master-and-slaves/255735 "2020-11-18T00:58:13Z")
I don't usually work much with Elastic, I have been reading about Elastic Configuration. It is my understanding that Elastic is designed to run clustered, where several nodes, at least 3, should be master. And there are … --- ## [Difference between Ingest Node and Coordinating Node?](https://discuss.elastic.co/t/difference-between-ingest-node-and-coordinating-node/255753)
**Author:** [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\ **Replies:** 7\ **Last updated:** [November 18, 2020, 12:23am UTC](https://discuss.elastic.co/t/difference-between-ingest-node-and-coordinating-node/255753 "2020-11-18T00:23:18Z")
Do Ingest node apply only to indexing from ingest pipelines? And Coordinating Node to all other kinds of indexing? Also client and Coordinating nodes are same right? --- ## [Upgrade ES 5.6.3 to ES 7.8.0: Slower queries. Trying to explain/find out why](https://discuss.elastic.co/t/upgrade-es-5-6-3-to-es-7-8-0-slower-queries-trying-to-explain-find-out-why/255293)
**Author:** [@pvdp](https://discuss.elastic.co/u/pvdp)\ **Replies:** 2\ **Last updated:** [November 17, 2020, 11:52pm UTC](https://discuss.elastic.co/t/upgrade-es-5-6-3-to-es-7-8-0-slower-queries-trying-to-explain-find-out-why/255293 "2020-11-17T23:52:16Z")
We are upgrading from ES 5.6.3 to ES 7.8.0. As part of this, we are doing performance tests to see if there is an improvement or actual degradation in performance. For this we've setup 2 identical clusters in AWS, each… --- ## [ILM Policy for non writing existing index](https://discuss.elastic.co/t/ilm-policy-for-non-writing-existing-index/255217)
**Author:** [@ksaha](https://discuss.elastic.co/u/ksaha)\ **Replies:** 1\ **Last updated:** [November 17, 2020, 11:20pm UTC](https://discuss.elastic.co/t/ilm-policy-for-non-writing-existing-index/255217 "2020-11-17T23:20:39Z")
Hi, I am exploring ILM in Elasticsearch. I am having one question, can we apply ILM policy to non writing existing index? Thanks. --- ## [Elasticsearch v 7.8.0 issue with disk quota](https://discuss.elastic.co/t/elasticsearch-v-7-8-0-issue-with-disk-quota/255334)
**Author:** [@nipul\_sawant](https://discuss.elastic.co/u/nipul_sawant)\ **Replies:** 2\ **Last updated:** [November 17, 2020, 11:18pm UTC](https://discuss.elastic.co/t/elasticsearch-v-7-8-0-issue-with-disk-quota/255334 "2020-11-17T23:18:27Z")
Elasticsearch v 7.8.0 stop working after hits reached 25GB. Any suggestions ? --- ## [DSL Aggregations: How to get results, even if one of the aggs field is not present?](https://discuss.elastic.co/t/dsl-aggregations-how-to-get-results-even-if-one-of-the-aggs-field-is-not-present/255360)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 1\ **Last updated:** [November 17, 2020, 11:17pm UTC](https://discuss.elastic.co/t/dsl-aggregations-how-to-get-results-even-if-one-of-the-aggs-field-is-not-present/255360 "2020-11-17T23:17:15Z")
We have to do aggregations for host.os.name, host.name, host.ip 99% data has host.os.name 95% has host.name 25% has host.ip How to ensure I get all the results, even if the field is not present? So i'm looking for … --- ## [Compare two fields and achieve a left outer comparison](https://discuss.elastic.co/t/compare-two-fields-and-achieve-a-left-outer-comparison/255377)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 1\ **Last updated:** [November 17, 2020, 11:16pm UTC](https://discuss.elastic.co/t/compare-two-fields-and-achieve-a-left-outer-comparison/255377 "2020-11-17T23:16:11Z")
I've following data { "index" : { "\_index" : "host\_compare", "\_id" : "1" } } { "host\_name" : "myhost2" , "host\_count" : "10", "@timestamp": "2020-11-12T02:00:00Z" } { "index" : { "\_index" : "host\_compare", "\_id" : "3" }… --- ## [org.elasticsearch.index.mapper.MapperParsingException after a pipeline](https://discuss.elastic.co/t/org-elasticsearch-index-mapper-mapperparsingexception-after-a-pipeline/255484)
**Author:** [@samsalvatico](https://discuss.elastic.co/u/samsalvatico)\ **Replies:** 1\ **Last updated:** [November 17, 2020, 11:10pm UTC](https://discuss.elastic.co/t/org-elasticsearch-index-mapper-mapperparsingexception-after-a-pipeline/255484 "2020-11-17T23:10:46Z")
Hi, this is my scenario: I receive messages by Fluentbit through an elastic ingestion pipeline. This is my ingestion pipeline: { "jsonize\_account\_backend":{ "processors":\[ { "gr… --- ## [Cluster crashed: "Server gave HTTP response to HTTPS client"](https://discuss.elastic.co/t/cluster-crashed-server-gave-http-response-to-https-client/255762)
**Author:** [@realscout](https://discuss.elastic.co/u/realscout)\ **Replies:** 2\ **Last updated:** [November 17, 2020, 10:48pm UTC](https://discuss.elastic.co/t/cluster-crashed-server-gave-http-response-to-https-client/255762 "2020-11-17T22:48:13Z")
Today, our ElasticSearch cluster stopped responding. We were not performing any sort of maintenance or making any changes. Any request to the cluster caused it to respond with: "Get "https://X.X.X.X:X/": http: server g… --- ## [Is Storage Size under Index Management includes replicated data storage size as well?](https://discuss.elastic.co/t/is-storage-size-under-index-management-includes-replicated-data-storage-size-as-well/255662)
**Author:** [@KunwarAkanksha](https://discuss.elastic.co/u/KunwarAkanksha)\ **Replies:** 1\ **Last updated:** [November 17, 2020, 10:45pm UTC](https://discuss.elastic.co/t/is-storage-size-under-index-management-includes-replicated-data-storage-size-as-well/255662 "2020-11-17T22:45:57Z")
I am wondering if the storage size under Index Management is calculated for data and replicated data both. Example if my replication Factor is 2 , then do storage size tells the ( data storage + 2 \* replicated data stor… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=760) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=762)