# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=231 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 232 --- ## [Overwriting a whole index without downtime best practices](https://discuss.elastic.co/t/overwriting-a-whole-index-without-downtime-best-practices/338039)
**Author:** [@krezno](https://discuss.elastic.co/u/krezno)\ **Replies:** 1\ **Last updated:** [July 10, 2023, 9:15pm UTC](https://discuss.elastic.co/t/overwriting-a-whole-index-without-downtime-best-practices/338039 "2023-07-10T21:15:46Z")
I have several batch pipelines that produce a new version of the result each time. The data can't have any downtime so I can't just delete the index before writing. The current solution is to create a new index with the … --- ## [UNASSIGNED state after REPLICA\_ADDED](https://discuss.elastic.co/t/unassigned-state-after-replica-added/336326)
**Author:** [@Itay\_Bittan](https://discuss.elastic.co/u/Itay_Bittan)\ **Replies:** 24\ **Last updated:** [July 10, 2023, 8:31pm UTC](https://discuss.elastic.co/t/unassigned-state-after-replica-added/336326 "2023-07-10T20:31:54Z")
We are indexing around 7TB on a daily basis. All the indices are being replaced once a day with a new ones (fresh data). Each index represent one customer (business). The variety of indices is big, from a few kilobyte… --- ## [{Invalid NEST response built from a successful (200) low level call on POST: /\_bulk}](https://discuss.elastic.co/t/invalid-nest-response-built-from-a-successful-200-low-level-call-on-post-bulk/338042)
**Author:** [@Janderson\_Goncalves](https://discuss.elastic.co/u/Janderson_Goncalves)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 7:46pm UTC](https://discuss.elastic.co/t/invalid-nest-response-built-from-a-successful-200-low-level-call-on-post-bulk/338042 "2023-07-10T19:46:00Z")
I am trying to create a Middleware that captures the Request and Response of a given application and indexes it in Elastic. But the elastic always returns the following error and already when debugging is going an object… --- ## [Search\_phase\_execution\_exception: \[no\_shard\_available\_action\_exception\] Reason: null; \[no\_shard\_available\_action\_exception\] Reason: null (500)](https://discuss.elastic.co/t/search-phase-execution-exception-no-shard-available-action-exception-reason-null-no-shard-available-action-exception-reason-null-500/338003)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 2\ **Last updated:** [July 10, 2023, 5:28pm UTC](https://discuss.elastic.co/t/search-phase-execution-exception-no-shard-available-action-exception-reason-null-no-shard-available-action-exception-reason-null-500/338003 "2023-07-10T17:28:56Z")
Hi after i've receive disk full i try to remove some indices from this path: /opt/elasticsearch/var/lib/elasticsearch/indices/ after that APM dashboard not load and give below errors. seems some indices that related t… --- ## [In Dev Tools Console, \_reindex gets error of Client request timeout](https://discuss.elastic.co/t/in-dev-tools-console-reindex-gets-error-of-client-request-timeout/337920)
**Author:** [@Mike\_Z](https://discuss.elastic.co/u/Mike_Z)\ **Replies:** 4\ **Last updated:** [July 10, 2023, 4:38pm UTC](https://discuss.elastic.co/t/in-dev-tools-console-reindex-gets-error-of-client-request-timeout/337920 "2023-07-10T16:38:04Z")
When trying to duplicate an index by the \_reindex command, it always gets an error saying "502, Bad Gateway" and "Client request timeout". However, the command GET \_cat/indices/\_all shows, right after the above error, t… --- ## [When Search goes to Replica shard?](https://discuss.elastic.co/t/when-search-goes-to-replica-shard/338021)
**Author:** [@siddhartha\_c](https://discuss.elastic.co/u/siddhartha_c)\ **Replies:** 4\ **Last updated:** [July 10, 2023, 2:52pm UTC](https://discuss.elastic.co/t/when-search-goes-to-replica-shard/338021 "2023-07-10T14:52:01Z")
I have a scenario , wherein I would need to perform searches in Elastic , but the number of "concurrent searches" are very less. In this case , can I assume that the searches will go to Primary shards? My understanding … --- ## [Recognition of similar words in a search?](https://discuss.elastic.co/t/recognition-of-similar-words-in-a-search/337311)
**Author:** [@Paul-III](https://discuss.elastic.co/u/Paul-III)\ **Replies:** 1\ **Last updated:** [July 10, 2023, 2:32pm UTC](https://discuss.elastic.co/t/recognition-of-similar-words-in-a-search/337311 "2023-07-10T14:32:46Z")
Is Elastic Search able to recognize that search for housedoor is the same as search for house door and so delivering results for both? --- ## [Reindexed Documents are not showing up, Response says it has created but it does not show up in destination index](https://discuss.elastic.co/t/reindexed-documents-are-not-showing-up-response-says-it-has-created-but-it-does-not-show-up-in-destination-index/337987)
**Author:** [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Replies:** 9\ **Last updated:** [July 10, 2023, 2:15pm UTC](https://discuss.elastic.co/t/reindexed-documents-are-not-showing-up-response-says-it-has-created-but-it-does-not-show-up-in-destination-index/337987 "2023-07-10T14:15:13Z")
http://localhost:9201/restored\_index/\_search Response: { "took": 2, "timed\_out": false, "\_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits":… --- ## [Creating Alerts for rollup jobs](https://discuss.elastic.co/t/creating-alerts-for-rollup-jobs/338030)
**Author:** [@akhil\_reddy](https://discuss.elastic.co/u/akhil_reddy)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 1:47pm UTC](https://discuss.elastic.co/t/creating-alerts-for-rollup-jobs/338030 "2023-07-10T13:47:16Z")
Hi, Is it possible to get alerts for rollup jobs. If I schedule my rollup job to trigger for every hour, can I get alerts for the stats in rollup jobs. Ex: an alert for number of documents processed and number of rollup… --- ## [ElasticSearch cluster down due to high memory usage](https://discuss.elastic.co/t/elasticsearch-cluster-down-due-to-high-memory-usage/337975)
**Author:** [@maulik\_trapasiya](https://discuss.elastic.co/u/maulik_trapasiya)\ **Replies:** 1\ **Last updated:** [July 10, 2023, 12:32pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-down-due-to-high-memory-usage/337975 "2023-07-10T12:32:26Z")
I have ran some queries on ES, which fetched huge amount of data and due to that Memory utilization reached high and ES cluster went down. Below is the error that ES-java client has thrown {"error":{"root\_cause":\[{"typ… --- ## [Increase the number of shards in the cluster](https://discuss.elastic.co/t/increase-the-number-of-shards-in-the-cluster/337916)
**Author:** [@karlanakamura](https://discuss.elastic.co/u/karlanakamura)\ **Replies:** 4\ **Last updated:** [July 10, 2023, 12:04pm UTC](https://discuss.elastic.co/t/increase-the-number-of-shards-in-the-cluster/337916 "2023-07-10T12:04:54Z")
Hello, I'm using version 8.6.0 of elastic cloud. I tried to create an index but got this message: Validation Failed: 1: this action would add \[2\] shards, but this cluster currently has \[2000\]/\[2000\] maximum normal sh… --- ## [What is the function of CPU and Memory for elastic](https://discuss.elastic.co/t/what-is-the-function-of-cpu-and-memory-for-elastic/338017)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 11:19am UTC](https://discuss.elastic.co/t/what-is-the-function-of-cpu-and-memory-for-elastic/338017 "2023-07-10T11:19:10Z")
Hi there, just want to confirm, as far as i know. memory is used by elastic for JVM, shard. is it correct? or anything else? and for cpu, what actually elastic does with cpu other than to run the service? your explana… --- ## [.net ElasticsearchClient internal json serializer](https://discuss.elastic.co/t/net-elasticsearchclient-internal-json-serializer/337219)
**Author:** [@Jere](https://discuss.elastic.co/u/Jere)\ **Replies:** 1\ **Last updated:** [July 10, 2023, 10:54am UTC](https://discuss.elastic.co/t/net-elasticsearchclient-internal-json-serializer/337219 "2023-07-10T10:54:37Z")
Hi, using the .net ElasticsearchClient is it possible to configure the DefaultRequestResponseSerializer to prevent the following exception? Elastic.Transport.UnexpectedTransportException Message=The maximum configur… --- ## [The es service on all nodes stops unexpectedly](https://discuss.elastic.co/t/the-es-service-on-all-nodes-stops-unexpectedly/337979)
**Author:** [@alanzc](https://discuss.elastic.co/u/alanzc)\ **Replies:** 1\ **Last updated:** [July 10, 2023, 10:41am UTC](https://discuss.elastic.co/t/the-es-service-on-all-nodes-stops-unexpectedly/337979 "2023-07-10T10:41:58Z")
When I update openjdk from 1.8-u312 to 1.8-u372, then 12 hours later I got this error from all nodes. Does anyone know the reason? \[2023-07-07T08:47:54,794\]\[ERROR\]\[o.e.b.ElasticsearchUncaughtExceptionHandler\] \[rcvaes01\]… --- ## [Splitting Using Runtime Field / Scripting Field](https://discuss.elastic.co/t/splitting-using-runtime-field-scripting-field/336468)
**Author:** [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\ **Replies:** 2\ **Last updated:** [July 10, 2023, 10:25am UTC](https://discuss.elastic.co/t/splitting-using-runtime-field-scripting-field/336468 "2023-07-10T10:25:51Z")
In one of the alerts, in the field host.ip, I am seeing a bunch of IP addresses. So I want to create a scripted or runtime field where I want to split each IP address and place them in a new field like host.ip1 and host.… --- ## [How to take the backup of 3months data of elasticsearch?](https://discuss.elastic.co/t/how-to-take-the-backup-of-3months-data-of-elasticsearch/337653)
**Author:** [@merson](https://discuss.elastic.co/u/merson)\ **Replies:** 10\ **Last updated:** [July 10, 2023, 9:28am UTC](https://discuss.elastic.co/t/how-to-take-the-backup-of-3months-data-of-elasticsearch/337653 "2023-07-10T09:28:41Z")
In elasticsearch, it is runned three months and the size of elk is 40gb then I want to backup the elasticsearch datas. so what to do the backup of elasticsearch for 3months without using snapshot and restore. --- ## [Which Elasticsearch node should I send my query to?](https://discuss.elastic.co/t/which-elasticsearch-node-should-i-send-my-query-to/337937)
**Author:** [@ANUBHAV\_GUPTA](https://discuss.elastic.co/u/ANUBHAV_GUPTA)\ **Replies:** 3\ **Last updated:** [July 10, 2023, 6:45am UTC](https://discuss.elastic.co/t/which-elasticsearch-node-should-i-send-my-query-to/337937 "2023-07-10T06:45:51Z")
Let's say I have a cluster with total of 10 nodes where 4 are master eligible and rest are data nodes. So, how can I decide to which endpoint I should ping to have the best availability? --- ## [How to provide source Field in \_msearch query in ElasticSearch java client version 8](https://discuss.elastic.co/t/how-to-provide-source-field-in-msearch-query-in-elasticsearch-java-client-version-8/337924)
**Author:** [@maulik\_trapasiya](https://discuss.elastic.co/u/maulik_trapasiya)\ **Replies:** 2\ **Last updated:** [July 10, 2023, 3:19am UTC](https://discuss.elastic.co/t/how-to-provide-source-field-in-msearch-query-in-elasticsearch-java-client-version-8/337924 "2023-07-10T03:19:14Z")
My Elasticsearch's documents are of high size. My service is Java application and its using Elasticsearch java client version 8. Need to run \_msearch query on ES. MultisearchBody don't have field of \_source. in ES native… --- ## [Ingest Pipeline for parsing multiline fields giving provided Grok expressions do not match field value error error](https://discuss.elastic.co/t/ingest-pipeline-for-parsing-multiline-fields-giving-provided-grok-expressions-do-not-match-field-value-error-error/337699)
**Author:** [@SecretAsianMan](https://discuss.elastic.co/u/SecretAsianMan)\ **Replies:** 1\ **Last updated:** [July 9, 2023, 9:40pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-parsing-multiline-fields-giving-provided-grok-expressions-do-not-match-field-value-error-error/337699 "2023-07-09T21:40:24Z")
I am trying to parse a multiline log file as shown below. This is the processor that I have currently configured for the multiline log file. \[ { "grok": { "field": "message", "patterns": \[ "… --- ## [Reindexing an index which had document added by ingest pipeline](https://discuss.elastic.co/t/reindexing-an-index-which-had-document-added-by-ingest-pipeline/337951)
**Author:** [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Replies:** 12\ **Last updated:** [July 9, 2023, 8:02pm UTC](https://discuss.elastic.co/t/reindexing-an-index-which-had-document-added-by-ingest-pipeline/337951 "2023-07-09T20:02:52Z")
I have an index my-idx-09-2022. I made a ingest pipeline so that all the updates from now of my-idx-09-2022 will go to a new index i.e my-idx-new-09-2023. Python code: def create\_write\_redirect\_pipeline(source\_client, … --- ## [Gork regex](https://discuss.elastic.co/t/gork-regex/337623)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 8\ **Last updated:** [July 9, 2023, 6:29pm UTC](https://discuss.elastic.co/t/gork-regex/337623 "2023-07-09T18:29:28Z")
Hi I use this logstash gork: %{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{DATA:id} \[%{DATA}\] %{DATA:jboss\_errors}(?=:|$) here is the log: 2023-06-30 09:09:55,941 ERROR CUS.InEP-AAAA-123194144 \[invocation\] WF… --- ## [How does Allocation of shards happens, when a node leaves cluster?](https://discuss.elastic.co/t/how-does-allocation-of-shards-happens-when-a-node-leaves-cluster/337960)
**Author:** [@Shashank\_Agrawal](https://discuss.elastic.co/u/Shashank_Agrawal)\ **Replies:** 3\ **Last updated:** [July 9, 2023, 5:09pm UTC](https://discuss.elastic.co/t/how-does-allocation-of-shards-happens-when-a-node-leaves-cluster/337960 "2023-07-09T17:09:33Z")
I want to know the exact procedure followed, for the allocation of shards on a node when the node leaves the cluster. Facts I know - 1.) ES waits for sometime before the reassigning the shards. 2.) For primary shards, … --- ## [Elasticsearch service not starting](https://discuss.elastic.co/t/elasticsearch-service-not-starting/337954)
**Author:** [@Jefferson\_Lourthusam](https://discuss.elastic.co/u/Jefferson_Lourthusam)\ **Replies:** 5\ **Last updated:** [July 9, 2023, 3:17pm UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/337954 "2023-07-09T15:17:49Z")
Elasticsearch service not starting , we can see below in Elasticsearch-STG logs low disk watermark \[85%\] exceeded on free: 37.4gb\[14.9%\], replicas will not be assigned to this node --- ## [Ingest pipeline routing documents to appropriate target index requires permissions on target index](https://discuss.elastic.co/t/ingest-pipeline-routing-documents-to-appropriate-target-index-requires-permissions-on-target-index/337923)
**Author:** [@Jurgen\_Wagner\_DVT](https://discuss.elastic.co/u/Jurgen_Wagner_DVT)\ **Replies:** 4\ **Last updated:** [July 8, 2023, 9:20pm UTC](https://discuss.elastic.co/t/ingest-pipeline-routing-documents-to-appropriate-target-index-requires-permissions-on-target-index/337923 "2023-07-08T21:20:46Z")
Suppose you don't trust data-feeding users to place documents into the right index, so you create a virtual index with an ingestion pipeline that determines the proper target index alias based on a few fields in each doc… --- ## [Extract value from path in logstash](https://discuss.elastic.co/t/extract-value-from-path-in-logstash/337936)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [July 8, 2023, 12:20pm UTC](https://discuss.elastic.co/t/extract-value-from-path-in-logstash/337936 "2023-07-08T12:20:18Z")
Hi need to extract value from path in logstash, here is my logpath: /data/app/20230707/\*/\* /data/app1/20230707/host1/\*.log /data/app2/20230707/host2/\*.log need to extract these field from path (FYI: hostname must be … --- ## [Filtering logic in elastic search output](https://discuss.elastic.co/t/filtering-logic-in-elastic-search-output/337922)
**Author:** [@Minika](https://discuss.elastic.co/u/Minika)\ **Replies:** 0\ **Last updated:** [July 8, 2023, 12:29am UTC](https://discuss.elastic.co/t/filtering-logic-in-elastic-search-output/337922 "2023-07-08T00:29:07Z")
Hi, I am trying to apply a filter logic in OCP Logstash pipeline. My pipeline receive logs from filebeat which contain a fields tag named logtype(that states the type of log) My motive is to use the logtype value and sen… --- ## [Mocking Search Results in new Java API](https://discuss.elastic.co/t/mocking-search-results-in-new-java-api/337012)
**Author:** [@silentfilm](https://discuss.elastic.co/u/silentfilm)\ **Replies:** 3\ **Last updated:** [July 7, 2023, 9:30pm UTC](https://discuss.elastic.co/t/mocking-search-results-in-new-java-api/337012 "2023-07-07T21:30:45Z")
Are there any examples of how to mock an Elasticsearch search result for the Java API for unit tests with Mockito? Do you mock the entire search result or individual hits? If I search the Internet for examples I only see… --- ## [Elasticsearch.service: Main process exited, code=killed, status=9/KILL](https://discuss.elastic.co/t/elasticsearch-service-main-process-exited-code-killed-status-9-kill/337796)
**Author:** [@bbkunbi](https://discuss.elastic.co/u/bbkunbi)\ **Replies:** 3\ **Last updated:** [July 7, 2023, 9:23pm UTC](https://discuss.elastic.co/t/elasticsearch-service-main-process-exited-code-killed-status-9-kill/337796 "2023-07-07T21:23:04Z")
Errror: elasticsearch.service: Main process exited, code=killed, status=9/KILL ul 06 17:32:05 linux systemd\[1\]: elasticsearch.service: Main process exited, code=killed, status=9/KILL Jul 06 17:32:05 linux systemd\[1\]: e… --- ## [Add new field to index based on maths calculation from other fields in the same index](https://discuss.elastic.co/t/add-new-field-to-index-based-on-maths-calculation-from-other-fields-in-the-same-index/337571)
**Author:** [@patcan](https://discuss.elastic.co/u/patcan)\ **Replies:** 12\ **Last updated:** [July 7, 2023, 9:15pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-based-on-maths-calculation-from-other-fields-in-the-same-index/337571 "2023-07-07T21:15:07Z")
Hi, I use elastic-agent on EKS with kubernetes integration. One of the field such as kubernetes.volume.fs.used.pct in the index provides incorrect values I was able to get the correct value using the following formula… --- ## [Where to change auto\_expand\_replicas for enrich indices?](https://discuss.elastic.co/t/where-to-change-auto-expand-replicas-for-enrich-indices/337907)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 0\ **Last updated:** [July 7, 2023, 4:51pm UTC](https://discuss.elastic.co/t/where-to-change-auto-expand-replicas-for-enrich-indices/337907 "2023-07-07T16:51:51Z")
Hello, I need to change the auto\_expand\_replicas for the indices created by enrich policies, the .enrich-\* indices, but I could not find any system template with this mapping, so it seems to be hard-coded elsewhere. Cu… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=230) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=232)