# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=769 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 770 --- ## [Trouble when upgrading ES from 7.3.2 to 7.9.3](https://discuss.elastic.co/t/trouble-when-upgrading-es-from-7-3-2-to-7-9-3/254550)
**Author:** [@yetanotheruser](https://discuss.elastic.co/u/yetanotheruser)\ **Replies:** 3\ **Last updated:** [November 9, 2020, 12:47pm UTC](https://discuss.elastic.co/t/trouble-when-upgrading-es-from-7-3-2-to-7-9-3/254550 "2020-11-09T12:47:54Z")
I'm trying to upgrade an ES cluster from 7.3.2 to 7.9.3. I'm configuring ES using Ansible and make REST API calls using Ansible's get\_url module. After installation/upgrade and start of ES, it tries to make REST API call… --- ## [Csv response from Dev tool search query](https://discuss.elastic.co/t/csv-response-from-dev-tool-search-query/254744)
**Author:** [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 12:30pm UTC](https://discuss.elastic.co/t/csv-response-from-dev-tool-search-query/254744 "2020-11-09T12:30:45Z")
Hi All, Is there a way to get csv response from Get Search call. We are able to get json response, However our requirement is to get response in csv format. --- ## [Index expansion ratio](https://discuss.elastic.co/t/index-expansion-ratio/254746)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [November 9, 2020, 11:52am UTC](https://discuss.elastic.co/t/index-expansion-ratio/254746 "2020-11-09T11:52:02Z")
Can you please share the latest details on index expansion ratio as in the recent years I see there are many improvements (7.x version) to optimize the index storage size? Usually I used to consider 0.8 to 1.4 as index … --- ## [Bucketed aggregation on nested fields](https://discuss.elastic.co/t/bucketed-aggregation-on-nested-fields/254729)
**Author:** [@Matteo\_Grolla](https://discuss.elastic.co/u/Matteo_Grolla)\ **Replies:** 0\ **Last updated:** [November 9, 2020, 10:17am UTC](https://discuss.elastic.co/t/bucketed-aggregation-on-nested-fields/254729 "2020-11-09T10:17:28Z")
Good morning, I need to compute a bucketed aggregation on nested fields where the buckets correspond to the nested objects. Can you tell me if it’s possible at all and in case help me with the query? For clarity I ref… --- ## [Restart Elasticsearch cluster](https://discuss.elastic.co/t/restart-elasticsearch-cluster/254026)
**Author:** [@WLM](https://discuss.elastic.co/u/WLM)\ **Replies:** 8\ **Last updated:** [November 9, 2020, 9:54am UTC](https://discuss.elastic.co/t/restart-elasticsearch-cluster/254026 "2020-11-09T09:54:24Z")
Hello I have 6 elasticsearch instanse are runnig in docker containers on 3 physical node. When one of the elasticsearch instanse is restrating, all of indexs become inaccessible. How i can fix it? Config of elasticsearc… --- ## [400 Bad Request when trying to reindex Magento 2](https://discuss.elastic.co/t/400-bad-request-when-trying-to-reindex-magento-2/254688)
**Author:** [@StruggleSnuggl3](https://discuss.elastic.co/u/StruggleSnuggl3)\ **Replies:** 2\ **Last updated:** [November 9, 2020, 9:54am UTC](https://discuss.elastic.co/t/400-bad-request-when-trying-to-reindex-magento-2/254688 "2020-11-09T09:54:07Z")
I have a Magento 2 store running with Elasticsearch 7.x and Litemage. I’m getting the following error when trying to run an index job on Magento 2: Design Config Grid index has been rebuilt successfully in 00:00:00 Cus… --- ## [Elasticsearch memory is not completely utilized](https://discuss.elastic.co/t/elasticsearch-memory-is-not-completely-utilized/253380)
**Author:** [@Varun\_S](https://discuss.elastic.co/u/Varun_S)\ **Replies:** 5\ **Last updated:** [November 9, 2020, 8:42am UTC](https://discuss.elastic.co/t/elasticsearch-memory-is-not-completely-utilized/253380 "2020-11-09T08:42:24Z")
Hi Team, I have deployed elasticsearch in k8s environment with below configuration. Es-client:- replicas: 1 resources: limits: cpu: 2 memory: "4Gi" requests: cpu: "500m" memory: "2Gi" es\_java\_opts: "-Xms2g -X… --- ## [Index filebeat-7.8.1 gets created automatically](https://discuss.elastic.co/t/index-filebeat-7-8-1-gets-created-automatically/254227)
**Author:** [@Anirbaan\_Chowdhury](https://discuss.elastic.co/u/Anirbaan_Chowdhury)\ **Replies:** 4\ **Last updated:** [November 9, 2020, 8:34am UTC](https://discuss.elastic.co/t/index-filebeat-7-8-1-gets-created-automatically/254227 "2020-11-09T08:34:34Z")
Hi Experts, I am sending logs from various instances to my ES cluster using filebeat. I can see that everytime I start sending data, a new index "filebeat-7.8.1" gets created which does not comply with the index templa… --- ## [Errors io.netty.handler.codec.DecoderException: java.lang.IllegalArgumentException: newPosition \> limit](https://discuss.elastic.co/t/errors-io-netty-handler-codec-decoderexception-java-lang-illegalargumentexception-newposition-limit/254711)
**Author:** [@sreekanth](https://discuss.elastic.co/u/sreekanth)\ **Replies:** 3\ **Last updated:** [November 9, 2020, 7:33am UTC](https://discuss.elastic.co/t/errors-io-netty-handler-codec-decoderexception-java-lang-illegalargumentexception-newposition-limit/254711 "2020-11-09T07:33:45Z")
ES Version: 6.8.6 Lucene Version: 7.7.2 We have a 3 nodes cluster where all 3 nodes are marked as "mdi". When we loop on /\_cat/nodes, we saw one of the node randomly go out of cluster and entire cluster is in unstable… --- ## [Node does not cluster setting \[cluster.routing.allocation.include\] filters](https://discuss.elastic.co/t/node-does-not-cluster-setting-cluster-routing-allocation-include-filters/254671)
**Author:** [@ludaca](https://discuss.elastic.co/u/ludaca)\ **Replies:** 3\ **Last updated:** [November 9, 2020, 7:31am UTC](https://discuss.elastic.co/t/node-does-not-cluster-setting-cluster-routing-allocation-include-filters/254671 "2020-11-09T07:31:42Z")
Hello, ES version 5.6.10 16 data nodes 3 master nodes Can't perform any shard's relocation: Error while starting relocation { "error": { "root\_cause": \[ { "type": "remote\_transport\_exception", … --- ## [How do I process an array with enrich processors](https://discuss.elastic.co/t/how-do-i-process-an-array-with-enrich-processors/254713)
**Author:** [@yhseo](https://discuss.elastic.co/u/yhseo)\ **Replies:** 0\ **Last updated:** [November 9, 2020, 7:03am UTC](https://discuss.elastic.co/t/how-do-i-process-an-array-with-enrich-processors/254713 "2020-11-09T07:03:50Z")
The current work is to look at the tags of the original text whenever the original text comes in, and when a matching word comes in, we are trying to apply the classification system of the above source index. I am trying… --- ## ["type":"parse\_exception","reason":"request body is required"},"status":400}](https://discuss.elastic.co/t/type-parse-exception-reason-request-body-is-required-status-400/254664)
**Author:** [@PADMAVATHI\_GUNTURU](https://discuss.elastic.co/u/PADMAVATHI_GUNTURU)\ **Replies:** 4\ **Last updated:** [November 9, 2020, 7:01am UTC](https://discuss.elastic.co/t/type-parse-exception-reason-request-body-is-required-status-400/254664 "2020-11-09T07:01:18Z")
When using the \_bulk API, curl command, I'm seeing this error. I have given 3 JSON files. 2 files were worked, one file is thrown this error. I don't understand the reason here. The 3rd file input data is like the below … --- ## [When ECSLayout is default for Elasticsearch and Kibana log configuration?](https://discuss.elastic.co/t/when-ecslayout-is-default-for-elasticsearch-and-kibana-log-configuration/254710)
**Author:** [@Bingu\_Shim](https://discuss.elastic.co/u/Bingu_Shim)\ **Replies:** 0\ **Last updated:** [November 9, 2020, 6:40am UTC](https://discuss.elastic.co/t/when-ecslayout-is-default-for-elasticsearch-and-kibana-log-configuration/254710 "2020-11-09T06:40:05Z")
Hi, I'm trying to change the log format of Elasticsearch and Kibana to ECS format The Current Document guided using ESJsonLayout, which is different fro ECS format. As far as I know ECS is the standard format for Elas… --- ## [Filebeat and Data stream](https://discuss.elastic.co/t/filebeat-and-data-stream/253841)
**Author:** [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 5:10am UTC](https://discuss.elastic.co/t/filebeat-and-data-stream/253841 "2020-11-09T05:10:13Z")
Hello, What would be the best way to get the Filebeat legacy template converted into a data stream template or some other ILM? It seems like data streams are recommended based on this guide - https://www.elastic.co/guid… --- ## [Scripted Field Unique Count of Sum Value](https://discuss.elastic.co/t/scripted-field-unique-count-of-sum-value/254215)
**Author:** [@111207](https://discuss.elastic.co/u/111207)\ **Replies:** 8\ **Last updated:** [November 9, 2020, 3:19am UTC](https://discuss.elastic.co/t/scripted-field-unique-count-of-sum-value/254215 "2020-11-09T03:19:48Z")
I need to create visualization metric for count of ID of value is 0 I have imported data to elasticsearch And the data in each ID has multiple lines, I would like to sum value by ID. Then Unique Count is a number whose … --- ## [\_default\_ mapping of template is not compatible with index sorting](https://discuss.elastic.co/t/default-mapping-of-template-is-not-compatible-with-index-sorting/251735)
**Author:** [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 3:11am UTC](https://discuss.elastic.co/t/default-mapping-of-template-is-not-compatible-with-index-sorting/251735 "2020-11-09T03:11:31Z")
version: 6.8.0 we create a \_default\_ mapping of index template with the index sorting settings. when we put data to create a new index of this template, then it will throw "unknown index sort field:\[${index sort settin… --- ## [A new support profile search for trace API](https://discuss.elastic.co/t/a-new-support-profile-search-for-trace-api/252467)
**Author:** [@hackerwin7](https://discuss.elastic.co/u/hackerwin7)\ **Replies:** 2\ **Last updated:** [November 9, 2020, 3:10am UTC](https://discuss.elastic.co/t/a-new-support-profile-search-for-trace-api/252467 "2020-11-09T03:10:29Z")
Hi，teams Can we create a new API for trace for search profile. as we all know the profile API can list the shard request execution time and phase. With the only help of profile API, some cases we can not know the true … --- ## [Snapshot stored in GCS bucket will be encrypted](https://discuss.elastic.co/t/snapshot-stored-in-gcs-bucket-will-be-encrypted/254700)
**Author:** [@knagasri](https://discuss.elastic.co/u/knagasri)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 2:49am UTC](https://discuss.elastic.co/t/snapshot-stored-in-gcs-bucket-will-be-encrypted/254700 "2020-11-09T02:49:58Z")
If the elasticsearch data is backed up to gcs bucket. Is data will be in encrypted form. If not how to achieve this? --- ## [Relationship in elasticsearch](https://discuss.elastic.co/t/relationship-in-elasticsearch/254358)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 2\ **Last updated:** [November 9, 2020, 2:04am UTC](https://discuss.elastic.co/t/relationship-in-elasticsearch/254358 "2020-11-09T02:04:01Z")
I have two tables in mysql and i want to join both tables and extract data in elk stack. I have join both tables and create view and import in elk stack but the thing is that data is duplicate . Is that possible through… --- ## [Zeek with elasticsearch](https://discuss.elastic.co/t/zeek-with-elasticsearch/254231)
**Author:** [@shabi037](https://discuss.elastic.co/u/shabi037)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 1:45am UTC](https://discuss.elastic.co/t/zeek-with-elasticsearch/254231 "2020-11-09T01:45:49Z")
Hi all, I am running Zeek (Bro) in an offline mode to analyze the pcap file. I get the logs in the "/opt/zeek/logs/current" directory. To further analyze the received logs I am passing those to Filebeat to be visualized… --- ## [Elasticsearch upgrade from 6.8.0 to 7.9.2](https://discuss.elastic.co/t/elasticsearch-upgrade-from-6-8-0-to-7-9-2/254223)
**Author:** [@maadhav](https://discuss.elastic.co/u/maadhav)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 1:43am UTC](https://discuss.elastic.co/t/elasticsearch-upgrade-from-6-8-0-to-7-9-2/254223 "2020-11-09T01:43:28Z")
Hi Team We have around 60 nodes docker based production cluster running with version 6.8. We need to upgrade this to version 7.9.2. We have used discovery settings such as discovery.zen.ping.unicast.hosts , discovery… --- ## [Rollup experimental](https://discuss.elastic.co/t/rollup-experimental/254364)
**Author:** [@schophil](https://discuss.elastic.co/u/schophil)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 1:36am UTC](https://discuss.elastic.co/t/rollup-experimental/254364 "2020-11-09T01:36:21Z")
Hi, I am looking into using the rollup feature on my project. It seems very interesting and the early tests I performed are positive. However I am conflicted about the "experimental feature". I woud like to use this in… --- ## [Range query in doc lvl security](https://discuss.elastic.co/t/range-query-in-doc-lvl-security/254431)
**Author:** [@pbabik](https://discuss.elastic.co/u/pbabik)\ **Replies:** 1\ **Last updated:** [November 9, 2020, 1:33am UTC](https://discuss.elastic.co/t/range-query-in-doc-lvl-security/254431 "2020-11-09T01:33:14Z")
Hi, I am trying to set policy/rule for a user role in elasticsearch, to allow them to search 2 weeks old data only. One solution would be rotating aliases, and set permissions for those, but it isn't ideal. Better solut… --- ## [Does watcher action webhook involve retrying?](https://discuss.elastic.co/t/does-watcher-action-webhook-involve-retrying/254686)
**Author:** [@goodmirek5849](https://discuss.elastic.co/u/goodmirek5849)\ **Replies:** 0\ **Last updated:** [November 8, 2020, 4:05pm UTC](https://discuss.elastic.co/t/does-watcher-action-webhook-involve-retrying/254686 "2020-11-08T16:05:05Z")
We use Elasticsearch in ElasticCloud and use Watcher. When the Watcher raises an alert, it invokes a configured webhook action. It works most of the time. I need to know whether the Watcher retries the webhook action, … --- ## [Getting all documents in Elastic Search Shard](https://discuss.elastic.co/t/getting-all-documents-in-elastic-search-shard/254690)
**Author:** [@asagarwal10](https://discuss.elastic.co/u/asagarwal10)\ **Replies:** 0\ **Last updated:** [November 8, 2020, 7:41pm UTC](https://discuss.elastic.co/t/getting-all-documents-in-elastic-search-shard/254690 "2020-11-08T19:41:31Z")
I am trying to get all documents in a shard. When creating the index, I used the query: PUT mortgages-stage { "mappings": { "\_routing": { "required": true } } } When adding documents to the index, I … --- ## [How do the "query" and "suggest" blocks relate in a suggestion request?](https://discuss.elastic.co/t/how-do-the-query-and-suggest-blocks-relate-in-a-suggestion-request/254685)
**Author:** [@Timothy\_Wall](https://discuss.elastic.co/u/Timothy_Wall)\ **Replies:** 2\ **Last updated:** [November 8, 2020, 7:09pm UTC](https://discuss.elastic.co/t/how-do-the-query-and-suggest-blocks-relate-in-a-suggestion-request/254685 "2020-11-08T19:09:50Z")
If I'm doing a suggestion request like below, how do the "query" and "suggest" blocks relate to one another? POST /\_search { "query": {...}, "suggest": {...}, } --- ## [Elasticsearch RestHighLevelClient behind a corporate firewall via a proxy](https://discuss.elastic.co/t/elasticsearch-resthighlevelclient-behind-a-corporate-firewall-via-a-proxy/254642)
**Author:** [@Ilya\_I](https://discuss.elastic.co/u/Ilya_I)\ **Replies:** 1\ **Last updated:** [November 8, 2020, 6:29pm UTC](https://discuss.elastic.co/t/elasticsearch-resthighlevelclient-behind-a-corporate-firewall-via-a-proxy/254642 "2020-11-08T18:29:41Z")
I am trying to access cloud Elasticsearch installation from our network that requires using a proxy for external requests. This is the snippet of code I use to pass Elasticsearch credentials and our proxy settings: Cred… --- ## [Snapshot retention question](https://discuss.elastic.co/t/snapshot-retention-question/254677)
**Author:** [@mwise29](https://discuss.elastic.co/u/mwise29)\ **Replies:** 9\ **Last updated:** [November 8, 2020, 4:40pm UTC](https://discuss.elastic.co/t/snapshot-retention-question/254677 "2020-11-08T16:40:58Z")
I'm new to ElasticSearch administration. From what I've gathered so far for snapshots is they are always incremental. My question is regarding snapshot retention. The documentation mentions that the snapshots can be c… --- ## [One word fields (keyword vs text types)](https://discuss.elastic.co/t/one-word-fields-keyword-vs-text-types/252408)
**Author:** [@aviadhaham](https://discuss.elastic.co/u/aviadhaham)\ **Replies:** 9\ **Last updated:** [November 8, 2020, 4:23pm UTC](https://discuss.elastic.co/t/one-word-fields-keyword-vs-text-types/252408 "2020-11-08T16:23:36Z")
If, for example, I have a field in which I know that the data ingested to it is only one-word strings. Why would I still use type "text" and not type "keyword" for this field's mapping? Setting it to be of type "keyword… --- ## [Aggregate "array" length's across documents](https://discuss.elastic.co/t/aggregate-array-lengths-across-documents/254646)
**Author:** [@Courtan](https://discuss.elastic.co/u/Courtan)\ **Replies:** 1\ **Last updated:** [November 8, 2020, 2:53pm UTC](https://discuss.elastic.co/t/aggregate-array-lengths-across-documents/254646 "2020-11-08T14:53:55Z")
I am trying to count Array Elements across all documents and sum them up, but I keep getting this error: "No field found for \[offers\] in mapping with types \[ \]" Mappings { "mappings": { "properties": { "of… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=768) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=770)