# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=638 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 639 --- ## [ES Scroll is returning duplicate/same results](https://discuss.elastic.co/t/es-scroll-is-returning-duplicate-same-results/274327)
**Author:** [@vedantkdesai](https://discuss.elastic.co/u/vedantkdesai)\ **Replies:** 7\ **Last updated:** [May 28, 2021, 12:59pm UTC](https://discuss.elastic.co/t/es-scroll-is-returning-duplicate-same-results/274327 "2021-05-28T12:59:15Z")
Hi Team, I was using the ES scroll mechanism for pagination. I found that we are getting the same/duplicate results in the subsequent scroll request made. Could you please help us here? Thanks in advance! --- ## [I am trying to change the format of date and time using pipeline but doesn't work](https://discuss.elastic.co/t/i-am-trying-to-change-the-format-of-date-and-time-using-pipeline-but-doesnt-work/274329)
**Author:** [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 12:30pm UTC](https://discuss.elastic.co/t/i-am-trying-to-change-the-format-of-date-and-time-using-pipeline-but-doesnt-work/274329 "2021-05-28T12:30:54Z")
Here is the pipeline : PUT \_ingest/pipeline/dateformat { "description" : "To change Date format", "processors" : \[ { "date" : { "field" : "EndDate", "target\_field" : "newEndDate", "formats" : \["date\_time"\] } } … --- ## [Search for Aggregation results](https://discuss.elastic.co/t/search-for-aggregation-results/274030)
**Author:** [@krishnapss](https://discuss.elastic.co/u/krishnapss)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 10:33am UTC](https://discuss.elastic.co/t/search-for-aggregation-results/274030 "2021-05-28T10:33:18Z")
Hi, Could someone help me to achieve this functionality. Aggregation results are 500 to 5000 on different fields (data is in millions). I have used CompositeAggregation for pagination. But for user it is not great t… --- ## [Get a page of products but return all product\_tags for the whole unpaginated result set](https://discuss.elastic.co/t/get-a-page-of-products-but-return-all-product-tags-for-the-whole-unpaginated-result-set/274283)
**Author:** [@kiko](https://discuss.elastic.co/u/kiko)\ **Replies:** 5\ **Last updated:** [May 28, 2021, 10:28am UTC](https://discuss.elastic.co/t/get-a-page-of-products-but-return-all-product-tags-for-the-whole-unpaginated-result-set/274283 "2021-05-28T10:28:19Z")
Hi I have a query, which filters products and paginates the results (by ?size URL argument). Is there a way to return just a page (?size=12) of products, but return the available product\_tags for the whole lot (hits.tot… --- ## [SRE-Ansible To Install ELK Stack](https://discuss.elastic.co/t/sre-ansible-to-install-elk-stack/274086)
**Author:** [@Ashish\_Jindal](https://discuss.elastic.co/u/Ashish_Jindal)\ **Replies:** 5\ **Last updated:** [May 28, 2021, 8:13am UTC](https://discuss.elastic.co/t/sre-ansible-to-install-elk-stack/274086 "2021-05-28T08:13:24Z")
Our task is related to SRE-Ansible To Install ELK Stack i.e. Write an ansible script to install ELK stack and filebeat locally. To do so, create a directory structure as mentioned below ansible\_ELK |\_site.yml |\_… --- ## [Native Authentication](https://discuss.elastic.co/t/native-authentication/274286)
**Author:** [@Ashwin\_Srivastav](https://discuss.elastic.co/u/Ashwin_Srivastav)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 6:32am UTC](https://discuss.elastic.co/t/native-authentication/274286 "2021-05-28T06:32:17Z")
I am new to Elasticsearch. I got the idea that in the Native Authentication realm, the user credentials are hashed and stored in the .security index. So the question which came in my mind is that for a multi-node cluste… --- ## [Elastic Scripting caching issue](https://discuss.elastic.co/t/elastic-scripting-caching-issue/273511)
**Author:** [@Divy\_Garg](https://discuss.elastic.co/u/Divy_Garg)\ **Replies:** 7\ **Last updated:** [May 28, 2021, 6:30am UTC](https://discuss.elastic.co/t/elastic-scripting-caching-issue/273511 "2021-05-28T06:30:04Z")
I am facing caching issue while using scripts. So I am using below script in elasticsearch query. But when it runs multiple times it gives circuit breaker exception( Too many dynamic script compilations within, max: \[7… --- ## ['ElasticPressProvider' was not found in '@10up/elasticpress-react'](https://discuss.elastic.co/t/elasticpressprovider-was-not-found-in-10up-elasticpress-react/274196)
**Author:** [@Sharmeela\_Jaimalani](https://discuss.elastic.co/u/Sharmeela_Jaimalani)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 5:26am UTC](https://discuss.elastic.co/t/elasticpressprovider-was-not-found-in-10up-elasticpress-react/274196 "2021-05-28T05:26:05Z")
Hello, I have installed the elasticpress-react with following npm command in my gastby project - (GitHub - 10up/elasticpress-react: Use ElasticPress with React.) "npm install @10up/elasticpress-react --save" After in… --- ## [Unassigned shards with new frozen tier](https://discuss.elastic.co/t/unassigned-shards-with-new-frozen-tier/274265)
**Author:** [@hilt86](https://discuss.elastic.co/u/hilt86)\ **Replies:** 4\ **Last updated:** [May 27, 2021, 11:37pm UTC](https://discuss.elastic.co/t/unassigned-shards-with-new-frozen-tier/274265 "2021-05-27T23:37:45Z")
I have spun up a brand new cluster on elastic cloud to test 7.13 and I have data\_hot and data\_frozen tiers. I've changed ILM policy for logs and metrics so that data that is older than a few hours (just to test) is move… --- ## [Limiting the scope of permissions for GCS backups](https://discuss.elastic.co/t/limiting-the-scope-of-permissions-for-gcs-backups/274166)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 9:11am UTC](https://discuss.elastic.co/t/limiting-the-scope-of-permissions-for-gcs-backups/274166 "2021-05-27T09:11:55Z")
According to the docs, the storage.Admin role needs to be granted to the GCP service account that will be used for GCS snapshotting. However, (unless I am mistaken) this is a rather broad role giving full access to all … --- ## [Performance implications of multiple primary shards for one index on the same data node](https://discuss.elastic.co/t/performance-implications-of-multiple-primary-shards-for-one-index-on-the-same-data-node/273859)
**Author:** [@dotjim](https://discuss.elastic.co/u/dotjim)\ **Replies:** 3\ **Last updated:** [May 27, 2021, 10:49pm UTC](https://discuss.elastic.co/t/performance-implications-of-multiple-primary-shards-for-one-index-on-the-same-data-node/273859 "2021-05-27T22:49:34Z")
Hi there, I have an Elastic Cloud cluster running v7.10.2 with three data nodes, three dedicated masters and three coordinates nodes. One index is currently 150GB, spanning three primary shards (one per data node) with… --- ## [Code Contribution - cannot import into IntelliJ](https://discuss.elastic.co/t/code-contribution-cannot-import-into-intellij/274187)
**Author:** [@VJvaLbhYbfr](https://discuss.elastic.co/u/VJvaLbhYbfr)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 11:10am UTC](https://discuss.elastic.co/t/code-contribution-cannot-import-into-intellij/274187 "2021-05-27T11:10:52Z")
Hello everyone, I wanted to contribute to Elastic project. I did read CONTRIBUTING.md and followed it both on Ubuntu 20.04 (in VirtualBox) and on Windows 10. Ubuntu 20.04: Importing project into IntelliJ (version 2021… --- ## [Is elastichsearch-oss still maintained?](https://discuss.elastic.co/t/is-elastichsearch-oss-still-maintained/274040)
**Author:** [@confuseduser](https://discuss.elastic.co/u/confuseduser)\ **Replies:** 10\ **Last updated:** [May 27, 2021, 7:09pm UTC](https://discuss.elastic.co/t/is-elastichsearch-oss-still-maintained/274040 "2021-05-27T19:09:54Z")
I am really confused at the moment. I have this apt repo config which pulls from: deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main The last update of elastichsearch-oss seems to be from january 2021 w… --- ## [Snapshot fails on ceph repository](https://discuss.elastic.co/t/snapshot-fails-on-ceph-repository/274197)
**Author:** [@faxm0dem](https://discuss.elastic.co/u/faxm0dem)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 2:25pm UTC](https://discuss.elastic.co/t/snapshot-fails-on-ceph-repository/274197 "2021-05-27T14:25:20Z")
Hi, Using ES 6.8.6 I'm trying to set up a snapshot repository. The type is fs and the underlying filesystem is ceph. The UID/GID of elasticsearch is the same on all nodes. When issuing the standard API call to create… --- ## [How can I get immediate subdirectories using the path hierarchy tokenizer?](https://discuss.elastic.co/t/how-can-i-get-immediate-subdirectories-using-the-path-hierarchy-tokenizer/274099)
**Author:** [@F\_eq\_kx](https://discuss.elastic.co/u/F_eq_kx)\ **Replies:** 4\ **Last updated:** [May 27, 2021, 1:25pm UTC](https://discuss.elastic.co/t/how-can-i-get-immediate-subdirectories-using-the-path-hierarchy-tokenizer/274099 "2021-05-27T13:25:26Z")
I'm using the path hierarchy tokenizer and, using the examples, I would like to give /User/alice/photos/2017/05 to Elasticsearch and have it list the immediate subdirectories (e.g. 15 and 16). How can I do this? --- ## [Curator action for deletion based on space and oldest data](https://discuss.elastic.co/t/curator-action-for-deletion-based-on-space-and-oldest-data/272362)
**Author:** [@Voula\_Mikr](https://discuss.elastic.co/u/Voula_Mikr)\ **Replies:** 6\ **Last updated:** [May 27, 2021, 12:16pm UTC](https://discuss.elastic.co/t/curator-action-for-deletion-based-on-space-and-oldest-data/272362 "2021-05-27T12:16:14Z")
Hi, I am trying to delete the oldest data based on space. I have two kind of data. The name of the one is \-2021-05-04, for example, and the name of the second type is like \-2021.05.04. My first… --- ## [Don't working scheduler in snapshot policy](https://discuss.elastic.co/t/dont-working-scheduler-in-snapshot-policy/274156)
**Author:** [@Valery1976](https://discuss.elastic.co/u/Valery1976)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 8:06am UTC](https://discuss.elastic.co/t/dont-working-scheduler-in-snapshot-policy/274156 "2021-05-27T08:06:33Z")
Hi, I'm was install 3nodes-cluster elasticsearch. Setted the snapshot policy for indecies rotation. Running the policy manually working success, but through scheduler don't work. Version of ElasticSearch is 7.12.1 G… --- ## [Index is not Rollover](https://discuss.elastic.co/t/index-is-not-rollover/273636)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [May 27, 2021, 7:42am UTC](https://discuss.elastic.co/t/index-is-not-rollover/273636 "2021-05-27T07:42:56Z")
I have 8 node cluster 3 master 3 hot node and 2 warm node.I have an index called log-wlb-system in hot node it will rotate every index after 1 day or when its size reach to 1gb and in warm node the retention is 2 days … --- ## [Reindex API is extremely slow](https://discuss.elastic.co/t/reindex-api-is-extremely-slow/273510)
**Author:** [@abhinav2](https://discuss.elastic.co/u/abhinav2)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 6:57am UTC](https://discuss.elastic.co/t/reindex-api-is-extremely-slow/273510 "2021-05-27T06:57:26Z")
I'm getting a write speed of 680 KB/s per node for the reindex API. Cluster is a 16 data node cluster with 3 master nodes. Each data node has 1.3 TB gp3 Disk with 3000 IOPS and 125 MBps throughput. Reindex API is using… --- ## [Terms query success on ID, fails on everything else](https://discuss.elastic.co/t/terms-query-success-on-id-fails-on-everything-else/274075)
**Author:** [@pyrovoice](https://discuss.elastic.co/u/pyrovoice)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 6:47am UTC](https://discuss.elastic.co/t/terms-query-success-on-id-fails-on-everything-else/274075 "2021-05-27T06:47:50Z")
Doing the following search: curl --location --request POST 'localhost:9200/author/\_search?pretty' \\ --header 'Content-Type: application/json' \\ --data-raw '{ "query": { "terms": { "\_id": \[ "myName" \] … --- ## [Help in a search query](https://discuss.elastic.co/t/help-in-a-search-query/274112)
**Author:** [@appydam](https://discuss.elastic.co/u/appydam)\ **Replies:** 2\ **Last updated:** [May 27, 2021, 6:24am UTC](https://discuss.elastic.co/t/help-in-a-search-query/274112 "2021-05-27T06:24:59Z")
I have {id, title, short\_description and description} in my document. I have to do search query on description text and want to reach title. Like if a person enters something in the description then matching titles appe… --- ## [ElasticSearch Windows 2019 service fails to start](https://discuss.elastic.co/t/elasticsearch-windows-2019-service-fails-to-start/274137)
**Author:** [@ddemland](https://discuss.elastic.co/u/ddemland)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 6:13am UTC](https://discuss.elastic.co/t/elasticsearch-windows-2019-service-fails-to-start/274137 "2021-05-27T06:13:56Z")
I have installed ElasticSerach on Windows 2019 server. When I start the service manually it starts, but seems to be a little slow. When I reboot the system the Event log shows the following error: The Elasticsearch serv… --- ## [Elastic-APM Invalid index name \[\_license\]](https://discuss.elastic.co/t/elastic-apm-invalid-index-name-license/274131)
**Author:** [@pszm](https://discuss.elastic.co/u/pszm)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 5:01am UTC](https://discuss.elastic.co/t/elastic-apm-invalid-index-name-license/274131 "2021-05-27T05:01:24Z")
We are trying to get elastic-apm install (for now on our development systems). According to Homebrew, we have the latest elasticsearch-oss (7.10.2), kibana-oss (7.10.2) and today installed apm-server-oss (which is at ve… --- ## [Metricbeat Elasticsearch Module not working](https://discuss.elastic.co/t/metricbeat-elasticsearch-module-not-working/274020)
**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 3:54am UTC](https://discuss.elastic.co/t/metricbeat-elasticsearch-module-not-working/274020 "2021-05-27T03:54:44Z")
Issue - Trying to set up metricbeat to collect ES metric but upon testing the elasticsearch-xpack module, its not able to connect except for "node\_stats" metric and generating the error as shown below. Can someone poin… --- ## [Give A User Permissions To On-board New Users](https://discuss.elastic.co/t/give-a-user-permissions-to-on-board-new-users/274059)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 3:08am UTC](https://discuss.elastic.co/t/give-a-user-permissions-to-on-board-new-users/274059 "2021-05-27T03:08:06Z")
I want to create a user who can create and manage new user onboarding. What I did is, created a user and assign a role with manage\_security permission for the cluster. Problem: the user who is assigned with manage\_secu… --- ## [After enable xpack i cant change passwords](https://discuss.elastic.co/t/after-enable-xpack-i-cant-change-passwords/274092)
**Author:** [@Dimitri\_Goldshtein](https://discuss.elastic.co/u/Dimitri_Goldshtein)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 1:22am UTC](https://discuss.elastic.co/t/after-enable-xpack-i-cant-change-passwords/274092 "2021-05-27T01:22:07Z")
After enable xpack i cant change passwords, commen to discovery.seed\_hosts and cluster.initial\_master\_nodes did not help! I get this error when trying to setup password: /usr/share/elasticsearch/bin# sudo ./elast… --- ## [How to decide storage size of an index in Elasticsearch](https://discuss.elastic.co/t/how-to-decide-storage-size-of-an-index-in-elasticsearch/274076)
**Author:** [@rknd](https://discuss.elastic.co/u/rknd)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 11:10pm UTC](https://discuss.elastic.co/t/how-to-decide-storage-size-of-an-index-in-elasticsearch/274076 "2021-05-26T23:10:06Z")
I want to ask, How should i decide storage size for an index ? Is there any storage limit for shards or am i able to store 20 gb per shard and will it perform same ? Thanks for answering and explaining --- ## [Elasticsearch gc overhead](https://discuss.elastic.co/t/elasticsearch-gc-overhead/274089)
**Author:** [@muralikri32](https://discuss.elastic.co/u/muralikri32)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 11:09pm UTC](https://discuss.elastic.co/t/elasticsearch-gc-overhead/274089 "2021-05-26T23:09:08Z")
Hi team, i am using ES 6.6.X version in K8 environment and running as 3 node cluster. each node has 64GB ram and i configured heap memory as 32GB. it is quite running fine for 2 months and recently i could see lot of gc… --- ## [Unable to post data via ElasticLowLevelClient](https://discuss.elastic.co/t/unable-to-post-data-via-elasticlowlevelclient/274078)
**Author:** [@NeilRedfern](https://discuss.elastic.co/u/NeilRedfern)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 2:42pm UTC](https://discuss.elastic.co/t/unable-to-post-data-via-elasticlowlevelclient/274078 "2021-05-26T14:42:54Z")
Hello, I'm attempting to add data into our Elastic cloud instance following the examples within the post Exporting Azure Log data to the ELK stack with Azure Functions and Event Hubs However despite my best efforts I r… --- ## [Legacy index templates](https://discuss.elastic.co/t/legacy-index-templates/273763)
**Author:** [@6NMgfDwZ3](https://discuss.elastic.co/u/6NMgfDwZ3)\ **Replies:** 0\ **Last updated:** [May 23, 2021, 3:48pm UTC](https://discuss.elastic.co/t/legacy-index-templates/273763 "2021-05-23T15:48:21Z")
Hello, Environment: Elasticsearch 7.12.1 (Basic License) Kibana 7.12.1 (Basic License) Filebeat 7.12.1 (Basic License) Ubuntu 21.04 I have an index template named myapp the index pattern myapp-\*. This index template… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=637) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=639)