# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=564 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 565 --- ## [Authorization implementation on elastic search results](https://discuss.elastic.co/t/authorization-implementation-on-elastic-search-results/285975)
**Author:** [@hafoosdev](https://discuss.elastic.co/u/hafoosdev)\ **Replies:** 0\ **Last updated:** [October 6, 2021, 4:53am UTC](https://discuss.elastic.co/t/authorization-implementation-on-elastic-search-results/285975 "2021-10-06T04:53:48Z")
We have a requirement to show/hide data to logged in User based on User permissions on each record that we have in Elasticsearch. The algorithm to check if the User has permissions on a given record requires to check mul… --- ## [Confusion on ILM policy and please help me to understand it](https://discuss.elastic.co/t/confusion-on-ilm-policy-and-please-help-me-to-understand-it/285969)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 0\ **Last updated:** [October 6, 2021, 3:37am UTC](https://discuss.elastic.co/t/confusion-on-ilm-policy-and-please-help-me-to-understand-it/285969 "2021-10-06T03:37:39Z")
Hi Team, I am working on a system where I expect daily I am going to ingest a data about 15 GB and my total retention of data should be about 365 days. However my active data would be about 1 month of indices. After 1 m… --- ## [FS Crawler appeared to work but Kibana displays 0 results](https://discuss.elastic.co/t/fs-crawler-appeared-to-work-but-kibana-displays-0-results/285936)
**Author:** [@neergttocsdivad](https://discuss.elastic.co/u/neergttocsdivad)\ **Replies:** 5\ **Last updated:** [October 6, 2021, 3:32am UTC](https://discuss.elastic.co/t/fs-crawler-appeared-to-work-but-kibana-displays-0-results/285936 "2021-10-06T03:32:11Z")
Hello I have created three document collections with which to experiment with Elastic (7.15) Each collection is stored in a separate folder. The folders are named: papers, numb3rs & novels. I have used FS Crawler 2.7… --- ## [How to combine data from different index in one view while viewing in kibana?](https://discuss.elastic.co/t/how-to-combine-data-from-different-index-in-one-view-while-viewing-in-kibana/284425)
**Author:** [@shi](https://discuss.elastic.co/u/shi)\ **Replies:** 7\ **Last updated:** [October 5, 2021, 4:47pm UTC](https://discuss.elastic.co/t/how-to-combine-data-from-different-index-in-one-view-while-viewing-in-kibana/284425 "2021-10-05T16:47:22Z")
I am trying to put traffic logs to Elasticsearch. Suppose there are 100 traffic rows. Suppose there are around 25 columns for each row. Suppose 15 columns of all of the 100 traffic logs are same . rest of the columns … --- ## [Super Helpful Feature Request regarding Elasticsearch node data](https://discuss.elastic.co/t/super-helpful-feature-request-regarding-elasticsearch-node-data/285722)
**Author:** [@pushshift](https://discuss.elastic.co/u/pushshift)\ **Replies:** 2\ **Last updated:** [October 5, 2021, 3:56pm UTC](https://discuss.elastic.co/t/super-helpful-feature-request-regarding-elasticsearch-node-data/285722 "2021-10-05T15:56:40Z")
I know as a sysadmin, it should be my responsibility to have a list somewhere that contains all the servers / server names that belong to a specific cluster. Unfortunately, sometimes when one is in a rush to meet a deadl… --- ## [Data tiers, searcheable snapshots and the frozen tier](https://discuss.elastic.co/t/data-tiers-searcheable-snapshots-and-the-frozen-tier/285928)
**Author:** [@ffknob](https://discuss.elastic.co/u/ffknob)\ **Replies:** 4\ **Last updated:** [October 5, 2021, 2:01pm UTC](https://discuss.elastic.co/t/data-tiers-searcheable-snapshots-and-the-frozen-tier/285928 "2021-10-05T14:01:48Z")
Hello I need some help figuring out data tiers, searcheable snapshots and the frozen tier. Consider this: GET \_cat/nodes?v&h=name,node.role&s=name name node.role es01 hms es02 mw es03 cm es04 fm POST \_snapshot/orde… --- ## [How 2 node elastic cluster will failover to secondary node if the primary node fails](https://discuss.elastic.co/t/how-2-node-elastic-cluster-will-failover-to-secondary-node-if-the-primary-node-fails/285919)
**Author:** [@Arcadix](https://discuss.elastic.co/u/Arcadix)\ **Replies:** 1\ **Last updated:** [October 5, 2021, 12:57pm UTC](https://discuss.elastic.co/t/how-2-node-elastic-cluster-will-failover-to-secondary-node-if-the-primary-node-fails/285919 "2021-10-05T12:57:46Z")
Hi, We have 1+1 Elasticsearch Clustering setup on windows server. we need to know which Elastic node URL should call in our application to have a proper Fail over. if one node is down how the application identify that… --- ## [Fvh highlighter problem with inverted proximity search](https://discuss.elastic.co/t/fvh-highlighter-problem-with-inverted-proximity-search/285916)
**Author:** [@Wonder\_Garance](https://discuss.elastic.co/u/Wonder_Garance)\ **Replies:** 0\ **Last updated:** [October 5, 2021, 12:31pm UTC](https://discuss.elastic.co/t/fvh-highlighter-problem-with-inverted-proximity-search/285916 "2021-10-05T12:31:11Z")
Hi, I use Elasticsearch 7.9.3, I configured 2 text fileds, one of them is for exact search. Mapping: "text\_html": { "type": "text", "term\_vector": "with\_positions\_offsets", "store": true, "a… --- ## [Unable to store .security index in s3 bucket](https://discuss.elastic.co/t/unable-to-store-security-index-in-s3-bucket/283734)
**Author:** [@Vaidesh](https://discuss.elastic.co/u/Vaidesh)\ **Replies:** 3\ **Last updated:** [October 5, 2021, 11:22am UTC](https://discuss.elastic.co/t/unable-to-store-security-index-in-s3-bucket/283734 "2021-10-05T11:22:32Z")
While taking back up of .security-7 index to S3 bucket getting below error, kindly help. "failures" : \[ { "index" : ".security-7", "index\_uuid" : ".security-7", "shard\_id" : 0, "rea… --- ## [Term does not return the exact results](https://discuss.elastic.co/t/term-does-not-return-the-exact-results/285909)
**Author:** [@Tung\_Doan](https://discuss.elastic.co/u/Tung_Doan)\ **Replies:** 0\ **Last updated:** [October 5, 2021, 11:11am UTC](https://discuss.elastic.co/t/term-does-not-return-the-exact-results/285909 "2021-10-05T11:11:04Z")
I have the following nested schema: dataCollectorInfo.files.indexedData.objects.measures.properties.unit "\_source": { "dataCollectorInfo": { "files": { "indexedData": { "objects": { "measures": { "p… --- ## [Error after upgrading to 7.15](https://discuss.elastic.co/t/error-after-upgrading-to-7-15/285890)
**Author:** [@Mads\_Lundegaard](https://discuss.elastic.co/u/Mads_Lundegaard)\ **Replies:** 7\ **Last updated:** [October 5, 2021, 9:13am UTC](https://discuss.elastic.co/t/error-after-upgrading-to-7-15/285890 "2021-10-05T09:13:49Z")
I am getting errors after I upgraded my client from 7.13.4 to 7.15. I suspect this is due to incompatible versions between server and client, but I am not sure. I tried upgrading the server, running in AWS but could not … --- ## [Internal Server Error](https://discuss.elastic.co/t/internal-server-error/283612)
**Author:** [@Busra\_Duygu](https://discuss.elastic.co/u/Busra_Duygu)\ **Replies:** 2\ **Last updated:** [October 5, 2021, 8:19am UTC](https://discuss.elastic.co/t/internal-server-error/283612 "2021-10-05T08:19:44Z")
Hello dear friends :slight\_smile: I have a problem. First thing I want to do is: I want to backup the indexes I created before, so I created some folders in "C:\\ "and added them to my elasticsearch.yml file.Then I restar… --- ## [Why Elasticsearch send more data between the cluster nodes](https://discuss.elastic.co/t/why-elasticsearch-send-more-data-between-the-cluster-nodes/285046)
**Author:** [@ravindrababua](https://discuss.elastic.co/u/ravindrababua)\ **Replies:** 6\ **Last updated:** [October 5, 2021, 6:36am UTC](https://discuss.elastic.co/t/why-elasticsearch-send-more-data-between-the-cluster-nodes/285046 "2021-10-05T06:36:01Z")
We have observed ELK sending the data with in the cluster nodes, we are not sending large data as a input to Elasticsearch. when the data is tool large it throws circuit break exception and cluster goes to YELLOW, but i… --- ## [Does watcher input uses script](https://discuss.elastic.co/t/does-watcher-input-uses-script/285868)
**Author:** [@Munavir\_Chavody](https://discuss.elastic.co/u/Munavir_Chavody)\ **Replies:** 0\ **Last updated:** [October 5, 2021, 3:11am UTC](https://discuss.elastic.co/t/does-watcher-input-uses-script/285868 "2021-10-05T03:11:06Z")
I'm getting dynamic execution circuit breaking error for many watchers in our environment saying that the input search query failed to compile. What is the reason for the same. How can we avoid such errors at all? Erro… --- ## [Document Scoring](https://discuss.elastic.co/t/document-scoring/285850)
**Author:** [@leemase004](https://discuss.elastic.co/u/leemase004)\ **Replies:** 0\ **Last updated:** [October 4, 2021, 8:41pm UTC](https://discuss.elastic.co/t/document-scoring/285850 "2021-10-04T20:41:19Z")
I know you can boost a fields score when running query (with the ^ symbol), but I am wondering if you can boost a fields score on/during ingestion when the data is initially coming into Elastic (via ingest pipelines or p… --- ## [Improving log ingestion speed and faster elasticsearch indexing](https://discuss.elastic.co/t/improving-log-ingestion-speed-and-faster-elasticsearch-indexing/285841)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 5\ **Last updated:** [October 4, 2021, 7:23pm UTC](https://discuss.elastic.co/t/improving-log-ingestion-speed-and-faster-elasticsearch-indexing/285841 "2021-10-04T19:23:37Z")
Hello, I currently have a single node set up (a single machine where E-L-K is installed ) where a couple of gzip files are being ingested , each of these files would be of ~600MB and has several thousand lines of logs… --- ## [How to generate Rules for anomaly detection ML?](https://discuss.elastic.co/t/how-to-generate-rules-for-anomaly-detection-ml/285707)
**Author:** [@MLsuper](https://discuss.elastic.co/u/MLsuper)\ **Replies:** 4\ **Last updated:** [October 4, 2021, 3:49pm UTC](https://discuss.elastic.co/t/how-to-generate-rules-for-anomaly-detection-ml/285707 "2021-10-04T15:49:11Z")
When I go to Kibana-\>Alerts and Actions I only see Alerts and Connectors and not Rules, I want to create a rule/generate alerts for anomaly detection(ML job) but I do not see Rules or anomaly detection alert, how do I en… --- ## [Need help in understanding query search of elastic search with sub string search](https://discuss.elastic.co/t/need-help-in-understanding-query-search-of-elastic-search-with-sub-string-search/285833)
**Author:** [@S\_basu](https://discuss.elastic.co/u/S_basu)\ **Replies:** 0\ **Last updated:** [October 4, 2021, 2:52pm UTC](https://discuss.elastic.co/t/need-help-in-understanding-query-search-of-elastic-search-with-sub-string-search/285833 "2021-10-04T14:52:32Z")
Hi All, We have created an index with analyzer which uses "keyword" as tokenizer. We have values such in a field as role\_number which may or may not contain any special character such as - or \_, for examplecontains data… --- ## [Is it possible to change the repo backing a searchable snapshot without interruption?](https://discuss.elastic.co/t/is-it-possible-to-change-the-repo-backing-a-searchable-snapshot-without-interruption/285832)
**Author:** [@BenB196](https://discuss.elastic.co/u/BenB196)\ **Replies:** 0\ **Last updated:** [October 4, 2021, 2:49pm UTC](https://discuss.elastic.co/t/is-it-possible-to-change-the-repo-backing-a-searchable-snapshot-without-interruption/285832 "2021-10-04T14:49:46Z")
Hi All, I'm trying to find out if it is possible to change the backing repository of a searchable snapshot? Looking at the docs, it doesn't appear to mention this. Example of what I'm trying to do: Have an index, ind… --- ## [Sinding date field to elastic search through logstash](https://discuss.elastic.co/t/sinding-date-field-to-elastic-search-through-logstash/285811)
**Author:** [@mangeshs](https://discuss.elastic.co/u/mangeshs)\ **Replies:** 5\ **Last updated:** [October 4, 2021, 12:40pm UTC](https://discuss.elastic.co/t/sinding-date-field-to-elastic-search-through-logstash/285811 "2021-10-04T12:40:00Z")
I have log format as 2021-10-04 14:20:27,552 --- and I want field in Elasticsearch as date type but i am getting string type my logstash config file is input { file{ path =\> "D:/Project/ogs/\*\*/\*.log" #path =\> … --- ## [My Indices are getting deleted automatically](https://discuss.elastic.co/t/my-indices-are-getting-deleted-automatically/285805)
**Author:** [@nnice](https://discuss.elastic.co/u/nnice)\ **Replies:** 2\ **Last updated:** [October 4, 2021, 11:58am UTC](https://discuss.elastic.co/t/my-indices-are-getting-deleted-automatically/285805 "2021-10-04T11:58:28Z")
Dear Community, I'm new to ELK. Need you assistance why my indices are getting deleted. if you could let me know what is the issue. I'll be highly obliged for that. Thank you soooo muchhhhh # ======================== E… --- ## [Warm data tier nodes are being queried in recent data queries](https://discuss.elastic.co/t/warm-data-tier-nodes-are-being-queried-in-recent-data-queries/285798)
**Author:** [@dordadush](https://discuss.elastic.co/u/dordadush)\ **Replies:** 1\ **Last updated:** [October 4, 2021, 11:28am UTC](https://discuss.elastic.co/t/warm-data-tier-nodes-are-being-queried-in-recent-data-queries/285798 "2021-10-04T11:28:44Z")
Hello everyone! :slight\_smile: I have added a warm data node in my Elastic Cloud deployment. I have also configured the relevant ILM policies to use the Warm phase accordingly. The problem is that the warm nodes are b… --- ## [Elasticsearch - Change nodes to data only](https://discuss.elastic.co/t/elasticsearch-change-nodes-to-data-only/285007)
**Author:** [@voom](https://discuss.elastic.co/u/voom)\ **Replies:** 1\ **Last updated:** [October 4, 2021, 11:02am UTC](https://discuss.elastic.co/t/elasticsearch-change-nodes-to-data-only/285007 "2021-10-04T11:02:25Z")
Hi All, I have an 8 node cluster which is made up of the following roles: 3 master-only nodes 5 master + data nodes I am planning on having the 5 master + data nodes converted to data only and was planning on followi… --- ## [Reindex API - Need to improve performance](https://discuss.elastic.co/t/reindex-api-need-to-improve-performance/285453)
**Author:** [@Seba](https://discuss.elastic.co/u/Seba)\ **Replies:** 2\ **Last updated:** [October 4, 2021, 9:40am UTC](https://discuss.elastic.co/t/reindex-api-need-to-improve-performance/285453 "2021-10-04T09:40:07Z")
Hello, The goal is to accelerate the reindex of indexes from an ELK cluster (version 7.6.2) composed of 3 nodes each with all the roles to a datastream in an ELK cluster (version 7.11) installed in a container system w… --- ## [Need a little help for elasticsearch query](https://discuss.elastic.co/t/need-a-little-help-for-elasticsearch-query/285794)
**Author:** [@FALEN](https://discuss.elastic.co/u/FALEN)\ **Replies:** 0\ **Last updated:** [October 4, 2021, 9:02am UTC](https://discuss.elastic.co/t/need-a-little-help-for-elasticsearch-query/285794 "2021-10-04T09:02:57Z")
Hi everyone, Need help to convert this kibana query body to elastic query api format. My goal is to create an api query, that returns all the data between spesific times. So i can reingest this data to logstash, and p… --- ## [How to minimize the reindexing time](https://discuss.elastic.co/t/how-to-minimize-the-reindexing-time/285739)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 7\ **Last updated:** [October 3, 2021, 11:31am UTC](https://discuss.elastic.co/t/how-to-minimize-the-reindexing-time/285739 "2021-10-03T11:31:42Z")
Using ELK 7.13 cluster 3 master 6 data nodes and 2 coordinate nodes Master node configuration Heap Size: 8gb Storage: 50gb(ssd) CPU: 4 Data node configuration Heap Size: 28gb Storage: 1tb(ssd) CPU: 8 Coordinate no… --- ## [ILM keeps losing track of indices](https://discuss.elastic.co/t/ilm-keeps-losing-track-of-indices/284055)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 1\ **Last updated:** [October 4, 2021, 7:40am UTC](https://discuss.elastic.co/t/ilm-keeps-losing-track-of-indices/284055 "2021-10-04T07:40:11Z")
We keep seeing the issue described here where Elasticsearch is not rolling over indices properly. \[2021-09-13T12:39:16,671\]\[ERROR\]\[o.e.x.i.IndexLifecycleRunner\] \[ecom-repository01\] policy \[test-policy\] for index \[error… --- ## [Audit log rolling file retention question](https://discuss.elastic.co/t/audit-log-rolling-file-retention-question/283796)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 1\ **Last updated:** [October 4, 2021, 7:30am UTC](https://discuss.elastic.co/t/audit-log-rolling-file-retention-question/283796 "2021-10-04T07:30:23Z")
Our elasticsearch \_audit.json files get rolled over daily to -audit-YYYY-MM-DD.json files and appear to be kept for 8 days. For some test nodes, I'd like to reduce the number of days these are kept for disk space usage.… --- ## [Elasticsearch Reindex is slow compare to REALLOCATION](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 7\ **Last updated:** [October 4, 2021, 7:19am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785 "2021-10-04T07:19:27Z")
Hello Team I am using Elasticsearch version 7.8.0. I have opened this thread for my understanding in regards to Elasticsearch internal operations. A Cluster Version : 5.6.3.0 B Cluster Version : 7.8.0.0 We are not u… --- ## [Alternative option of shingle token filter](https://discuss.elastic.co/t/alternative-option-of-shingle-token-filter/285787)
**Author:** [@ankurpatel](https://discuss.elastic.co/u/ankurpatel)\ **Replies:** 0\ **Last updated:** [October 4, 2021, 7:18am UTC](https://discuss.elastic.co/t/alternative-option-of-shingle-token-filter/285787 "2021-10-04T07:18:59Z")
I would like to implement auto suggest functionality just like google search. Shingle Token Filter is best suitable option for my requirement. Let me explain my index details. Index size is 16gb - 20 million documents.… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=563) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=565)