# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1300 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1301 --- ## [Curator for Elasticsearch 7.0](https://discuss.elastic.co/t/curator-for-elasticsearch-7-0/177851)
**Author:** [@topher](https://discuss.elastic.co/u/topher)\ **Replies:** 6\ **Last updated:** [May 14, 2019, 2:57pm UTC](https://discuss.elastic.co/t/curator-for-elasticsearch-7-0/177851 "2019-05-14T14:57:29Z")
I use Curator 5.6 to keep my logs under control size wise, which for the past year or so has worked wonderfully. Last week, I updated to version 7.0 of the Elastic stack. This morning I notice that my data is significa… --- ## [Scheduled Watcher Tasks Failing](https://discuss.elastic.co/t/scheduled-watcher-tasks-failing/177336)
**Author:** [@ccampbell](https://discuss.elastic.co/u/ccampbell)\ **Replies:** 13\ **Last updated:** [May 14, 2019, 2:26pm UTC](https://discuss.elastic.co/t/scheduled-watcher-tasks-failing/177336 "2019-05-14T14:26:23Z")
I've just completed an upgrade to 6.7.0 (for Elasticsearch, Logstash, and Kibana), but my scheduled watches are not running correctly. These watches worked under 6.4.0 (although I had to start and stop the watcher servic… --- ## [Query to get exact cardinality counts](https://discuss.elastic.co/t/query-to-get-exact-cardinality-counts/180836)
**Author:** [@lask001](https://discuss.elastic.co/u/lask001)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 2:24pm UTC](https://discuss.elastic.co/t/query-to-get-exact-cardinality-counts/180836 "2019-05-14T14:24:45Z")
I'm trying to get an exact count of documents that meet a specific criteria. Aggs queries are somewhat helpful, but I need to be able to return results as well so hoping someone knows a good way to approach this problem.… --- ## [How to group surnames in Elasticsearch index by metaphone code? Should I use russian analyzer somehow?](https://discuss.elastic.co/t/how-to-group-surnames-in-elasticsearch-index-by-metaphone-code-should-i-use-russian-analyzer-somehow/181019)
**Author:** [@OksanaBarshchyk](https://discuss.elastic.co/u/OksanaBarshchyk)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 2:20pm UTC](https://discuss.elastic.co/t/how-to-group-surnames-in-elasticsearch-index-by-metaphone-code-should-i-use-russian-analyzer-somehow/181019 "2019-05-14T14:20:31Z")
How to group surnames in Elasticsearch index by metaphone code? Should I use russian analyzer somehow? --- ## [Warning on new elk](https://discuss.elastic.co/t/warning-on-new-elk/181013)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 1:46pm UTC](https://discuss.elastic.co/t/warning-on-new-elk/181013 "2019-05-14T13:46:26Z")
Following message in log file. and warning pop up on kibana saying 1 of 5 shard failed when I try to go in discover tab on kibana and select index. this is not {"type": "server", "timestamp": "2019-05-14T08:29:06,050-0… --- ## [Sum bucket aggregation (high cardinality fields) - optimize query](https://discuss.elastic.co/t/sum-bucket-aggregation-high-cardinality-fields-optimize-query/181012)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 1:42pm UTC](https://discuss.elastic.co/t/sum-bucket-aggregation-high-cardinality-fields-optimize-query/181012 "2019-05-14T13:42:36Z")
Hi! I need to get a specific metric per index. In the query, I have an IP field cardinality and a filtered cardinality of the same IP addresses grouped by a field. I need to subtract the filtered value from the total nu… --- ## [CCR on Elastic Cloud](https://discuss.elastic.co/t/ccr-on-elastic-cloud/180795)
**Author:** [@val](https://discuss.elastic.co/u/val)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 1:35pm UTC](https://discuss.elastic.co/t/ccr-on-elastic-cloud/180795 "2019-05-14T13:35:17Z")
We'd like to set up CCR between two 6.7.2 clusters in the same ES Cloud region. However, CCR seems to be disabled by default on Elastic Cloud (i.e. xpack.ccr.enabled is false in the nodes and cluster settings). Does an… --- ## [Lucene query string to match missing fields](https://discuss.elastic.co/t/lucene-query-string-to-match-missing-fields/180993)
**Author:** [@michele\_crudele](https://discuss.elastic.co/u/michele_crudele)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 11:57am UTC](https://discuss.elastic.co/t/lucene-query-string-to-match-missing-fields/180993 "2019-05-14T11:57:33Z")
I'm in the need to write a Lucene query string (I cannot use ES DSL unfortunately) to match all documents where a given field is missing: e.g. something like \_missing\_:mydoc.field. I tried the \_exists\_ special field, but… --- ## [Not all Cloudtrail logs arrive in Elasticsearch](https://discuss.elastic.co/t/not-all-cloudtrail-logs-arrive-in-elasticsearch/180990)
**Author:** [@davidbien](https://discuss.elastic.co/u/davidbien)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 11:37am UTC](https://discuss.elastic.co/t/not-all-cloudtrail-logs-arrive-in-elasticsearch/180990 "2019-05-14T11:37:46Z")
Hi, I'm having a problem with sending cloudtrail logs to elasticsearch. Some of the logs don't reach ES and I don't get any errors regarding this? I use lambda for this and most of the logs are put fine but, for example… --- ## [Fastest way to retrieve all ids in an index?](https://discuss.elastic.co/t/fastest-way-to-retrieve-all-ids-in-an-index/180955)
**Author:** [@akshaymaniyar](https://discuss.elastic.co/u/akshaymaniyar)\ **Replies:** 2\ **Last updated:** [May 14, 2019, 11:06am UTC](https://discuss.elastic.co/t/fastest-way-to-retrieve-all-ids-in-an-index/180955 "2019-05-14T11:06:21Z")
I have an index which has around 300 million documents. What is the fastest recommended way to retrieve all the documentIDs from the index? Currently I am using the below python script for doing a scan and scroll to ret… --- ## [Elastic search 7.0 cluster master node handover problem](https://discuss.elastic.co/t/elastic-search-7-0-cluster-master-node-handover-problem/180968)
**Author:** [@8a2de8f67ab9a7e963a8](https://discuss.elastic.co/u/8a2de8f67ab9a7e963a8)\ **Replies:** 6\ **Last updated:** [May 14, 2019, 11:01am UTC](https://discuss.elastic.co/t/elastic-search-7-0-cluster-master-node-handover-problem/180968 "2019-05-14T11:01:54Z")
\[2019-05-14T17:29:05,931\]\[WARN \]\[o.e.c.c.ClusterFormationFailureHelper\] \[node-1-search\] master not discovered or elected yet, an election requires a node with id \[oTcvvEb8R2G73xLl0U3srQ\], have discovered which is not a … --- ## [No filter clause still the query\_cache showing data](https://discuss.elastic.co/t/no-filter-clause-still-the-query-cache-showing-data/180981)
**Author:** [@pokaleshrey](https://discuss.elastic.co/u/pokaleshrey)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 10:39am UTC](https://discuss.elastic.co/t/no-filter-clause-still-the-query-cache-showing-data/180981 "2019-05-14T10:39:55Z")
Hi, I do not have any filter clause in my queries. But when i do a /\_nodes/stats/indices/query\_cache, there is some data present in memory that is cached. I wanted to know what is other type of data that can go into q… --- ## [Puppet 6 support fo Elastic modules](https://discuss.elastic.co/t/puppet-6-support-fo-elastic-modules/177604)
**Author:** [@kzgrzendek](https://discuss.elastic.co/u/kzgrzendek)\ **Replies:** 2\ **Last updated:** [May 14, 2019, 10:36am UTC](https://discuss.elastic.co/t/puppet-6-support-fo-elastic-modules/177604 "2019-05-14T10:36:24Z")
Hello everyone :slight\_smile: As I am new on this forum, I hope I'm posting in the right category and that my post isn't a duplicate (although I didn't found any precise info about this topic). When I'm looking at the … --- ## [Matching Search types in elasticsearch](https://discuss.elastic.co/t/matching-search-types-in-elasticsearch/180973)
**Author:** [@vinay123](https://discuss.elastic.co/u/vinay123)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 10:04am UTC](https://discuss.elastic.co/t/matching-search-types-in-elasticsearch/180973 "2019-05-14T10:04:42Z")
Hello Everyone, my question is in search type once a match is identified for a given search type it is selected and subsequent SEARCH TYPES are should not be evaluated. A match should occurs is when all of the CONDI… --- ## [Elasticsearch not reassigning shards after node rejoined](https://discuss.elastic.co/t/elasticsearch-not-reassigning-shards-after-node-rejoined/178424)
**Author:** [@rsyne](https://discuss.elastic.co/u/rsyne)\ **Replies:** 23\ **Last updated:** [May 14, 2019, 9:14am UTC](https://discuss.elastic.co/t/elasticsearch-not-reassigning-shards-after-node-rejoined/178424 "2019-05-14T09:14:46Z")
Hi My cluster (running Elasticsearch 6.4.3) consists of three nodes running on seperate VMs. One VM crashed. After it was brought back to life, the node rejoined the cluster, but shards reassignment did not start. CPU u… --- ## [Is there an ES equivalent for Solr Tagger Handler?](https://discuss.elastic.co/t/is-there-an-es-equivalent-for-solr-tagger-handler/179062)
**Author:** [@Thales\_Valias](https://discuss.elastic.co/u/Thales_Valias)\ **Replies:** 6\ **Last updated:** [May 14, 2019, 9:14am UTC](https://discuss.elastic.co/t/is-there-an-es-equivalent-for-solr-tagger-handler/179062 "2019-05-14T09:14:02Z")
Solr has this Tagger Handler feature, is there any ES out of the box equivalent for that? How does the tagger works ? The tagger handler relies on a dedicated collection in which it stores the entities to be extracted.… --- ## [Need Details on ops/sec](https://discuss.elastic.co/t/need-details-on-ops-sec/180729)
**Author:** [@Balmukund](https://discuss.elastic.co/u/Balmukund)\ **Replies:** 9\ **Last updated:** [May 14, 2019, 9:11am UTC](https://discuss.elastic.co/t/need-details-on-ops-sec/180729 "2019-05-14T09:11:59Z")
Hi, I am using Search query which gives the result as below. All,Min Throughput,relative-kibana-discover\_50%,0.02,ops/s All,Median Throughput,relative-kibana-discover\_50%,0.02,ops/s All,Max Throughput,relative-kibana… --- ## [Number of Shards open Issue - 7.0.1](https://discuss.elastic.co/t/number-of-shards-open-issue-7-0-1/180785)
**Author:** [@cmcdowell03](https://discuss.elastic.co/u/cmcdowell03)\ **Replies:** 5\ **Last updated:** [May 14, 2019, 9:11am UTC](https://discuss.elastic.co/t/number-of-shards-open-issue-7-0-1/180785 "2019-05-14T09:11:43Z")
I am attempting to upgrade from 6.\* to 7.0.1 and noticed there is a hard limit set for shards. My question is, is there any advice for limiting Shards open or re-configuring this limit? Per the documentation, the limit… --- ## [Unique IDs on joinType data structures](https://discuss.elastic.co/t/unique-ids-on-jointype-data-structures/180959)
**Author:** [@HJK181](https://discuss.elastic.co/u/HJK181)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 8:55am UTC](https://discuss.elastic.co/t/unique-ids-on-jointype-data-structures/180959 "2019-05-14T08:55:59Z")
As suggests in this issue: Possible to index duplicate documents with same id and routing id. · Issue #31976 · elastic/elasticsearch · GitHub I want to start a discussion about the problem described in Possible to index… --- ## [Unable to upgrade System owned cluster from 6.6.1 to 6.7.1](https://discuss.elastic.co/t/unable-to-upgrade-system-owned-cluster-from-6-6-1-to-6-7-1/180925)
**Author:** [@Jude\_Jerome](https://discuss.elastic.co/u/Jude_Jerome)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 6:40am UTC](https://discuss.elastic.co/t/unable-to-upgrade-system-owned-cluster-from-6-6-1-to-6-7-1/180925 "2019-05-14T06:40:53Z")
Hello Together, I am trying to upgrade my System owned cluster from 6.6.1 to 6.7.1 and while upgrading Stopping Index Lifecycle Management steps makes more time and it is failed with below error Error: Starting ste… --- ## [How can I download content from Elasticsearch](https://discuss.elastic.co/t/how-can-i-download-content-from-elasticsearch/180535)
**Author:** [@smshabab11](https://discuss.elastic.co/u/smshabab11)\ **Replies:** 4\ **Last updated:** [May 14, 2019, 5:37am UTC](https://discuss.elastic.co/t/how-can-i-download-content-from-elasticsearch/180535 "2019-05-14T05:37:31Z")
I have an UI to upload document which is getting indexed in Elasticsearch 7.0. I want to download the file from elasticsearch after getting indexed by clicking download button . Need suggestion ASAP. --- ## [How to compress data automatically?](https://discuss.elastic.co/t/how-to-compress-data-automatically/180908)
**Author:** [@david3721](https://discuss.elastic.co/u/david3721)\ **Replies:** 2\ **Last updated:** [May 14, 2019, 3:31am UTC](https://discuss.elastic.co/t/how-to-compress-data-automatically/180908 "2019-05-14T03:31:20Z")
I have very huge data very day,I have to delete manually. how to automate data compression elasticsearch? --- ## [Parsing Spring Boot Log Files (Request Body)](https://discuss.elastic.co/t/parsing-spring-boot-log-files-request-body/180906)
**Author:** [@vsk1](https://discuss.elastic.co/u/vsk1)\ **Replies:** 0\ **Last updated:** [May 14, 2019, 2:51am UTC](https://discuss.elastic.co/t/parsing-spring-boot-log-files-request-body/180906 "2019-05-14T02:51:29Z")
Hi, I'm using fluentd to ship the spring boot application logs(using logback) to elastic search. Its a comma separated(; as delimiter) and key value data... I need this to be able to query by the various keys of the da… --- ## [Merge Document increasing some doc count why?](https://discuss.elastic.co/t/merge-document-increasing-some-doc-count-why/180727)
**Author:** [@Dilip\_Kumar](https://discuss.elastic.co/u/Dilip_Kumar)\ **Replies:** 3\ **Last updated:** [May 14, 2019, 1:21am UTC](https://discuss.elastic.co/t/merge-document-increasing-some-doc-count-why/180727 "2019-05-14T01:21:50Z")
Before merge my index status is green open client 4m5zLWWweSzzWsYoJ4MewpeQ 5 1 344914 79523 1.1gb 654.6mb when i did merge operation by curl -XPOST 192.168.1.101:9200/client/\_forcemerge?max\_num\_segments=1 I found some… --- ## [Data/write/bulk \[s\]\[r\] - Some task running from more than 70days](https://discuss.elastic.co/t/data-write-bulk-s-r-some-task-running-from-more-than-70days/180128)
**Author:** [@Vikash\_Kumar1](https://discuss.elastic.co/u/Vikash_Kumar1)\ **Replies:** 5\ **Last updated:** [May 14, 2019, 1:17am UTC](https://discuss.elastic.co/t/data-write-bulk-s-r-some-task-running-from-more-than-70days/180128 "2019-05-14T01:17:49Z")
Hi All, I wanted to understand what is the below task that is executing from past 76 days. Is it expected to run for such a long time or is there any issue with my elasticsearch cluster. Also, we frequently face long G… --- ## [Removing Duplicate documents in ElasticSearch](https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876)
**Author:** [@Praneet\_Khandelwal](https://discuss.elastic.co/u/Praneet_Khandelwal)\ **Replies:** 1\ **Last updated:** [May 14, 2019, 12:23am UTC](https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876 "2019-05-14T00:23:45Z")
I have documents coming in from Logstash to Elasticsearch. I would like the following functionality. whenever a new document comes, it gets indexed in ES only if it is already not present in ES. As of now, I was trying t… --- ## [Overide seeting for CSV import after import](https://discuss.elastic.co/t/overide-seeting-for-csv-import-after-import/180898)
**Author:** [@stheologo](https://discuss.elastic.co/u/stheologo)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 10:43pm UTC](https://discuss.elastic.co/t/overide-seeting-for-csv-import-after-import/180898 "2019-05-13T22:43:25Z")
Hi, Is there a way to override the settings after you have done the CSV import ? This document shows you how to do it before you do the import, https://www.elastic.co/blog/importing-csv-and-log-data-into-elasticsearch-… --- ## [Hii I want to read Resumes using filebeat . Resumes are in (.docx) format](https://discuss.elastic.co/t/hii-i-want-to-read-resumes-using-filebeat-resumes-are-in-docx-format/180497)
**Author:** [@Vishnu\_mk](https://discuss.elastic.co/u/Vishnu_mk)\ **Replies:** 3\ **Last updated:** [May 13, 2019, 10:14pm UTC](https://discuss.elastic.co/t/hii-i-want-to-read-resumes-using-filebeat-resumes-are-in-docx-format/180497 "2019-05-13T22:14:17Z")
Please provide me a documentation if you can . I am not able to understand how to approach --- ## [Ingest (from filebeat): How to include date from log file header in each log line?](https://discuss.elastic.co/t/ingest-from-filebeat-how-to-include-date-from-log-file-header-in-each-log-line/180797)
**Author:** [@sockit](https://discuss.elastic.co/u/sockit)\ **Replies:** 3\ **Last updated:** [May 13, 2019, 9:40pm UTC](https://discuss.elastic.co/t/ingest-from-filebeat-how-to-include-date-from-log-file-header-in-each-log-line/180797 "2019-05-13T21:40:00Z")
I am using filebeat to send data directly to an elasticsearch ingest pipeline. The (daily) log files each have a date for the file in a header line at the beginning of the file: \` \*\* Application log \*\* Current day: 02/… --- ## [Node not able to rejoin cluster after network connectivity restored](https://discuss.elastic.co/t/node-not-able-to-rejoin-cluster-after-network-connectivity-restored/180890)
**Author:** [@bmm](https://discuss.elastic.co/u/bmm)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 9:07pm UTC](https://discuss.elastic.co/t/node-not-able-to-rejoin-cluster-after-network-connectivity-restored/180890 "2019-05-13T21:07:54Z")
Hi, I have a 3-node cluster, each of them are master eligible and discovery.zen.minimum\_master\_nodes: 2 Denoting nodes as A, B, C for simplicity. After a network failure, node C got disconnected from two other nodes, an… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1299) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1301)