# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1205 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1206 --- ## [How to get total count in composite aggregation or any aggregation?](https://discuss.elastic.co/t/how-to-get-total-count-in-composite-aggregation-or-any-aggregation/191328)
**Author:** [@Prakash.Panjwani](https://discuss.elastic.co/u/Prakash.Panjwani)\ **Replies:** 1\ **Last updated:** [August 2, 2019, 5:57am UTC](https://discuss.elastic.co/t/how-to-get-total-count-in-composite-aggregation-or-any-aggregation/191328 "2019-08-02T05:57:30Z")
Hi, I have documents with property tags{scope, tag} as nested document. I have used composite aggregation to find out unique tags available . Composite aggregation does not return total number of buckets available. I … --- ## [Elasticsearch AND Hive table overwrite Action](https://discuss.elastic.co/t/elasticsearch-and-hive-table-overwrite-action/193450)
**Author:** [@pradige](https://discuss.elastic.co/u/pradige)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 5:55am UTC](https://discuss.elastic.co/t/elasticsearch-and-hive-table-overwrite-action/193450 "2019-08-02T05:55:32Z")
Hi I try to set connect ES and Hive so ckeck all document. so I did the process below download some jar file that elasticsearch deploy set cloudera configuration first of all , just simple case of create hive tab… --- ## [Group Documents by it's similarity](https://discuss.elastic.co/t/group-documents-by-its-similarity/193446)
**Author:** [@yelji](https://discuss.elastic.co/u/yelji)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 5:36am UTC](https://discuss.elastic.co/t/group-documents-by-its-similarity/193446 "2019-08-02T05:36:04Z")
Hello, I want to group the similar documents. Here is sample documents document 1 : { \_id : 1, message : "Have an adventurous and romantic valentines day this year" } document 2 : { \_id : 2, message : "Great ride w… --- ## [Heavy IO on Master-data node](https://discuss.elastic.co/t/heavy-io-on-master-data-node/193282)
**Author:** [@Su\_Ki](https://discuss.elastic.co/u/Su_Ki)\ **Replies:** 4\ **Last updated:** [August 2, 2019, 4:17am UTC](https://discuss.elastic.co/t/heavy-io-on-master-data-node/193282 "2019-08-02T04:17:04Z")
Hello, We have set an elasticsearch cluster that is consisted total of 8 nodes, where 6 of them are eligible master node and data node. The rest 2 are data only node. And they are all same spec servers. As we do perfor… --- ## [Is there any way to use client.search() in elastic clients(node.js) without index parameter](https://discuss.elastic.co/t/is-there-any-way-to-use-client-search-in-elastic-clients-node-js-without-index-parameter/193299)
**Author:** [@Jin1129](https://discuss.elastic.co/u/Jin1129)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 10:31am UTC](https://discuss.elastic.co/t/is-there-any-way-to-use-client-search-in-elastic-clients-node-js-without-index-parameter/193299 "2019-08-01T10:31:17Z")
I have an account that only accessible to an index. So index name had to be written in node.url for avoiding security exception but I got error code 400(bad Request) instead of security Exception, because of missing … --- ## [Filter an existing field to generate a new field](https://discuss.elastic.co/t/filter-an-existing-field-to-generate-a-new-field/193427)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 1\ **Last updated:** [August 2, 2019, 1:07am UTC](https://discuss.elastic.co/t/filter-an-existing-field-to-generate-a-new-field/193427 "2019-08-02T01:07:20Z")
Given the following event input inside Kibana Discover Page: cloudwatch\_logs.log\_group: /aws/lambda/b2\_raw\_processor @version: 1 message: 2019-08-01 15:30:19,207 - INFO - RAWPROC - PROCESS - SUCCESS - file\_type:video - … --- ## [Elasticsearch with dynamic data](https://discuss.elastic.co/t/elasticsearch-with-dynamic-data/193415)
**Author:** [@fox](https://discuss.elastic.co/u/fox)\ **Replies:** 1\ **Last updated:** [August 2, 2019, 12:48am UTC](https://discuss.elastic.co/t/elasticsearch-with-dynamic-data/193415 "2019-08-02T00:48:23Z")
I'd like to use Elasticsearch as a cache for our group service. It will replace an LDAP cache. I've done some experiments and it seems to be as responsive as the LDAPs. However, the data are quite dynamic--memberships… --- ## [Bootstrapping Elasticsearch 7.2.0 on AWS ECS](https://discuss.elastic.co/t/bootstrapping-elasticsearch-7-2-0-on-aws-ecs/193422)
**Author:** [@Suryatej\_Mukkamalla](https://discuss.elastic.co/u/Suryatej_Mukkamalla)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 11:16pm UTC](https://discuss.elastic.co/t/bootstrapping-elasticsearch-7-2-0-on-aws-ecs/193422 "2019-08-01T23:16:56Z")
Hello, I am trying to get a cluster up and running on AWS. I have the ECS cluster created, containers assigned, but whenever the Elasticsearch tasks try to begin, it fails on the bootstrapping section with the following… --- ## [Cluster unable to elect new master](https://discuss.elastic.co/t/cluster-unable-to-elect-new-master/193403)
**Author:** [@dashwood](https://discuss.elastic.co/u/dashwood)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 10:51pm UTC](https://discuss.elastic.co/t/cluster-unable-to-elect-new-master/193403 "2019-08-01T22:51:45Z")
Our data cluster seems to be unable to elect a new master. Due to unfortunate circumstances our master nodes have gone offline and cannot be recovered. (This happened when the cluster was active and didn't impact it at a… --- ## [Aliases using Index Templates and best practice for managing templates/policies](https://discuss.elastic.co/t/aliases-using-index-templates-and-best-practice-for-managing-templates-policies/193417)
**Author:** [@vikasp](https://discuss.elastic.co/u/vikasp)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 10:36pm UTC](https://discuss.elastic.co/t/aliases-using-index-templates-and-best-practice-for-managing-templates-policies/193417 "2019-08-01T22:36:53Z")
I am using logstash to push the data to elasticsearch and my ouptut plugin is like this: elasticsearch { hosts =\> \["x.x.x.x:9200","x.x.x.x:9200","x.x.x.x:9200"\] index =\> "union\_thread-%{+YYYY.MM}" user =\> "xxxxxxx" … --- ## [Linkedin OAuth 2.0 settings using elasticsearch](https://discuss.elastic.co/t/linkedin-oauth-2-0-settings-using-elasticsearch/192597)
**Author:** [@manideep](https://discuss.elastic.co/u/manideep)\ **Replies:** 4\ **Last updated:** [August 1, 2019, 8:26pm UTC](https://discuss.elastic.co/t/linkedin-oauth-2-0-settings-using-elasticsearch/192597 "2019-08-01T20:26:32Z")
Hi All, I am trying to access the Linkedin content via api using elasticsearch. I have created a linkedin app and trying to give the access and redirect url to http://localhost:9200. after click on the allow button, … --- ## [Watcher PDF/PNG auto-generated attached report broken when sending through email ( version 7.2)](https://discuss.elastic.co/t/watcher-pdf-png-auto-generated-attached-report-broken-when-sending-through-email-version-7-2/191510)
**Author:** [@tazzking](https://discuss.elastic.co/u/tazzking)\ **Replies:** 3\ **Last updated:** [August 1, 2019, 7:28pm UTC](https://discuss.elastic.co/t/watcher-pdf-png-auto-generated-attached-report-broken-when-sending-through-email-version-7-2/191510 "2019-08-01T19:28:48Z")
Hello Everyone, I have configured the watcher to send email with dashboard preview attachment as PDF or PNG, it generate the report (can be found in Kibana report) and send the email. But somehow the email attachment ge… --- ## [Remote reindex with Search Guard basic auth](https://discuss.elastic.co/t/remote-reindex-with-search-guard-basic-auth/192713)
**Author:** [@Onncek](https://discuss.elastic.co/u/Onncek)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 6:54pm UTC](https://discuss.elastic.co/t/remote-reindex-with-search-guard-basic-auth/192713 "2019-08-01T18:54:06Z")
Hello, I know SearchGuard is not an Elastic product, but I still try to ask if anyone knows how to do remote reindex when I using Search Guard. SG is used with Basic Authentication and with my own certificates. Could so… --- ## [ElasticSearch missing documents by list of id's as size of list increases](https://discuss.elastic.co/t/elasticsearch-missing-documents-by-list-of-ids-as-size-of-list-increases/193382)
**Author:** [@Sakib\_Rahman](https://discuss.elastic.co/u/Sakib_Rahman)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 5:54pm UTC](https://discuss.elastic.co/t/elasticsearch-missing-documents-by-list-of-ids-as-size-of-list-increases/193382 "2019-08-01T17:54:34Z")
Copying over from SO. I am querying ElasticSearch by a long list of id's. I usually break this list down to chunks and then perform the query with a subset of the list and aggregate the results later. ES seems to be mis… --- ## [Compute term frequency diff](https://discuss.elastic.co/t/compute-term-frequency-diff/193381)
**Author:** [@aczapszys](https://discuss.elastic.co/u/aczapszys)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 5:18pm UTC](https://discuss.elastic.co/t/compute-term-frequency-diff/193381 "2019-08-01T17:18:55Z")
I have an use case where I am comparing input titles to a corpus of titles to determine whether I have seen this title before. Using the default scoring match on the title field works great for ranking, but I also want … --- ## [Cannot connect to the localhost when running the Dockerfile](https://discuss.elastic.co/t/cannot-connect-to-the-localhost-when-running-the-dockerfile/192964)
**Author:** [@vampire](https://discuss.elastic.co/u/vampire)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 5:10pm UTC](https://discuss.elastic.co/t/cannot-connect-to-the-localhost-when-running-the-dockerfile/192964 "2019-08-01T17:10:17Z")
I have used the Dockerfile from this repo https://github.com/deviantony/docker-elk/tree/searchguard/elasticsearch and I have modified the dockerfile to install the search guard plugin, which looks like the following: F… --- ## [Primary and Replica index keep appearing on the same node](https://discuss.elastic.co/t/primary-and-replica-index-keep-appearing-on-the-same-node/193377)
**Author:** [@RayS](https://discuss.elastic.co/u/RayS)\ **Replies:** 3\ **Last updated:** [August 1, 2019, 4:37pm UTC](https://discuss.elastic.co/t/primary-and-replica-index-keep-appearing-on-the-same-node/193377 "2019-08-01T16:37:32Z")
I run a 4 node cluster with 3 data nodes. The other node is used to load balance Kibana traffic. I keep getting Primary and Replica index on the same node and not sure how this is happening. Here is an example from a … --- ## [\[ElasticSearch Query\] Select all not null fields](https://discuss.elastic.co/t/elasticsearch-query-select-all-not-null-fields/193340)
**Author:** [@amoussoubaruch](https://discuss.elastic.co/u/amoussoubaruch)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 3:44pm UTC](https://discuss.elastic.co/t/elasticsearch-query-select-all-not-null-fields/193340 "2019-08-01T15:44:18Z")
Hi evryone, I face some issue and i don't really know how i can solve that. I have one index with more than 600 fields. Sometimes i can have arround 500 fields with null values. I want to make a select query and have a… --- ## [Get data from SQL Tables](https://discuss.elastic.co/t/get-data-from-sql-tables/193356)
**Author:** [@Farouk6072](https://discuss.elastic.co/u/Farouk6072)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 3:07pm UTC](https://discuss.elastic.co/t/get-data-from-sql-tables/193356 "2019-08-01T15:07:10Z")
In my Logstash i have this statement : statement =\> "Select table1, table2, table3" I have this index : index =\> "test" My question how to get data from table1 on index "test"? in Canva --- ## [Elastic Heap and GC count increases when searching with thousands of characters](https://discuss.elastic.co/t/elastic-heap-and-gc-count-increases-when-searching-with-thousands-of-characters/193351)
**Author:** [@Ram\_Kumar1](https://discuss.elastic.co/u/Ram_Kumar1)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 2:48pm UTC](https://discuss.elastic.co/t/elastic-heap-and-gc-count-increases-when-searching-with-thousands-of-characters/193351 "2019-08-01T14:48:38Z")
Hi Everyone, Having an issue when user searches with thousands of characters Steps to reproduce : Enter 5K junk characters like '\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*' or '........................' and do multiple searches, after that ela… --- ## [Upgrade elastic in kubernetes](https://discuss.elastic.co/t/upgrade-elastic-in-kubernetes/193232)
**Author:** [@soandosnc](https://discuss.elastic.co/u/soandosnc)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 2:36pm UTC](https://discuss.elastic.co/t/upgrade-elastic-in-kubernetes/193232 "2019-08-01T14:36:00Z")
I have elastic 7.2 installed in kubernetes (via the elastic helm chart, secure example). How can I upgrade this to 7.3 (preferably in a rolling way)? --- ## [Sorting based on the field value](https://discuss.elastic.co/t/sorting-based-on-the-field-value/193348)
**Author:** [@shani\_angarkadu](https://discuss.elastic.co/u/shani_angarkadu)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 2:35pm UTC](https://discuss.elastic.co/t/sorting-based-on-the-field-value/193348 "2019-08-01T14:35:05Z")
Hi , Thanks in advance, I have a field name 'contact' , there are few documents where contact = ", " if contact = ", " those values has to come in the end. tried default sort , and asc but it is not efficient. --- ## [Boost docs similar to top result of query](https://discuss.elastic.co/t/boost-docs-similar-to-top-result-of-query/193346)
**Author:** [@jamartins](https://discuss.elastic.co/u/jamartins)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 2:32pm UTC](https://discuss.elastic.co/t/boost-docs-similar-to-top-result-of-query/193346 "2019-08-01T14:32:55Z")
Hi guys, I'd like to know how can I boost docs based on their similarity to the top result of the query according to a field, despite not matching with the query itself. Imagine this situation: I have an index of movie… --- ## [Scrolling Transport Error Elastic Search with python](https://discuss.elastic.co/t/scrolling-transport-error-elastic-search-with-python/188938)
**Author:** [@roma\_sodha](https://discuss.elastic.co/u/roma_sodha)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 2:21pm UTC](https://discuss.elastic.co/t/scrolling-transport-error-elastic-search-with-python/188938 "2019-08-01T14:21:52Z")
Hi, I get below error while doing scrolling with python. Any one encountered such error before.? Scrolling... Traceback (most recent call last): File , line 78, in page = es.scroll(scroll\_id= sid,scroll='2m') Fil… --- ## [In time series data in ElasticSearch, need to find the count of documents till a point at which the value changes](https://discuss.elastic.co/t/in-time-series-data-in-elasticsearch-need-to-find-the-count-of-documents-till-a-point-at-which-the-value-changes/193343)
**Author:** [@matterixn](https://discuss.elastic.co/u/matterixn)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 2:17pm UTC](https://discuss.elastic.co/t/in-time-series-data-in-elasticsearch-need-to-find-the-count-of-documents-till-a-point-at-which-the-value-changes/193343 "2019-08-01T14:17:36Z")
Lets say there are documents such as below, with these fields "time": X, "value": 1, "key": M "time": X-1, "value": 1, "key": M "time": X-2, "value": 1, "key": M "time": X-3, "value": 1, "key": M "time": X-4, "value": 1… --- ## [Significant terms aggregation on large dataset](https://discuss.elastic.co/t/significant-terms-aggregation-on-large-dataset/193341)
**Author:** [@carasel](https://discuss.elastic.co/u/carasel)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 2:09pm UTC](https://discuss.elastic.co/t/significant-terms-aggregation-on-large-dataset/193341 "2019-08-01T14:09:03Z")
I would like to do a significant terms aggregation across a lot of data. I currently have about 1TB of data in about 5 million top level documents, which all have a few hundred nested documents. The field that I want to … --- ## [ES won't start - where can I check logs?](https://discuss.elastic.co/t/es-wont-start-where-can-i-check-logs/192720)
**Author:** [@tryptych](https://discuss.elastic.co/u/tryptych)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 1:44pm UTC](https://discuss.elastic.co/t/es-wont-start-where-can-i-check-logs/192720 "2019-08-01T13:44:12Z")
I recently upgraded ES to 5.6 and it won't start now, and gives no clues as to why. Where are the log files for ES? I have a folder at /var/log/elasticsearch but it is empty. /var/log/messages only shows that it started … --- ## [Payload multi indices](https://discuss.elastic.co/t/payload-multi-indices/192847)
**Author:** [@nitingarg](https://discuss.elastic.co/u/nitingarg)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 1:39pm UTC](https://discuss.elastic.co/t/payload-multi-indices/192847 "2019-08-01T13:39:18Z")
I want to use first indices output m\_id as input second indices ind1,ind2/\_search { "query": { "bool": { "should": \[ { "term": { "linked\_m\_id": "11197" } }, { "term": { "m\_id": "ctx.payload.hits.hits.0.fiel… --- ## [Lowercase-Processor is missing in Documentation](https://discuss.elastic.co/t/lowercase-processor-is-missing-in-documentation/192898)
**Author:** [@PeterPain](https://discuss.elastic.co/u/PeterPain)\ **Replies:** 1\ **Last updated:** [August 1, 2019, 1:35pm UTC](https://discuss.elastic.co/t/lowercase-processor-is-missing-in-documentation/192898 "2019-08-01T13:35:38Z")
Hey dear Elastic-Team, I was looking through the documentation for ingest-pipelines and I've noticed that the documentation for the lowercase-processor is missing since version 6.2. i know this is not a big deal but w… --- ## [ES 2.4.1 fail on startup](https://discuss.elastic.co/t/es-2-4-1-fail-on-startup/193102)
**Author:** [@mhssm](https://discuss.elastic.co/u/mhssm)\ **Replies:** 2\ **Last updated:** [August 1, 2019, 1:08pm UTC](https://discuss.elastic.co/t/es-2-4-1-fail-on-startup/193102 "2019-08-01T13:08:35Z")
I have trouble starting ES 2.4.1 on an Ubuntu 14.04 box. ES has been installed using apt-get and is invoked by init (running as default "elasticsearch" user): admin@LOG:/tmp$ sudo -i service elasticsearch start \* Start… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1204) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1206)