# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=764 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 765 --- ## [Disk Usage problem](https://discuss.elastic.co/t/disk-usage-problem/255292)
**Author:** [@since1976](https://discuss.elastic.co/u/since1976)\ **Replies:** 3\ **Last updated:** [November 16, 2020, 5:26am UTC](https://discuss.elastic.co/t/disk-usage-problem/255292 "2020-11-16T05:26:52Z")
Hi, A bit of background in our environment, our logstash is installed with elastiflow. 1 cluster of 4 nodes elasticsearch with 2 P and 2 R in 2 zones. The setup is collecting logs from 2 WAN routers with netflow configu… --- ## [Secure Communications from Beats to Elasticsearch](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460)
**Author:** [@inf](https://discuss.elastic.co/u/inf)\ **Replies:** 3\ **Last updated:** [November 16, 2020, 4:27am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460 "2020-11-16T04:27:27Z")
Using Filebeat's article on ... found here as reference, we see that it says that "Filebeat can use any of the following authentication methods". 3 authentication methods were then listed. My question is: are we only ab… --- ## [How to write json array to elasticsearch's objetct field by es rest client?](https://discuss.elastic.co/t/how-to-write-json-array-to-elasticsearchs-objetct-field-by-es-rest-client/255224)
**Author:** [@KaimingWan](https://discuss.elastic.co/u/KaimingWan)\ **Replies:** 0\ **Last updated:** [November 12, 2020, 2:24pm UTC](https://discuss.elastic.co/t/how-to-write-json-array-to-elasticsearchs-objetct-field-by-es-rest-client/255224 "2020-11-12T14:24:03Z")
I use elasticsearch6.8 and jdk8. The key problem is hot to write json array string to elasticsearch's object type field. Following code show how I tried to write to es. My mapping is as follows: { "mapping": { "\_… --- ## [Node cold but phase hot](https://discuss.elastic.co/t/node-cold-but-phase-hot/254826)
**Author:** [@alexfritz](https://discuss.elastic.co/u/alexfritz)\ **Replies:** 7\ **Last updated:** [November 15, 2020, 8:55pm UTC](https://discuss.elastic.co/t/node-cold-but-phase-hot/254826 "2020-11-15T20:55:20Z")
Hello I am running Version: 6.8 when I look at \_cat/shards/$index it says that the node is cold. When I look at $index/\_ilm/explain it says the index is in phase hot: { "indices": { "7a900401ea8845819a5cf5931… --- ## [Curl Output is not complete](https://discuss.elastic.co/t/curl-output-is-not-complete/255048)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 6\ **Last updated:** [November 15, 2020, 5:33pm UTC](https://discuss.elastic.co/t/curl-output-is-not-complete/255048 "2020-11-15T17:33:36Z")
Hello Team I am using Elasticsearch Version 7.8.0. I am executing an query on Elasticsearch index. The output of CURL command is not complete, IT shows incomplete records only. I suppose there is some limit / default … --- ## [Wrong doc\_count in term\_vectors?](https://discuss.elastic.co/t/wrong-doc-count-in-term-vectors/255440)
**Author:** [@flm](https://discuss.elastic.co/u/flm)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 2:58pm UTC](https://discuss.elastic.co/t/wrong-doc-count-in-term-vectors/255440 "2020-11-15T14:58:15Z")
doc\_count is incremented when adding the same document several times. To reproduce execute twice: PUT /test\_index/\_doc/1 { "somefield": "somedata" } There is 1 document in the index but doc\_count=2: { "\_index"… --- ## [Rollup jobs: Error while inserting same job id, but can't view metadata](https://discuss.elastic.co/t/rollup-jobs-error-while-inserting-same-job-id-but-cant-view-metadata/255321)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [November 15, 2020, 9:27am UTC](https://discuss.elastic.co/t/rollup-jobs-error-while-inserting-same-job-id-but-cant-view-metadata/255321 "2020-11-15T09:27:31Z")
I was creating a job in 7.9.3 Elastic, but below is the error I'm getting. I had created before and deleted similar job. Any idea where to find the metadata and clear it off? {"error":{"root\_cause":\[{"type":"status\_exce… --- ## [Query data from C# program the result not equal to the data filter from kibana](https://discuss.elastic.co/t/query-data-from-c-program-the-result-not-equal-to-the-data-filter-from-kibana/255428)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 8:13am UTC](https://discuss.elastic.co/t/query-data-from-c-program-the-result-not-equal-to-the-data-filter-from-kibana/255428 "2020-11-15T08:13:51Z")
I am new to C#, and I want to write a program to query data from elasticsearch. First of all, I load kibana sample log and want to query from my C# program, and below is my C# program. I want to filter clientip is 20.10… --- ## [How to remove duplicate fields within doc?](https://discuss.elastic.co/t/how-to-remove-duplicate-fields-within-doc/254316)
**Author:** [@Kyle\_Neal](https://discuss.elastic.co/u/Kyle_Neal)\ **Replies:** 1\ **Last updated:** [November 15, 2020, 7:08am UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-fields-within-doc/254316 "2020-11-15T07:08:02Z")
Hi, Trying to migrate from 5.X to 6.X but have issues when reading ES data that was stored before the migration. The error I'm getting back is: {error,{400, \[{\<\<"error"\>\>, \[{\<\<"root\_cause"\>\>, … --- ## [Logstash-\>Elasticsearch document deduplication efficiency & optimization](https://discuss.elastic.co/t/logstash-elasticsearch-document-deduplication-efficiency-optimization/255336)
**Author:** [@petersedivec](https://discuss.elastic.co/u/petersedivec)\ **Replies:** 2\ **Last updated:** [November 14, 2020, 9:25pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-document-deduplication-efficiency-optimization/255336 "2020-11-14T21:25:23Z")
Quick Summary: Looking for feedback on our data deduplication approach using generated doc IDs for time-based event data vs leveraging ES for generating document IDs. Data Overview: We are working with an event dataset … --- ## [Node.role: \[data\_cold\]](https://discuss.elastic.co/t/node-role-data-cold/255410)
**Author:** [@mutt13y](https://discuss.elastic.co/u/mutt13y)\ **Replies:** 1\ **Last updated:** [November 14, 2020, 6:56pm UTC](https://discuss.elastic.co/t/node-role-data-cold/255410 "2020-11-14T18:56:17Z")
Hi According to I should set my cold nodes to data cold I have node.attr.class: cold node.roles: \["data\_cold"\] set in elasticsearch.yml Yet I get this on startup \[2020-11-14T12:24:27,891\]\[ERROR\]\[o.e.b.Bootstr… --- ## [Newbie: CentOS + Docker + Elastic Stack = Master not discovered yet](https://discuss.elastic.co/t/newbie-centos-docker-elastic-stack-master-not-discovered-yet/255240)
**Author:** [@Elitlogik](https://discuss.elastic.co/u/Elitlogik)\ **Replies:** 16\ **Last updated:** [November 14, 2020, 6:10pm UTC](https://discuss.elastic.co/t/newbie-centos-docker-elastic-stack-master-not-discovered-yet/255240 "2020-11-14T18:10:18Z")
I'm trying to start learning Elastic Stack, but I can't get it to even start. I have: Installed CentOS 8 Minimal Installed Docker, following this: https://docs.docker.com/engine/install/centos/ Managing non-root and … --- ## [Elasticsearch cluster is crashing often](https://discuss.elastic.co/t/elasticsearch-cluster-is-crashing-often/254640)
**Author:** [@alx2020](https://discuss.elastic.co/u/alx2020)\ **Replies:** 6\ **Last updated:** [November 14, 2020, 3:20pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-is-crashing-often/254640 "2020-11-14T15:20:55Z")
We have several large clusters v1.7. Please don't ask why :frowning: - we are on the way to upgrade to v7 but need some hints to make it stable. Recently one of the clusters started crashing several times per day and we… --- ## [Elasticsearch service won't start, nested elasticsearch node state exception](https://discuss.elastic.co/t/elasticsearch-service-wont-start-nested-elasticsearch-node-state-exception/255376)
**Author:** [@penguinairlines](https://discuss.elastic.co/u/penguinairlines)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 10:01pm UTC](https://discuss.elastic.co/t/elasticsearch-service-wont-start-nested-elasticsearch-node-state-exception/255376 "2020-11-13T22:01:48Z")
Hello, I noticed one of my elasticsearch nodes is down and started digging into the issue. I found the service won't start and notice a couple of key errors: ElasticsearchException\[failed to bind service\]; nested: Corr… --- ## [When we're writing to the Elasticsearch cluster, Reads are 10000x slower. (Understanding slow term query)](https://discuss.elastic.co/t/when-were-writing-to-the-elasticsearch-cluster-reads-are-10000x-slower-understanding-slow-term-query/255022)
**Author:** [@rex-remind](https://discuss.elastic.co/u/rex-remind)\ **Replies:** 10\ **Last updated:** [November 13, 2020, 9:31pm UTC](https://discuss.elastic.co/t/when-were-writing-to-the-elasticsearch-cluster-reads-are-10000x-slower-understanding-slow-term-query/255022 "2020-11-13T21:31:46Z")
Hello, I have recently indexed 25M docs into 2 nodes (+1 master) of 5 shards x 3 Availability Zones. I'm running in AWS ES, large boxes with with plenty of EBS storage, resource usage is low, average doc size is \< 1KiB.… --- ## [ML Jobs](https://discuss.elastic.co/t/ml-jobs/255357)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 9:25pm UTC](https://discuss.elastic.co/t/ml-jobs/255357 "2020-11-13T21:25:38Z")
Is there a way to do ML jobs for a high count of a specific field ? like i want high count of event.code 4626 --- ## [Correct Sizing and compression on Elasticsearch](https://discuss.elastic.co/t/correct-sizing-and-compression-on-elasticsearch/255349)
**Author:** [@xziten](https://discuss.elastic.co/u/xziten)\ **Replies:** 4\ **Last updated:** [November 13, 2020, 8:54pm UTC](https://discuss.elastic.co/t/correct-sizing-and-compression-on-elasticsearch/255349 "2020-11-13T20:54:36Z")
Hi, I am trying to get some ratio about Elasticsearch cluster. I have this base : \*\*For 100 Gb of raw data / day :\*\* \* 110 Gb of data are indexed (~ +10% for datatype) \* 220 Gb with 1 replica \* 6600 Gb per month \* +… --- ## [Index Templates - Differences between \_template & \_index\_template](https://discuss.elastic.co/t/index-templates-differences-between-template-index-template/255367)
**Author:** [@Eduardo\_Iglesias](https://discuss.elastic.co/u/Eduardo_Iglesias)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 8:09pm UTC](https://discuss.elastic.co/t/index-templates-differences-between-template-index-template/255367 "2020-11-13T20:09:24Z")
Hello everyone, Need help to identify the differences between the subject mentioned endpoints usage. \_index\_template endpoint POST \_index\_template/index-template-1 { "index\_patterns": \["logs-2019\*"\], "template": … --- ## [Encrypt data volume for Elasticsearch service on Azure](https://discuss.elastic.co/t/encrypt-data-volume-for-elasticsearch-service-on-azure/254980)
**Author:** [@anshuman](https://discuss.elastic.co/u/anshuman)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 7:10pm UTC](https://discuss.elastic.co/t/encrypt-data-volume-for-elasticsearch-service-on-azure/254980 "2020-11-13T19:10:33Z")
Do we have an option to encrypt the Elasticsearch data (or volume) which is offered as platform as a service on Azure using our own encryption key? --- ## [Elastic Query Regex](https://discuss.elastic.co/t/elastic-query-regex/254973)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 5:20pm UTC](https://discuss.elastic.co/t/elastic-query-regex/254973 "2020-11-13T17:20:33Z")
I am trying to write a regex query but it is not working accordingly am i doing something wrong I need to get back all values that ends in SVC in the field winlog.event\_data.TargetUserName { "query": { "regexp": { "… --- ## [Remove results with same id from Elasticsearch search result](https://discuss.elastic.co/t/remove-results-with-same-id-from-elasticsearch-search-result/255365)
**Author:** [@philippd](https://discuss.elastic.co/u/philippd)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 4:37pm UTC](https://discuss.elastic.co/t/remove-results-with-same-id-from-elasticsearch-search-result/255365 "2020-11-13T16:37:43Z")
Hi, I just joined the forums and would like to aks you for a little help with the following question I am not able to answer reading the docs: Let's assume we have a search result with 3 documents. Two of them share a … --- ## [How to check number of IPs being resolved by the Elasticsearch client?](https://discuss.elastic.co/t/how-to-check-number-of-ips-being-resolved-by-the-elasticsearch-client/255318)
**Author:** [@prasad98](https://discuss.elastic.co/u/prasad98)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 3:53pm UTC](https://discuss.elastic.co/t/how-to-check-number-of-ips-being-resolved-by-the-elasticsearch-client/255318 "2020-11-13T15:53:38Z")
I've some microservices which are dependent on the Elasticsearch cluster in production. This cluster is common for every microservice. Just to verify how many services IP is being resolved by Elasticsearch cluster, is t… --- ## [Beats Internal monitoring](https://discuss.elastic.co/t/beats-internal-monitoring/255350)
**Author:** [@MR\_551](https://discuss.elastic.co/u/MR_551)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 2:25pm UTC](https://discuss.elastic.co/t/beats-internal-monitoring/255350 "2020-11-13T14:25:03Z")
Hello everyone, Has anyone experienced an issue with internal monitoring for Beats? When i ship internal metrics for each Beat to a monitoring cluster, they get grouped under the same Beat on Stack Monitoring tab. I am … --- ## [Optimize bool filter query](https://discuss.elastic.co/t/optimize-bool-filter-query/255348)
**Author:** [@Daniele\_Renda](https://discuss.elastic.co/u/Daniele_Renda)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 1:45pm UTC](https://discuss.elastic.co/t/optimize-bool-filter-query/255348 "2020-11-13T13:45:39Z")
Hi, I'm using ES 7.10 deployed in Elastic Cloud. I developed a Java application that use High Level REST client. I want to make a query like this: index \>=1.67 && index\<=1.67 AND (sphereMin\<=-7 && sphereMax \>=-7 && cyl… --- ## [How to sort aggregated based on sub aggregation's doc\_count?](https://discuss.elastic.co/t/how-to-sort-aggregated-based-on-sub-aggregations-doc-count/255195)
**Author:** [@lessless](https://discuss.elastic.co/u/lessless)\ **Replies:** 6\ **Last updated:** [November 13, 2020, 9:41am UTC](https://discuss.elastic.co/t/how-to-sort-aggregated-based-on-sub-aggregations-doc-count/255195 "2020-11-13T09:41:22Z")
Hi, I'm completely stuck trying to figure out how to sort "users" based on "status\_codes" in the following query: { "size": 0, "aggs": { "users": { "terms": { "field": "user\_id" }, … --- ## [Apply scoring after filter](https://discuss.elastic.co/t/apply-scoring-after-filter/255309)
**Author:** [@Jalendra\_Bhanarkar](https://discuss.elastic.co/u/Jalendra_Bhanarkar)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 9:28am UTC](https://discuss.elastic.co/t/apply-scoring-after-filter/255309 "2020-11-13T09:28:45Z")
I want the score to be calculated after the filter is applied. Right now, the score gets calculated considering all the documents in the index. And other entity documents interfere with the score calculation which I don'… --- ## [java.lang.NoClassDefFoundError: org/elasticsearch/script/mustache/SearchTemplateRequestBuilder](https://discuss.elastic.co/t/java-lang-noclassdeffounderror-org-elasticsearch-script-mustache-searchtemplaterequestbuilder/255306)
**Author:** [@rakeshdkaatkar](https://discuss.elastic.co/u/rakeshdkaatkar)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 8:44am UTC](https://discuss.elastic.co/t/java-lang-noclassdeffounderror-org-elasticsearch-script-mustache-searchtemplaterequestbuilder/255306 "2020-11-13T08:44:43Z")
\[2020-11-13T14:09:32,460\]\[WARN \]\[o.e.h.AbstractHttpServerTransport\] \[L-356004158\] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/0:0:0:0:0:0:0:1:9200, remoteAddres… --- ## [Query - Regexp with special characters fails when used within multiple must\_not](https://discuss.elastic.co/t/query-regexp-with-special-characters-fails-when-used-within-multiple-must-not/255147)
**Author:** [@gonzalo2kx](https://discuss.elastic.co/u/gonzalo2kx)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 8:23am UTC](https://discuss.elastic.co/t/query-regexp-with-special-characters-fails-when-used-within-multiple-must-not/255147 "2020-11-13T08:23:49Z")
Good day, I am having issues when using regexp clause in my query for identifying and filtering out log messages produced by the filebeat, packetbeat, metricbeat, and auditbeat processes which contain the patterns below… --- ## [Adaptive replica selection in ES version 8](https://discuss.elastic.co/t/adaptive-replica-selection-in-es-version-8/255277)
**Author:** [@anon17386273](https://discuss.elastic.co/u/anon17386273)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 4:12am UTC](https://discuss.elastic.co/t/adaptive-replica-selection-in-es-version-8/255277 "2020-11-13T04:12:33Z")
Hi Does anyone know what adaptive replica selection mean? Below was extracted from https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-8.0.html Search and get requests are now routed to s… --- ## [Need suggestion for mapping](https://discuss.elastic.co/t/need-suggestion-for-mapping/255281)
**Author:** [@amanjain](https://discuss.elastic.co/u/amanjain)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 3:03am UTC](https://discuss.elastic.co/t/need-suggestion-for-mapping/255281 "2020-11-13T03:03:12Z")
Hi I am using category, which is a hierarchical field represented as level1\>level2\>level3\>level4 I want to make a query that can apply filter on level2 or level3 individually. So, I am using simple\_pattern\_split tokeni… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=763) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=765)