# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=797 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 798 --- ## [Best way to ensure logs from all hosts are being indexed](https://discuss.elastic.co/t/best-way-to-ensure-logs-from-all-hosts-are-being-indexed/250424)
**Author:** [@bruce289](https://discuss.elastic.co/u/bruce289)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 10:29pm UTC](https://discuss.elastic.co/t/best-way-to-ensure-logs-from-all-hosts-are-being-indexed/250424 "2020-09-29T22:29:50Z")
Hi all, The log pipeline for our Linux systems is: rsyslog =\> logstash =\> elasticsearch We want to put something in place that would alert us if logs from any of our approximately 5000 hosts are not being indexed in E… --- ## [Calculations with values of different documents in painless query](https://discuss.elastic.co/t/calculations-with-values-of-different-documents-in-painless-query/249280)
**Author:** [@alexfritz](https://discuss.elastic.co/u/alexfritz)\ **Replies:** 10\ **Last updated:** [September 29, 2020, 10:15pm UTC](https://discuss.elastic.co/t/calculations-with-values-of-different-documents-in-painless-query/249280 "2020-09-29T22:15:25Z")
Hello I would like to use a painless script to do some calculations. The problem is, the values are stored in different documents as time series. For example the index has a mapping of 2 fields fieldA and fieldB. There … --- ## [Elasticsearch 7.8.0 user authentication with X-Pack security fails](https://discuss.elastic.co/t/elasticsearch-7-8-0-user-authentication-with-x-pack-security-fails/249889)
**Author:** [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Replies:** 10\ **Last updated:** [September 29, 2020, 9:03pm UTC](https://discuss.elastic.co/t/elasticsearch-7-8-0-user-authentication-with-x-pack-security-fails/249889 "2020-09-29T21:03:00Z")
Hello, I'm experiencing user authentication issues for custom user, roles with X-Pack security. I'm using elasticsearch REST API for the user, roles, etc. setup (Not using Kibana). Responses with errors when checking … --- ## [How to curl PUT timestamp correctly](https://discuss.elastic.co/t/how-to-curl-put-timestamp-correctly/250256)
**Author:** [@buhruce](https://discuss.elastic.co/u/buhruce)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 9:16pm UTC](https://discuss.elastic.co/t/how-to-curl-put-timestamp-correctly/250256 "2020-09-29T21:16:55Z")
I admin a Configuration Management tool and it's data visualization is not good. So I'm scraping the data from this tool and dumping it into Elastic Search. Config Tool API -\> Python Script -\> Elast Search API. All the d… --- ## [Problem with dates in Javascript](https://discuss.elastic.co/t/problem-with-dates-in-javascript/249702)
**Author:** [@HockeyFan0000](https://discuss.elastic.co/u/HockeyFan0000)\ **Replies:** 2\ **Last updated:** [September 29, 2020, 6:52pm UTC](https://discuss.elastic.co/t/problem-with-dates-in-javascript/249702 "2020-09-29T18:52:24Z")
I've seen many people having this same issue but I've not found an answer that has resolved my problem. I know this was a bug in ES 7.5 but I'm using ES 6.8. I've got logs with dates/times in the format "09/23 14:16:16… --- ## [Creating ES Node Certs Without CA Key?](https://discuss.elastic.co/t/creating-es-node-certs-without-ca-key/250065)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 4:07pm UTC](https://discuss.elastic.co/t/creating-es-node-certs-without-ca-key/250065 "2020-09-29T16:07:56Z")
Hello, I was wondering if it was possible to create ES node certs using an existing CA without a CA key? The only thing I have found in the documentation is that you can only create ES node certs with existing CA if it … --- ## [Adding warm node to existing cluster, how to avoid existing indices to spread to these nodes](https://discuss.elastic.co/t/adding-warm-node-to-existing-cluster-how-to-avoid-existing-indices-to-spread-to-these-nodes/250239)
**Author:** [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\ **Replies:** 5\ **Last updated:** [September 29, 2020, 4:05pm UTC](https://discuss.elastic.co/t/adding-warm-node-to-existing-cluster-how-to-avoid-existing-indices-to-spread-to-these-nodes/250239 "2020-09-29T16:05:16Z")
I plan to add 3 warm nodes into my cluster. (version 7.9 and 15 hot nodes) I modified elasticsearch.yml with these new lines: node.attr.data: hot # on existing nodes node.attr.data: warm # on new nodes when I s… --- ## [Mapping elasticsearch](https://discuss.elastic.co/t/mapping-elasticsearch/250392)
**Author:** [@Bengagi\_Foued](https://discuss.elastic.co/u/Bengagi_Foued)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 3:48pm UTC](https://discuss.elastic.co/t/mapping-elasticsearch/250392 "2020-09-29T15:48:42Z")
hello is there a way to make elasticsearch store a field (which is not declared in the mapping) by default with type keyword instead of text ? --- ## [Is there a way to validate or dry run the index templates config before applying?](https://discuss.elastic.co/t/is-there-a-way-to-validate-or-dry-run-the-index-templates-config-before-applying/250244)
**Author:** [@ShenWan](https://discuss.elastic.co/u/ShenWan)\ **Replies:** 2\ **Last updated:** [September 29, 2020, 3:07pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-validate-or-dry-run-the-index-templates-config-before-applying/250244 "2020-09-29T15:07:58Z")
We plan to store index templates in Phabricator and would like to validate a diff before landing it and HTTP PUT to ES. --- ## [Kibana maps Sum Aggregation returns 0](https://discuss.elastic.co/t/kibana-maps-sum-aggregation-returns-0/250367)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 0\ **Last updated:** [September 29, 2020, 1:48pm UTC](https://discuss.elastic.co/t/kibana-maps-sum-aggregation-returns-0/250367 "2020-09-29T13:48:18Z")
Hi guys , i'm having a problem with sum aggregation on maps , so when i change the time filter to last 100 days it gives me the exact result but any lesser than that the result would be 0 , though i have data on document… --- ## [Rolling upgrade from ES 6.8 to ES 7.9](https://discuss.elastic.co/t/rolling-upgrade-from-es-6-8-to-es-7-9/250344)
**Author:** [@amishar](https://discuss.elastic.co/u/amishar)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 1:03pm UTC](https://discuss.elastic.co/t/rolling-upgrade-from-es-6-8-to-es-7-9/250344 "2020-09-29T13:03:13Z")
While performing rolling upgrade from 6.8 to 7.9, the documentation suggest that cluster continue to operate in backward compatibility mode until all nodes are upgraded to newer version. How do we avoid issues with newer… --- ## [Reducing Index Footprint](https://discuss.elastic.co/t/reducing-index-footprint/250318)
**Author:** [@shmoolik](https://discuss.elastic.co/u/shmoolik)\ **Replies:** 6\ **Last updated:** [September 29, 2020, 11:57am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318 "2020-09-29T11:57:04Z")
Hi there! My team is looking for ways to optimize large elasticsearch indices by reducing the index footprint. I read this article about Dynamic Mapping and was wondering if it can be used in order to reduce index foot… --- ## [Bucket sort aggregation with actual documents](https://discuss.elastic.co/t/bucket-sort-aggregation-with-actual-documents/249751)
**Author:** [@het](https://discuss.elastic.co/u/het)\ **Replies:** 2\ **Last updated:** [September 29, 2020, 11:27am UTC](https://discuss.elastic.co/t/bucket-sort-aggregation-with-actual-documents/249751 "2020-09-29T11:27:54Z")
Hi team, I wanted to get aggegatated data with pagination. So I have tried to use Bucket-Sort. I have referred this URL. Now, I am able to get aggegatated data, but the problem is, I am able to get the only the key… --- ## [Script score query with field value factor from an array of objects](https://discuss.elastic.co/t/script-score-query-with-field-value-factor-from-an-array-of-objects/250334)
**Author:** [@draganmilchevski](https://discuss.elastic.co/u/draganmilchevski)\ **Replies:** 0\ **Last updated:** [September 29, 2020, 10:22am UTC](https://discuss.elastic.co/t/script-score-query-with-field-value-factor-from-an-array-of-objects/250334 "2020-09-29T10:22:33Z")
Hi, I am trying to score the returned documents of a term query based on a score from the term object that was hit. Let me explain with examples. Suppose this is my index mapping: "documents" : { "mappings" : { … --- ## [Kubernetes logs split in kibana](https://discuss.elastic.co/t/kubernetes-logs-split-in-kibana/250339)
**Author:** [@Snir\_ben\_yosf](https://discuss.elastic.co/u/Snir_ben_yosf)\ **Replies:** 0\ **Last updated:** [September 29, 2020, 10:34am UTC](https://discuss.elastic.co/t/kubernetes-logs-split-in-kibana/250339 "2020-09-29T10:34:35Z")
I have Kubernetes system in Azure/eks/gcp and used the following instrustions to install fluent, elasticsearch and kibana: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch I am ab… --- ## [How to build a terms agg with a high cardinate filed and a low cardinate field](https://discuss.elastic.co/t/how-to-build-a-terms-agg-with-a-high-cardinate-filed-and-a-low-cardinate-field/250305)
**Author:** [@zjd-melo](https://discuss.elastic.co/u/zjd-melo)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 10:05am UTC](https://discuss.elastic.co/t/how-to-build-a-terms-agg-with-a-high-cardinate-filed-and-a-low-cardinate-field/250305 "2020-09-29T10:05:30Z")
Hello, there are two fields in my docs, cid and os. Cid is a high cardinate field, while os either android or ios. I want to know the difference between the following aggs, which is better. GET \_search { "size": 0,… --- ## [Painless script, get error dynamic method \[java.lang.Long, getMillis/0\] not found](https://discuss.elastic.co/t/painless-script-get-error-dynamic-method-java-lang-long-getmillis-0-not-found/250240)
**Author:** [@perezbrandon](https://discuss.elastic.co/u/perezbrandon)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 9:55am UTC](https://discuss.elastic.co/t/painless-script-get-error-dynamic-method-java-lang-long-getmillis-0-not-found/250240 "2020-09-29T09:55:00Z")
So when i run this script, doc\['date\_updated'\].value.getMillis() \> params.one\_day ? 1.9 i get the error dynamic method \[java.lang.Long, getMillis/0\] not found Im not sure what this error means or if im missing somethi… --- ## [Remote Cluster Index handling](https://discuss.elastic.co/t/remote-cluster-index-handling/250323)
**Author:** [@Amrita\_Mahule](https://discuss.elastic.co/u/Amrita_Mahule)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 9:47am UTC](https://discuss.elastic.co/t/remote-cluster-index-handling/250323 "2020-09-29T09:47:55Z")
Hi Team, We have created a new remote cluster(in addition to one existing cluster) and a new index in it. We are trying to create an alias of that newly created index but getting index not found error(We are able to se… --- ## [Readiness probe failed & Liveness probe failed for elastic search](https://discuss.elastic.co/t/readiness-probe-failed-liveness-probe-failed-for-elastic-search/250330)
**Author:** [@tushar1701](https://discuss.elastic.co/u/tushar1701)\ **Replies:** 0\ **Last updated:** [September 29, 2020, 9:45am UTC](https://discuss.elastic.co/t/readiness-probe-failed-liveness-probe-failed-for-elastic-search/250330 "2020-09-29T09:45:53Z")
Readiness probe failed: Waiting for elasticsearch cluster to become ready (request params: "wait\_for\_status=green&timeout=1s" ) Cluster is not yet ready (request params: "wait\_for\_status=green&timeout=1s" ) Warning Un… --- ## [Conditional grok parsing using elasticsearch processor pipeline](https://discuss.elastic.co/t/conditional-grok-parsing-using-elasticsearch-processor-pipeline/249977)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 9:44am UTC](https://discuss.elastic.co/t/conditional-grok-parsing-using-elasticsearch-processor-pipeline/249977 "2020-09-29T09:44:26Z")
We use the default apache error elasticsearch pipeline but, we have the requirement that needs a further parsing. We need an additional grok parsing for message field. How to perform grok parsing when field event\_type i… --- ## [ILM Policy - Help!\>!\>!\>](https://discuss.elastic.co/t/ilm-policy-help/250217)
**Author:** [@Gary\_Wilson1](https://discuss.elastic.co/u/Gary_Wilson1)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 9:34am UTC](https://discuss.elastic.co/t/ilm-policy-help/250217 "2020-09-29T09:34:19Z")
Hello, I have set up an index called filebeat-7-7-1\*, which creates a daily index for my filebeat logs. So far so good - elastic creates a new index daily, which fills up with logs and all the other good stuff I need. H… --- ## [ES regex match failure](https://discuss.elastic.co/t/es-regex-match-failure/250282)
**Author:** [@f4ct0r](https://discuss.elastic.co/u/f4ct0r)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 9:13am UTC](https://discuss.elastic.co/t/es-regex-match-failure/250282 "2020-09-29T09:13:36Z")
Hey guys, I wanna match some data about form-data in ES like ------WebKitFormBoundary9fB75AOSG0bMs1I4\\x0D\\x0AContent-Disposition: form-data; name=\\x22upload\\x22; filename=\\x22shell.php\\x22\\x0D\\x0AContent-Type: applicat… --- ## [Custom metrics from bunch of servers](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209)
**Author:** [@rmrfchik](https://discuss.elastic.co/u/rmrfchik)\ **Replies:** 2\ **Last updated:** [September 29, 2020, 8:36am UTC](https://discuss.elastic.co/t/custom-metrics-from-bunch-of-servers/250209 "2020-09-29T08:36:09Z")
I have a number of different servers (php, java...). All of the servers uses different user tracking cookies and are behind nginx. I already have access.log with all cookies I need and all of them are feeded to ES with … --- ## [How to know the which Elastic node executing the API call with in the Elastic Cluster?](https://discuss.elastic.co/t/how-to-know-the-which-elastic-node-executing-the-api-call-with-in-the-elastic-cluster/250136)
**Author:** [@gsece](https://discuss.elastic.co/u/gsece)\ **Replies:** 4\ **Last updated:** [September 29, 2020, 7:23am UTC](https://discuss.elastic.co/t/how-to-know-the-which-elastic-node-executing-the-api-call-with-in-the-elastic-cluster/250136 "2020-09-29T07:23:08Z")
We have 3 node elastic cluster and all are master eligible nodes. We use bulk or index api to post the document via API. How to identify which elastic node executing the API with in the cluster.. --- ## [Inserting documents are very slow in elastic 6.8.5 server](https://discuss.elastic.co/t/inserting-documents-are-very-slow-in-elastic-6-8-5-server/250255)
**Author:** [@Prabaharan\_Kathiresa](https://discuss.elastic.co/u/Prabaharan_Kathiresa)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 7:22am UTC](https://discuss.elastic.co/t/inserting-documents-are-very-slow-in-elastic-6-8-5-server/250255 "2020-09-29T07:22:23Z")
Hi Team we are seeing slowness when inserting documents in the elastic server. we are using 6.8.5 elastic server and using 6.8.5 RestHighLevelClient. IndexResponse resp = restHighLevelClient.index(indexRequest, Requ… --- ## [Elasticsearch transform index using scripted fields](https://discuss.elastic.co/t/elasticsearch-transform-index-using-scripted-fields/249385)
**Author:** [@geekganesh](https://discuss.elastic.co/u/geekganesh)\ **Replies:** 2\ **Last updated:** [September 29, 2020, 7:12am UTC](https://discuss.elastic.co/t/elasticsearch-transform-index-using-scripted-fields/249385 "2020-09-29T07:12:30Z")
I am using Elasic Cloud v7.5. I created two scripted fields. I am able to search and aggregate on the fields. I tried to transform to new index using the aggregate of scripted fields but the values are coming as zero in … --- ## [RAM/ Heap requirements for Elastic stack](https://discuss.elastic.co/t/ram-heap-requirements-for-elastic-stack/250295)
**Author:** [@Madhu19](https://discuss.elastic.co/u/Madhu19)\ **Replies:** 2\ **Last updated:** [September 29, 2020, 6:52am UTC](https://discuss.elastic.co/t/ram-heap-requirements-for-elastic-stack/250295 "2020-09-29T06:52:12Z")
I have setup ELK for watching logs on Kibana on Linux redhat-6.8 with 16gb RAM and allocated 6GB for Heap. Index size is 1.5gb per day with 7 days retention period. Observed when I query with very simple patterns on Kiba… --- ## [Unable to apply base Elasticsearch kubernetes image](https://discuss.elastic.co/t/unable-to-apply-base-elasticsearch-kubernetes-image/250231)
**Author:** [@jdevilla20](https://discuss.elastic.co/u/jdevilla20)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 5:47am UTC](https://discuss.elastic.co/t/unable-to-apply-base-elasticsearch-kubernetes-image/250231 "2020-09-29T05:47:14Z")
Hi I am trying to apply the Elasticsearch Docker image. I am trying to use the instructions defined here: When I run an "kubectl apply -f" on the yaml, I get an error: error: unable to recognize "test.yaml": no mat… --- ## [Fscrawler does not index to ES with https](https://discuss.elastic.co/t/fscrawler-does-not-index-to-es-with-https/247522)
**Author:** [@Yamini\_Shashank](https://discuss.elastic.co/u/Yamini_Shashank)\ **Replies:** 3\ **Last updated:** [September 29, 2020, 5:29am UTC](https://discuss.elastic.co/t/fscrawler-does-not-index-to-es-with-https/247522 "2020-09-29T05:29:28Z")
The following is the configuration on fscrawler: name: "docscluster" fs: url: "/etc/STORE/" update\_rate: "2s" excludes: - "\*/~\*" json\_support: false filename\_as\_id: true add\_filesize: true remove\_deleted… --- ## [\[Path.data\]](https://discuss.elastic.co/t/path-data/250177)
**Author:** [@Guillaume\_RENARD](https://discuss.elastic.co/u/Guillaume_RENARD)\ **Replies:** 1\ **Last updated:** [September 29, 2020, 3:32am UTC](https://discuss.elastic.co/t/path-data/250177 "2020-09-29T03:32:04Z")
Hello, I have to path.data path.data: \["/datassd/elasticsearch", "/data/elasticsearch"\] but one is quite small and always full, I want ot not use it anymore and to only use one. is it possible to remove it from con… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=796) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=798)