# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=805 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 806 --- ## [Applying ILM Policy but data not getting deleted](https://discuss.elastic.co/t/applying-ilm-policy-but-data-not-getting-deleted/248605)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 9\ **Last updated:** [September 20, 2020, 2:52am UTC](https://discuss.elastic.co/t/applying-ilm-policy-but-data-not-getting-deleted/248605 "2020-09-20T02:52:01Z")
Hello, I am working on configuring an ILM policy to delete any data in an index which is older than 30 days / more than 50GB. I have an index which has data about 97GB at the moment. I have configured an alias for … --- ## [Get data from elasticsearch and visualize it using python](https://discuss.elastic.co/t/get-data-from-elasticsearch-and-visualize-it-using-python/249221)
**Author:** [@atiqah\_yusri](https://discuss.elastic.co/u/atiqah_yusri)\ **Replies:** 4\ **Last updated:** [September 19, 2020, 9:08pm UTC](https://discuss.elastic.co/t/get-data-from-elasticsearch-and-visualize-it-using-python/249221 "2020-09-19T21:08:49Z")
Hi, currently, I have to develop a data analytics dashboard on a web platform. It is possible if I load my datasets to elasticsearch using logstash without using kibana for the visualisation? Because I prefer to use pyth… --- ## [Best web cache](https://discuss.elastic.co/t/best-web-cache/249206)
**Author:** [@highlandspring](https://discuss.elastic.co/u/highlandspring)\ **Replies:** 4\ **Last updated:** [September 19, 2020, 9:06am UTC](https://discuss.elastic.co/t/best-web-cache/249206 "2020-09-19T09:06:09Z")
I am working on a Business Intelligence projects, it has a large number of records, 4.6 million companies and 80 million directors. Currently I am replacing the full text queries with Elastic Search ones. Once this is … --- ## [What is better. Monthly Indices or 1 Index with more shards?](https://discuss.elastic.co/t/what-is-better-monthly-indices-or-1-index-with-more-shards/248982)
**Author:** [@defalt](https://discuss.elastic.co/u/defalt)\ **Replies:** 4\ **Last updated:** [September 19, 2020, 6:09am UTC](https://discuss.elastic.co/t/what-is-better-monthly-indices-or-1-index-with-more-shards/248982 "2020-09-19T06:09:07Z")
Hi, we are currently planning a cluster which takes about 1TB/year worth of data. When using a standard shard size of 5 this would mean that the shards are 250GB. As far as I know shards should only be around 50-70GB at… --- ## [How to authenticate to elastic search from java](https://discuss.elastic.co/t/how-to-authenticate-to-elastic-search-from-java/249165)
**Author:** [@guruthejus](https://discuss.elastic.co/u/guruthejus)\ **Replies:** 2\ **Last updated:** [September 19, 2020, 5:06am UTC](https://discuss.elastic.co/t/how-to-authenticate-to-elastic-search-from-java/249165 "2020-09-19T05:06:59Z")
Hi, I am trying to authenticate to ES in java. But getting malformed host header exception. Someone please help here. final CredentialsProvider credentialsProvider = new BasicCredential… --- ## [Index and query at the same time speed loss](https://discuss.elastic.co/t/index-and-query-at-the-same-time-speed-loss/249191)
**Author:** [@Sayon\_Bhattacharjee](https://discuss.elastic.co/u/Sayon_Bhattacharjee)\ **Replies:** 1\ **Last updated:** [September 19, 2020, 3:22am UTC](https://discuss.elastic.co/t/index-and-query-at-the-same-time-speed-loss/249191 "2020-09-19T03:22:15Z")
I currently have an Elasticsearch index which is receiving 1 document/second (possible that will increase but not over 10/second). However, this index may get queried hundreds of times a second. My question is whether c… --- ## [\_update\_by\_query throws null\_pointer\_exception](https://discuss.elastic.co/t/update-by-query-throws-null-pointer-exception/248574)
**Author:** [@Rui\_Goncalves](https://discuss.elastic.co/u/Rui_Goncalves)\ **Replies:** 2\ **Last updated:** [September 19, 2020, 12:52am UTC](https://discuss.elastic.co/t/update-by-query-throws-null-pointer-exception/248574 "2020-09-19T00:52:44Z")
I've created an index, and I'm trying to add one field. using this script: POST test\_kris\_assignment/\_update\_by\_query { "script": { "source": "ctx.\_source.field1.field2\_1 =10", "lang": "painless" }, "query": { "m… --- ## [I can't cancel task API enrich \_execute](https://discuss.elastic.co/t/i-cant-cancel-task-api-enrich-execute/248681)
**Author:** [@Tornis](https://discuss.elastic.co/u/Tornis)\ **Replies:** 1\ **Last updated:** [September 18, 2020, 9:36pm UTC](https://discuss.elastic.co/t/i-cant-cancel-task-api-enrich-execute/248681 "2020-09-18T21:36:31Z")
Hello guys! I have a problem and I believe it could be a bug! I created an enrichment policy pointing to an alias with 3 indexes. I started the API "\_enrich / policy/checa-fone/\_execute" and the following error is pres… --- ## [Calculate P95 with Transforms](https://discuss.elastic.co/t/calculate-p95-with-transforms/249018)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 1\ **Last updated:** [September 18, 2020, 5:45pm UTC](https://discuss.elastic.co/t/calculate-p95-with-transforms/249018 "2020-09-18T17:45:54Z")
Hi, I'm currently storing time-series data in my ES cluster and would like to understand how I can calculate the daily P95 values for two fields (traffic.in , traffic.out) and store the result in a separate index. I have… --- ## [Why query\_string doesn't honor the usual precedence rules?](https://discuss.elastic.co/t/why-query-string-doesnt-honor-the-usual-precedence-rules/248779)
**Author:** [@android.kc](https://discuss.elastic.co/u/android.kc)\ **Replies:** 3\ **Last updated:** [September 18, 2020, 4:36pm UTC](https://discuss.elastic.co/t/why-query-string-doesnt-honor-the-usual-precedence-rules/248779 "2020-09-18T16:36:25Z")
Can anyone please explain why I got all different results in the following query\_string queries (all are the same except the parentheses) A OR B AND C (A OR B) AND C A OR (B AND C) The index only has the thre… --- ## [Must\_exist](https://discuss.elastic.co/t/must-exist/249009)
**Author:** [@rihad](https://discuss.elastic.co/u/rihad)\ **Replies:** 5\ **Last updated:** [September 18, 2020, 4:26pm UTC](https://discuss.elastic.co/t/must-exist/249009 "2020-09-18T16:26:05Z")
Hi there! How do I use the must\_exist alias attribute that appeared in version 7.9? This is what I try: curl -sS -X POST http://localhost:9200/\_aliases?timeout=15m&master\_timeout=15m -H Content-Type: application/js… --- ## [Creating an index from another index with only certain fields and a different ID than the original index](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 3\ **Last updated:** [September 18, 2020, 4:02pm UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576 "2020-09-18T16:02:25Z")
Is there anyway to create an index from another index but only taking certain fields, not all? The new index would have a new ID as well and include a new alias. I was looking at the reindex API and the clone API as mayb… --- ## [Best practice to search fields not available to most documents](https://discuss.elastic.co/t/best-practice-to-search-fields-not-available-to-most-documents/249040)
**Author:** [@andrew.laraia](https://discuss.elastic.co/u/andrew.laraia)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 3:12pm UTC](https://discuss.elastic.co/t/best-practice-to-search-fields-not-available-to-most-documents/249040 "2020-09-18T15:12:46Z")
I'm new to the ELK stack. So far, so good. I am making progress, We're looking to use on an existing system get some real-time (or at least recent history) of client/user activity to aid in offering the most relevant ser… --- ## [Architecture validation](https://discuss.elastic.co/t/architecture-validation/248940)
**Author:** [@vero](https://discuss.elastic.co/u/vero)\ **Replies:** 3\ **Last updated:** [September 18, 2020, 2:51pm UTC](https://discuss.elastic.co/t/architecture-validation/248940 "2020-09-18T14:51:47Z")
Hi, I'm writing to you, because I need to your help / points of view about the architecture that I choose. In fact, I just started a new project, I'm in preparation architecture stage. Normally, I have, 4 stages: Coll… --- ## [2-node cluster debian](https://discuss.elastic.co/t/2-node-cluster-debian/248834)
**Author:** [@zen.xen](https://discuss.elastic.co/u/zen.xen)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 9:13am UTC](https://discuss.elastic.co/t/2-node-cluster-debian/248834 "2020-09-18T09:13:00Z")
Hello, I try to build 2-node cluster but it doesn't work. I have installed bitnami-elk-7.9.1-0-linux-debian-10-x86\_64-nami.ova , I made two nodes: debian-01: 192.168.11.96, and debian-02: 192.168.11.97 configuration f… --- ## [Date Long Overflow 7.9](https://discuss.elastic.co/t/date-long-overflow-7-9/248488)
**Author:** [@Escalus](https://discuss.elastic.co/u/Escalus)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 9:00am UTC](https://discuss.elastic.co/t/date-long-overflow-7-9/248488 "2020-09-18T09:00:18Z")
Hello, I get a long overflow in my date fields if the time is pior to 1970. The epoch\_time is a negativ long value and throws an arithmetic\_exception on reindex { "index" : "xxx", "type" : "\_doc", "id" : "pub.10… --- ## [Sharding hot vs warm Nodes](https://discuss.elastic.co/t/sharding-hot-vs-warm-nodes/245960)
**Author:** [@fvegaucr1](https://discuss.elastic.co/u/fvegaucr1)\ **Replies:** 7\ **Last updated:** [September 18, 2020, 8:22am UTC](https://discuss.elastic.co/t/sharding-hot-vs-warm-nodes/245960 "2020-09-18T08:22:43Z")
Hi, hope you all doing good, I will like to see if someone can help me with this. I have a cluster of 35 nodes running version 7.4.2 3 x of them are master nodes 2 x coordinators 8 x hot 2 x cold 20 x Warm We get … --- ## [High CPU usage on only 1 Data node](https://discuss.elastic.co/t/high-cpu-usage-on-only-1-data-node/248407)
**Author:** [@fgourichon](https://discuss.elastic.co/u/fgourichon)\ **Replies:** 6\ **Last updated:** [September 18, 2020, 7:54am UTC](https://discuss.elastic.co/t/high-cpu-usage-on-only-1-data-node/248407 "2020-09-18T07:54:26Z")
Hi there, I'm trying to understand why we sometimes get high CPU utilisation on only one Data node of our cluster. We mainly use 2 indexes (80GB for one, 500MB for the second) with 5 shards and one replica each which se… --- ## [Search performance loss in Elasticsearch 7](https://discuss.elastic.co/t/search-performance-loss-in-elasticsearch-7/248871)
**Author:** [@agutierrezrodriguez](https://discuss.elastic.co/u/agutierrezrodriguez)\ **Replies:** 8\ **Last updated:** [September 18, 2020, 7:33am UTC](https://discuss.elastic.co/t/search-performance-loss-in-elasticsearch-7/248871 "2020-09-18T07:33:08Z")
Hi! We are upgrading our Elasticsearch 5.6 clusters to Elasticsearch 7.6. We're detecting that the search performance is worst in Elasticsearch 7.6 than 5.6. We are testing empty clusters with the same installation pr… --- ## [Audit logging configuration via elasticsearch.yml](https://discuss.elastic.co/t/audit-logging-configuration-via-elasticsearch-yml/248357)
**Author:** [@itokai](https://discuss.elastic.co/u/itokai)\ **Replies:** 1\ **Last updated:** [September 18, 2020, 7:31am UTC](https://discuss.elastic.co/t/audit-logging-configuration-via-elasticsearch-yml/248357 "2020-09-18T07:31:08Z")
Hi, started to configure amount of logging recently and figured out how to do it in log4j2.properties, like below basic example for testing: ######## Server JSON ############################ appender.rolling.t… --- ## [High Search Rate spikes for AWS elasticsearch](https://discuss.elastic.co/t/high-search-rate-spikes-for-aws-elasticsearch/249023)
**Author:** [@mehak](https://discuss.elastic.co/u/mehak)\ **Replies:** 6\ **Last updated:** [September 18, 2020, 7:14am UTC](https://discuss.elastic.co/t/high-search-rate-spikes-for-aws-elasticsearch/249023 "2020-09-18T07:14:23Z")
Hi Team, We are seeing high SearchRate spike at a certain times in a day along with Index and Search latency. We are having a cluster of 6 nodes with 3 master ones and other 3 are data nodes with 200 GB EBS volume. Not … --- ## [Question about Snapshot and Restore](https://discuss.elastic.co/t/question-about-snapshot-and-restore/249041)
**Author:** [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 5:48am UTC](https://discuss.elastic.co/t/question-about-snapshot-and-restore/249041 "2020-09-18T05:48:31Z")
Suppose we have two cluster : A and B Data from A is getting synced to B every 12 hour. So incremental snapshots from A are getting restored on B. Suppose on a day, at first 12 hr, index1 was created with 10 docs. Now … --- ## [Elasticsearch: Aggregations request timezone field not working](https://discuss.elastic.co/t/elasticsearch-aggregations-request-timezone-field-not-working/249061)
**Author:** [@yunut](https://discuss.elastic.co/u/yunut)\ **Replies:** 0\ **Last updated:** [September 18, 2020, 5:46am UTC](https://discuss.elastic.co/t/elasticsearch-aggregations-request-timezone-field-not-working/249061 "2020-09-18T05:46:20Z")
Elasticsearch Version: 7.7.0v Rest client Vesion: 7.7.0v The query created by Postman and the request created by QueryBuilder are the same, but the TimeZone field is not applied to java requests. here is request targe… --- ## [How & When to process/convert data in ELK](https://discuss.elastic.co/t/how-when-to-process-convert-data-in-elk/248816)
**Author:** [@aspat](https://discuss.elastic.co/u/aspat)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 5:22am UTC](https://discuss.elastic.co/t/how-when-to-process-convert-data-in-elk/248816 "2020-09-18T05:22:37Z")
Hello, I am planning to use ELK My current SQL table is like below Time, Field ID, Filed\_name, Field\_Sub\_ID, value, extra value, type To view this data we have provided excel add-in. User can provide basic filter to… --- ## [Daily/hourly aggregation using transform](https://discuss.elastic.co/t/daily-hourly-aggregation-using-transform/248677)
**Author:** [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Replies:** 11\ **Last updated:** [September 18, 2020, 5:15am UTC](https://discuss.elastic.co/t/daily-hourly-aggregation-using-transform/248677 "2020-09-18T05:15:47Z")
Hello I want to have a daily aggregation index with daily avg, percentiles etc . I've created a daily transform with: (more complex than below) "pivot": { "group\_by": { "datetime": { "date\_histogram… --- ## [ELastic upgrade](https://discuss.elastic.co/t/elastic-upgrade/248768)
**Author:** [@chandra\_bera](https://discuss.elastic.co/u/chandra_bera)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 5:07am UTC](https://discuss.elastic.co/t/elastic-upgrade/248768 "2020-09-18T05:07:02Z")
Is there any video or documents to upgrade the Elastic stack 7.5 to 7.9 --- ## [How to Round up the result to the nearest integer](https://discuss.elastic.co/t/how-to-round-up-the-result-to-the-nearest-integer/248897)
**Author:** [@Karrie\_Koo](https://discuss.elastic.co/u/Karrie_Koo)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 2:01am UTC](https://discuss.elastic.co/t/how-to-round-up-the-result-to-the-nearest-integer/248897 "2020-09-18T02:01:50Z")
For the log text, {{heapUsedPct.0}} \<= This field shows the result with many decimal places, I want to round up to the nearest integer, but how can I do it? { "trigger": { "schedule": { "interval": "15m" } }, "in… --- ## [My first project ... beginner question](https://discuss.elastic.co/t/my-first-project-beginner-question/249000)
**Author:** [@leostereo](https://discuss.elastic.co/u/leostereo)\ **Replies:** 3\ **Last updated:** [September 18, 2020, 1:00am UTC](https://discuss.elastic.co/t/my-first-project-beginner-question/249000 "2020-09-18T01:00:26Z")
Hi guys , here I am learning elk. I managed to: 1 Create an index. PUT /dhcp\_leases { "settings": { "number\_of\_shards": 1 }, "mappings": { "properties": { "router": { "type": "text" }, "curre… --- ## [Visualize benchmark summary report in kibana](https://discuss.elastic.co/t/visualize-benchmark-summary-report-in-kibana/249015)
**Author:** [@sammi1](https://discuss.elastic.co/u/sammi1)\ **Replies:** 1\ **Last updated:** [September 17, 2020, 11:51pm UTC](https://discuss.elastic.co/t/visualize-benchmark-summary-report-in-kibana/249015 "2020-09-17T23:51:28Z")
Can we create a sample to visualize summary report in kibana? --- ## [Increasing index.max\_result\_window using CentOS CLI](https://discuss.elastic.co/t/increasing-index-max-result-window-using-centos-cli/246432)
**Author:** [@hishan42](https://discuss.elastic.co/u/hishan42)\ **Replies:** 11\ **Last updated:** [September 17, 2020, 11:28pm UTC](https://discuss.elastic.co/t/increasing-index-max-result-window-using-centos-cli/246432 "2020-09-17T23:28:13Z")
I am using graylog with mongodb and elasticsearch 6.8 on CentOS 7. Now I have received an error saying "Elasticsearch limits the search result to 10000 messages". How do I increase the index.max\_result\_window? I dont … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=804) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=806)