# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=801 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 802 --- ## [Elasticsearch support for ceph storage?](https://discuss.elastic.co/t/elasticsearch-support-for-ceph-storage/249740)
**Author:** [@sarthaksahu](https://discuss.elastic.co/u/sarthaksahu)\ **Replies:** 2\ **Last updated:** [September 24, 2020, 6:37am UTC](https://discuss.elastic.co/t/elasticsearch-support-for-ceph-storage/249740 "2020-09-24T06:37:47Z")
I do saw a blog discussing about support of ceph storage. But the information a year old. Can you please confirm is ceph storage support in ES 7.8.0. --- ## [Merging documents based on matched fields values](https://discuss.elastic.co/t/merging-documents-based-on-matched-fields-values/249728)
**Author:** [@abregman](https://discuss.elastic.co/u/abregman)\ **Replies:** 1\ **Last updated:** [September 24, 2020, 6:24am UTC](https://discuss.elastic.co/t/merging-documents-based-on-matched-fields-values/249728 "2020-09-24T06:24:17Z")
I'll start by saying that I'm not sure what I need is actually to merge documents, it's simply the most suitable subject I've found. Every document created in my environment today has the following fields: job\_name and … --- ## [Watcher, Transform UTC timestamp on the payload to local time](https://discuss.elastic.co/t/watcher-transform-utc-timestamp-on-the-payload-to-local-time/249724)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 3\ **Last updated:** [September 24, 2020, 6:08am UTC](https://discuss.elastic.co/t/watcher-transform-utc-timestamp-on-the-payload-to-local-time/249724 "2020-09-24T06:08:08Z")
In the payload of a watcher I have the timestamp field "@timestamp" : "2020-09-23T23:11:24.359Z", But this field is in UTC, and I want to change it to local time, I have tried the folowing "actions": { "send\_ema… --- ## [What is system requirement for elasticsearch (elk setup)](https://discuss.elastic.co/t/what-is-system-requirement-for-elasticsearch-elk-setup/248807)
**Author:** [@aspat](https://discuss.elastic.co/u/aspat)\ **Replies:** 4\ **Last updated:** [September 24, 2020, 5:33am UTC](https://discuss.elastic.co/t/what-is-system-requirement-for-elasticsearch-elk-setup/248807 "2020-09-24T05:33:12Z")
Hello, I am new to ELK. I have to provide architectural solution for archival system in our project. How our current system is, We have a service/software which generate site data(Oil industry data), this data is ins… --- ## [Circuit Breaker in Visualization](https://discuss.elastic.co/t/circuit-breaker-in-visualization/249307)
**Author:** [@umay\_fb](https://discuss.elastic.co/u/umay_fb)\ **Replies:** 6\ **Last updated:** [September 24, 2020, 4:53am UTC](https://discuss.elastic.co/t/circuit-breaker-in-visualization/249307 "2020-09-24T04:53:03Z")
i want to get a visualization table, but i get a error and this response. reason": { "type": "circuit\_breaking\_exception", "reason": "\[parent\] Data too large, data for \[\\] would be \[66133455312/61.5gb\], which… --- ## [Can't start elasticsearch. please help](https://discuss.elastic.co/t/cant-start-elasticsearch-please-help/249720)
**Author:** [@Tomz](https://discuss.elastic.co/u/Tomz)\ **Replies:** 16\ **Last updated:** [September 24, 2020, 3:56am UTC](https://discuss.elastic.co/t/cant-start-elasticsearch-please-help/249720 "2020-09-24T03:56:45Z")
when I try run "sudo systemctl start elasticsearch" show me Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for … --- ## [Document timestamp field 7.8](https://discuss.elastic.co/t/document-timestamp-field-7-8/249732)
**Author:** [@Dotnetrockstar](https://discuss.elastic.co/u/Dotnetrockstar)\ **Replies:** 2\ **Last updated:** [September 24, 2020, 3:24am UTC](https://discuss.elastic.co/t/document-timestamp-field-7-8/249732 "2020-09-24T03:24:44Z")
Is it possible to add a timestamp field to each document when created or last updated? Elastic version 7.8. I need to know last time document was updated. --- ## [How to normalize similarity score using 'more\_like\_this'](https://discuss.elastic.co/t/how-to-normalize-similarity-score-using-more-like-this/249721)
**Author:** [@Sangmin\_Lee](https://discuss.elastic.co/u/Sangmin_Lee)\ **Replies:** 0\ **Last updated:** [September 24, 2020, 12:54am UTC](https://discuss.elastic.co/t/how-to-normalize-similarity-score-using-more-like-this/249721 "2020-09-24T00:54:11Z")
Hi. I'm using 'more like this' to get the similarity of the given text. As you know, the score is not normalized. I want to get a normalized value \[0, 1\] so that I can cut out documents below the threshold score. I t… --- ## [Manually move replica shard to another node](https://discuss.elastic.co/t/manually-move-replica-shard-to-another-node/249554)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 6\ **Last updated:** [September 23, 2020, 5:08pm UTC](https://discuss.elastic.co/t/manually-move-replica-shard-to-another-node/249554 "2020-09-23T17:08:51Z")
I have an unassigned replica shard that I need to assign to a node but the only node available that has "room" already has the same primary shard assigned to it. My setup: 4 node cluster with 1 replica. I only have an i… --- ## [Automated Archival of data in indices](https://discuss.elastic.co/t/automated-archival-of-data-in-indices/249707)
**Author:** [@ted1621](https://discuss.elastic.co/u/ted1621)\ **Replies:** 3\ **Last updated:** [September 23, 2020, 10:10pm UTC](https://discuss.elastic.co/t/automated-archival-of-data-in-indices/249707 "2020-09-23T22:10:21Z")
Can anyone provide instructions on how I would set up a automated task to archive and move to cold storage any data in my indices that is over 45 days old. Also in regards to indices that are no longer receiving data st… --- ## [Invalid Certificate Errors when setting up SSL/HTTPS for Elasticsearch in Docker](https://discuss.elastic.co/t/invalid-certificate-errors-when-setting-up-ssl-https-for-elasticsearch-in-docker/249713)
**Author:** [@encompassJeff](https://discuss.elastic.co/u/encompassJeff)\ **Replies:** 0\ **Last updated:** [September 23, 2020, 9:45pm UTC](https://discuss.elastic.co/t/invalid-certificate-errors-when-setting-up-ssl-https-for-elasticsearch-in-docker/249713 "2020-09-23T21:45:00Z")
Hi there! Some basic background: My client has recently requested that we transition to using ELK stack to power our search and metrics. We are running Laravel 5.8/PHP 7.3 on the backend (feeding a ReactJS front-end via… --- ## [Only query new documents from Elasticsearch index](https://discuss.elastic.co/t/only-query-new-documents-from-elasticsearch-index/249703)
**Author:** [@mussa572](https://discuss.elastic.co/u/mussa572)\ **Replies:** 2\ **Last updated:** [September 23, 2020, 9:49pm UTC](https://discuss.elastic.co/t/only-query-new-documents-from-elasticsearch-index/249703 "2020-09-23T21:49:46Z")
I am writing a python program which will query elasticsearch index . I am using a match all query in the while loop which looks for the new documents . First problem is that I always get all the documents every time I q… --- ## [Performance when inserting/deleting many documents](https://discuss.elastic.co/t/performance-when-inserting-deleting-many-documents/249676)
**Author:** [@Roman\_Kagan](https://discuss.elastic.co/u/Roman_Kagan)\ **Replies:** 1\ **Last updated:** [September 23, 2020, 9:47pm UTC](https://discuss.elastic.co/t/performance-when-inserting-deleting-many-documents/249676 "2020-09-23T21:47:36Z")
Hello: We have a very old Elastic 2.2.0 instance running in production that contains approximately 26 million documents. We also insert 230k (yes - thousands) of documents a week and mark for deletion 30k documents a … --- ## [Number of hours in an interval of date](https://discuss.elastic.co/t/number-of-hours-in-an-interval-of-date/249683)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 0\ **Last updated:** [September 23, 2020, 4:11pm UTC](https://discuss.elastic.co/t/number-of-hours-in-an-interval-of-date/249683 "2020-09-23T16:11:03Z")
Hi guys , i want to create it a scripted field where i count the number of hours in an interval of date when the sum of the number of transactions is below a certain value, my fields are : nbre\_of\_transaction , date an… --- ## [Unable to create visualization from command prompt in Kibana 6.8](https://discuss.elastic.co/t/unable-to-create-visualization-from-command-prompt-in-kibana-6-8/249706)
**Author:** [@rajkumar.m](https://discuss.elastic.co/u/rajkumar.m)\ **Replies:** 0\ **Last updated:** [September 23, 2020, 8:37pm UTC](https://discuss.elastic.co/t/unable-to-create-visualization-from-command-prompt-in-kibana-6-8/249706 "2020-09-23T20:37:26Z")
Hi Team, I am unable to create visualization from command using curl in ELK 6.8. I have a working solution which I followed from the below link and it is working in ELK5. As per the solution shared, it is going to w… --- ## [Curator filtertype:alias matching an alias from the list](https://discuss.elastic.co/t/curator-filtertype-alias-matching-an-alias-from-the-list/249694)
**Author:** [@ebru](https://discuss.elastic.co/u/ebru)\ **Replies:** 0\ **Last updated:** [September 23, 2020, 6:47pm UTC](https://discuss.elastic.co/t/curator-filtertype-alias-matching-an-alias-from-the-list/249694 "2020-09-23T18:47:35Z")
Hi all, I have a few delete indices actions as part of curator cleanup but I am having trouble with this one. action: delete\_indices description: \>- BIG INDICES - Keep at least 3 indices at all times. Exclude indic… --- ## [Transforms - How to check next run time](https://discuss.elastic.co/t/transforms-how-to-check-next-run-time/249150)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 20\ **Last updated:** [September 23, 2020, 6:34pm UTC](https://discuss.elastic.co/t/transforms-how-to-check-next-run-time/249150 "2020-09-23T18:34:56Z")
Hi, I created a continuous Transforms to run once a day and calculate the P95 of some fields for the previous 24hrs. When I initially created and started this Transform it created the daily P95 values from all the previo… --- ## [Elasticsearch 7.8 worse heap management](https://discuss.elastic.co/t/elasticsearch-7-8-worse-heap-management/240996)
**Author:** [@Carlos\_Moya](https://discuss.elastic.co/u/Carlos_Moya)\ **Replies:** 39\ **Last updated:** [September 23, 2020, 5:50pm UTC](https://discuss.elastic.co/t/elasticsearch-7-8-worse-heap-management/240996 "2020-09-23T17:50:26Z")
Hello, We have an Elasticsearch cluster of 94 servers and last Wednesday we updated it from 7.6.2 to 7.8.0. On Friday one of our teams reported that a nightly process reduced its performance dramatically. We verified i… --- ## [Is there a way to choose the other master eligible node as master automatically if the elected master node goes down (in unexpected sceanrio's)?](https://discuss.elastic.co/t/is-there-a-way-to-choose-the-other-master-eligible-node-as-master-automatically-if-the-elected-master-node-goes-down-in-unexpected-sceanrios/249607)
**Author:** [@Elango\_P](https://discuss.elastic.co/u/Elango_P)\ **Replies:** 7\ **Last updated:** [September 23, 2020, 4:17pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-choose-the-other-master-eligible-node-as-master-automatically-if-the-elected-master-node-goes-down-in-unexpected-sceanrios/249607 "2020-09-23T16:17:56Z")
Hi, We are facing some issues while upgrading ES from 5.5.2 -\> 7.8.1. We have two nodes deployment setup. ES is installed in both nodes and both are master eligible. In earlier version we used to have two master nodes… --- ## [Generate a hash/CRC for part of a document?](https://discuss.elastic.co/t/generate-a-hash-crc-for-part-of-a-document/249641)
**Author:** [@HybridBanana](https://discuss.elastic.co/u/HybridBanana)\ **Replies:** 1\ **Last updated:** [September 23, 2020, 12:29pm UTC](https://discuss.elastic.co/t/generate-a-hash-crc-for-part-of-a-document/249641 "2020-09-23T12:29:41Z")
I need to generate a unique (or almost-certainly-unique) hash/CRC value for part of my ElasticSearch document. The sub-document being hashed can contain arbitrary data types, nested documents, and lists where order is u… --- ## [Multiple bucket filter](https://discuss.elastic.co/t/multiple-bucket-filter/249475)
**Author:** [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\ **Replies:** 3\ **Last updated:** [September 23, 2020, 12:22pm UTC](https://discuss.elastic.co/t/multiple-bucket-filter/249475 "2020-09-23T12:22:18Z")
Hi, Is there a way to use multiple conditions in a stream filter? I tried : transform": { "script": { "source": """ def docs = \[\]; def removes=ctx.payload.aggregations.event\_types.buc… --- ## [Find string in array](https://discuss.elastic.co/t/find-string-in-array/249617)
**Author:** [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\ **Replies:** 2\ **Last updated:** [September 23, 2020, 12:13pm UTC](https://discuss.elastic.co/t/find-string-in-array/249617 "2020-09-23T12:13:29Z")
Hi, I have an array that contains some bucket keys. i want to do something for each bucket if their key is in my array. here is my code so far: "script": { "source": """ def docs = \[\]; d… --- ## [Elastic Search API queries for index logs](https://discuss.elastic.co/t/elastic-search-api-queries-for-index-logs/249635)
**Author:** [@Anand\_219](https://discuss.elastic.co/u/Anand_219)\ **Replies:** 3\ **Last updated:** [September 23, 2020, 11:04am UTC](https://discuss.elastic.co/t/elastic-search-api-queries-for-index-logs/249635 "2020-09-23T11:04:29Z")
We are using EFK as log management for our application. I am playing with API of elastic search, while doing this I have few queries to get clarify. localhost:9200/\_cat/indices?bytes=b&s=store.size:desc&v getting outpu… --- ## [How to add criteria dynamically while searching against elasticsearch](https://discuss.elastic.co/t/how-to-add-criteria-dynamically-while-searching-against-elasticsearch/248333)
**Author:** [@ocmvin](https://discuss.elastic.co/u/ocmvin)\ **Replies:** 3\ **Last updated:** [September 23, 2020, 10:50am UTC](https://discuss.elastic.co/t/how-to-add-criteria-dynamically-while-searching-against-elasticsearch/248333 "2020-09-23T10:50:32Z")
I have a big document with a requirement to dynamically search E.g. Student { id, name, Address, Associated\_Subjects phone, home\_phone, mobile, year, dob, geolocation:(lat/long) height . . . etc } Ho… --- ## [Exclude data from certain interval from Machine Learning](https://discuss.elastic.co/t/exclude-data-from-certain-interval-from-machine-learning/249586)
**Author:** [@heric](https://discuss.elastic.co/u/heric)\ **Replies:** 2\ **Last updated:** [September 23, 2020, 10:43am UTC](https://discuss.elastic.co/t/exclude-data-from-certain-interval-from-machine-learning/249586 "2020-09-23T10:43:48Z")
Hi All, I have collected data to be used for machine learning for more than 2 weeks and accidentally i ruined my data collection for 1 interval, it is getting huge jump in value for this interval. now machine learning … --- ## [Enable anonymous access via api/env variables in elasticsearch with xpack enabled](https://discuss.elastic.co/t/enable-anonymous-access-via-api-env-variables-in-elasticsearch-with-xpack-enabled/248490)
**Author:** [@mrraj470](https://discuss.elastic.co/u/mrraj470)\ **Replies:** 1\ **Last updated:** [September 23, 2020, 9:38am UTC](https://discuss.elastic.co/t/enable-anonymous-access-via-api-env-variables-in-elasticsearch-with-xpack-enabled/248490 "2020-09-23T09:38:10Z")
I want to setup my elasticsearch in such a way that it is readable by everyone (without authentication header) but write access needs authentication. Of course, I can use anonymous user, but I don't find any article show… --- ## [Elastic does not index some of the data](https://discuss.elastic.co/t/elastic-does-not-index-some-of-the-data/249459)
**Author:** [@Orly\_Knop](https://discuss.elastic.co/u/Orly_Knop)\ **Replies:** 10\ **Last updated:** [September 23, 2020, 9:13am UTC](https://discuss.elastic.co/t/elastic-does-not-index-some-of-the-data/249459 "2020-09-23T09:13:48Z")
I have set up indexing of 3 types of data: items, single pages and files. All of them are represented as objects with properties. This is some examples of the properties: For some reason, if I include file content pro… --- ## [Failover mechanism of master nodes](https://discuss.elastic.co/t/failover-mechanism-of-master-nodes/249504)
**Author:** [@Shubham\_Mittal](https://discuss.elastic.co/u/Shubham_Mittal)\ **Replies:** 3\ **Last updated:** [September 23, 2020, 9:04am UTC](https://discuss.elastic.co/t/failover-mechanism-of-master-nodes/249504 "2020-09-23T09:04:14Z")
I have one web application in which IP address of one the 3 master eligible nodes is configured for using elasticsearch. Now if this configured node goes down, and my web app will hit the same IP address, will the reque… --- ## [Huge Data Transfer between Elasticsearch Nodes](https://discuss.elastic.co/t/huge-data-transfer-between-elasticsearch-nodes/248933)
**Author:** [@ajesh](https://discuss.elastic.co/u/ajesh)\ **Replies:** 8\ **Last updated:** [September 23, 2020, 9:01am UTC](https://discuss.elastic.co/t/huge-data-transfer-between-elasticsearch-nodes/248933 "2020-09-23T09:01:09Z")
Hi Team, We are running a 4 node cluster as below. Data Node 1 (hot data, master,ingest) Data Node 2 (hot data, master,ingest) Data Node 3 (warm data, voting-only master) Data Node 4 (warm data) We are seeing huge … --- ## [Default index routing allocation](https://discuss.elastic.co/t/default-index-routing-allocation/249621)
**Author:** [@Mathmarc](https://discuss.elastic.co/u/Mathmarc)\ **Replies:** 1\ **Last updated:** [September 23, 2020, 8:53am UTC](https://discuss.elastic.co/t/default-index-routing-allocation/249621 "2020-09-23T08:53:05Z")
Does there is a way to force all new index, to be route to some specific nodes ? Lets say I have 8 nodes with: 6 nodes with "node.attr.group": "data" 2 nodes without group For now every times a new index, is created… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=800) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=802)