# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=721 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 722 --- ## [Sane design thougts?](https://discuss.elastic.co/t/sane-design-thougts/261502)
**Author:** [@Andreas\_Falk](https://discuss.elastic.co/u/Andreas_Falk)\ **Replies:** 4\ **Last updated:** [January 20, 2021, 12:41pm UTC](https://discuss.elastic.co/t/sane-design-thougts/261502 "2021-01-20T12:41:56Z")
Hi, This is a x-post from Slack, but I think that it is perhaps easier to have a discussion here. :slight\_smile: We are setting up a new logcluster that should handle .. logs.. :slight\_smile: from our infrastructure mo… --- ## [Fleet agent tls settings](https://discuss.elastic.co/t/fleet-agent-tls-settings/261191)
**Author:** [@gregorys](https://discuss.elastic.co/u/gregorys)\ **Replies:** 8\ **Last updated:** [January 20, 2021, 12:09pm UTC](https://discuss.elastic.co/t/fleet-agent-tls-settings/261191 "2021-01-20T12:09:10Z")
Hi, I installed a fleet agent, its state in Kibana is online and its receiving logs over the UDP port configured in an integration. However I don't see a data stream, looks like it's not sending these logs to elasticse… --- ## [How to set elasticsearch jvm heap size?](https://discuss.elastic.co/t/how-to-set-elasticsearch-jvm-heap-size/261611)
**Author:** [@wangxr1985](https://discuss.elastic.co/u/wangxr1985)\ **Replies:** 10\ **Last updated:** [January 20, 2021, 11:54am UTC](https://discuss.elastic.co/t/how-to-set-elasticsearch-jvm-heap-size/261611 "2021-01-20T11:54:57Z")
If I have a 16C64G machine and install CentOS7.6 on it, There is 62gb available memory. total used free shared buff/cache available Mem: 62 34 0 … --- ## [Why elasticsearch index log size larger than the actual size on the server?](https://discuss.elastic.co/t/why-elasticsearch-index-log-size-larger-than-the-actual-size-on-the-server/261636)
**Author:** [@xax](https://discuss.elastic.co/u/xax)\ **Replies:** 0\ **Last updated:** [January 20, 2021, 11:29am UTC](https://discuss.elastic.co/t/why-elasticsearch-index-log-size-larger-than-the-actual-size-on-the-server/261636 "2021-01-20T11:29:18Z")
If you look picture no.1 my server have 64G and used 39G. Container elasticsearch and logstash use like 2.2G, docker image 3.4G = 5.6 G. Then the volume use like 30.7 G. Meanwhile, if i check my index size at index … --- ## [FiltersAggregation does not use query filter and/or parent aggregation filter](https://discuss.elastic.co/t/filtersaggregation-does-not-use-query-filter-and-or-parent-aggregation-filter/261629)
**Author:** [@panthony](https://discuss.elastic.co/u/panthony)\ **Replies:** 0\ **Last updated:** [January 20, 2021, 10:32am UTC](https://discuss.elastic.co/t/filtersaggregation-does-not-use-query-filter-and-or-parent-aggregation-filter/261629 "2021-01-20T10:32:26Z")
Hi, I'm not sure if the issue was already reported but I couldn't find anything. It appears that the FiltersAggregation does not leverage the parent's aggregation filter and/or the query filter. Using: PUT /logs/\_bul… --- ## [Understand log4j default strategy](https://discuss.elastic.co/t/understand-log4j-default-strategy/261622)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 0\ **Last updated:** [January 20, 2021, 9:04am UTC](https://discuss.elastic.co/t/understand-log4j-default-strategy/261622 "2021-01-20T09:04:30Z")
Hello all ! I'm not sure about my understanding of the default Elasticsearch log4j strategy : ######## Server JSON ############################ appender.rolling.type = RollingFile appender.rolling.name = rolling appen… --- ## [Pipeline processor to convert nested array to documents while reindexing](https://discuss.elastic.co/t/pipeline-processor-to-convert-nested-array-to-documents-while-reindexing/261547)
**Author:** [@darshan\_ambre](https://discuss.elastic.co/u/darshan_ambre)\ **Replies:** 5\ **Last updated:** [January 20, 2021, 8:17am UTC](https://discuss.elastic.co/t/pipeline-processor-to-convert-nested-array-to-documents-while-reindexing/261547 "2021-01-20T08:17:33Z")
Hi Team, Need a solution to take data from source index and have an ingest pipeline to extract data from array in each document of source index as individual document to destination index. e.g. main index data: {… --- ## [Rolling upgrade: How many times should I flush?](https://discuss.elastic.co/t/rolling-upgrade-how-many-times-should-i-flush/260907)
**Author:** [@Dennis1](https://discuss.elastic.co/u/Dennis1)\ **Replies:** 13\ **Last updated:** [January 20, 2021, 7:42am UTC](https://discuss.elastic.co/t/rolling-upgrade-how-many-times-should-i-flush/260907 "2021-01-20T07:42:42Z")
If I follow the steps in the Rolling Upgrade docs in the order they're written, it seems to suggest performing a flush for every node I upgrade in my cluster. Is this really necessary, or would performing the flush once… --- ## [Unable to create primary shard in hot-1 and replica shard in hot-2](https://discuss.elastic.co/t/unable-to-create-primary-shard-in-hot-1-and-replica-shard-in-hot-2/261606)
**Author:** [@aris\_lawrence](https://discuss.elastic.co/u/aris_lawrence)\ **Replies:** 5\ **Last updated:** [January 20, 2021, 4:45am UTC](https://discuss.elastic.co/t/unable-to-create-primary-shard-in-hot-1-and-replica-shard-in-hot-2/261606 "2021-01-20T04:45:17Z")
Elasticsearch version 7.7.1 We have the following setup in our elastic environment. 3 master only 2 ingest only 2 hot nodes (hot-1-p1 and hot-2-p1) 2 warm nodes (warm-1-p1 and warm-2-p1) 1 cold node (cold-p1) mast… --- ## [Get total no. of buckets using composite aggregation](https://discuss.elastic.co/t/get-total-no-of-buckets-using-composite-aggregation/261607)
**Author:** [@shreedhan](https://discuss.elastic.co/u/shreedhan)\ **Replies:** 0\ **Last updated:** [January 20, 2021, 1:25am UTC](https://discuss.elastic.co/t/get-total-no-of-buckets-using-composite-aggregation/261607 "2021-01-20T01:25:37Z")
Is there a way to get total no. of buckets using composite aggregation. I see there's a question asked almost 2 years ago https://discuss.elastic.co/t/composite-aggregation-buckets-total-count/137156/2. Has this changed?… --- ## [How to search with analyzed text](https://discuss.elastic.co/t/how-to-search-with-analyzed-text/261509)
**Author:** [@Myang\_G](https://discuss.elastic.co/u/Myang_G)\ **Replies:** 2\ **Last updated:** [January 20, 2021, 1:19am UTC](https://discuss.elastic.co/t/how-to-search-with-analyzed-text/261509 "2021-01-20T01:19:35Z")
Hi. I'd like to know if there's a way to use analyzed text when querying. The specific requirement is to search for a matched string in 'term' field && the document's 'term\_length' field should be less than equal to t… --- ## [Failed to parse date field, date\_time\_parse\_exception](https://discuss.elastic.co/t/failed-to-parse-date-field-date-time-parse-exception/261492)
**Author:** [@yulaika](https://discuss.elastic.co/u/yulaika)\ **Replies:** 6\ **Last updated:** [January 20, 2021, 1:02am UTC](https://discuss.elastic.co/t/failed-to-parse-date-field-date-time-parse-exception/261492 "2021-01-20T01:02:59Z")
Here is my configuration: "\[log\]\[timestamp\]" =\> "%{\[yearinlog\]}-%{\[monthinlog\]}-%{\[dateinlog\]} %{\[hourinlog\]}:%{\[minuteinlog\]}:%{\[secondinlog\]}" } } date { match =\> \[ "\[log\]\[timestamp\]", "yyyy-MM-dd HH:mm:ss.SSS00… --- ## [\_update\_by\_query column.column](https://discuss.elastic.co/t/update-by-query-column-column/261605)
**Author:** [@jcornejo](https://discuss.elastic.co/u/jcornejo)\ **Replies:** 5\ **Last updated:** [January 20, 2021, 12:41am UTC](https://discuss.elastic.co/t/update-by-query-column-column/261605 "2021-01-20T00:41:12Z")
I have this example mapping. I need to update the field "shirt.color" mapping { "shirts" : { "mappings" : { "properties" : { "shirt" : { "properties" : { "brand" : { "type" : "keyword" }, "color" : { "… --- ## [INITIALIZING shard with REALLOCATED\_REPLICA reason](https://discuss.elastic.co/t/initializing-shard-with-reallocated-replica-reason/261590)
**Author:** [@cmusser](https://discuss.elastic.co/u/cmusser)\ **Replies:** 1\ **Last updated:** [January 19, 2021, 11:03pm UTC](https://discuss.elastic.co/t/initializing-shard-with-reallocated-replica-reason/261590 "2021-01-19T23:03:33Z")
Hi, Our 5 node cluster (this is ES 7.10.1 on Ubuntu 18.04) went yellow last night and when we investigated it this morning, there is shard in the INITIALIZING state, with a "reason" of REALLOCATED\_REPLICA. The shard see… --- ## [Is it possible to convert input to a scaled float with the covert processor of an ingest node?](https://discuss.elastic.co/t/is-it-possible-to-convert-input-to-a-scaled-float-with-the-covert-processor-of-an-ingest-node/261598)
**Author:** [@kristall](https://discuss.elastic.co/u/kristall)\ **Replies:** 0\ **Last updated:** [January 19, 2021, 10:28pm UTC](https://discuss.elastic.co/t/is-it-possible-to-convert-input-to-a-scaled-float-with-the-covert-processor-of-an-ingest-node/261598 "2021-01-19T22:28:17Z")
is it possible to scale a float while converting? --- ## [Elasticsearch 7.7 - all shards failed and task cancelled with channel closed](https://discuss.elastic.co/t/elasticsearch-7-7-all-shards-failed-and-task-cancelled-with-channel-closed/261184)
**Author:** [@jamesp](https://discuss.elastic.co/u/jamesp)\ **Replies:** 5\ **Last updated:** [January 19, 2021, 7:57pm UTC](https://discuss.elastic.co/t/elasticsearch-7-7-all-shards-failed-and-task-cancelled-with-channel-closed/261184 "2021-01-19T19:57:24Z")
Hello all, I set up a 3 node cluster using the included JRE on Windows 2016 servers with 12 GB RAM (java min/max 6GB) which seemed to be running fine without a lot of traffic. After hooking it up to a search GUI I start… --- ## [Circuit Breaking Exception](https://discuss.elastic.co/t/circuit-breaking-exception/259405)
**Author:** [@gorek](https://discuss.elastic.co/u/gorek)\ **Replies:** 10\ **Last updated:** [January 19, 2021, 7:06pm UTC](https://discuss.elastic.co/t/circuit-breaking-exception/259405 "2021-01-19T19:06:20Z")
Hello, we using Elasticsearch for a while and we current have few hundred indices with about 120 millions documents in each index. From some days we often see warnings in elasticsearch logs: org.elasticsearch.common.b… --- ## [Cross Cluster Search horizontal scaling](https://discuss.elastic.co/t/cross-cluster-search-horizontal-scaling/261218)
**Author:** [@kannan\_raj](https://discuss.elastic.co/u/kannan_raj)\ **Replies:** 7\ **Last updated:** [January 19, 2021, 7:02pm UTC](https://discuss.elastic.co/t/cross-cluster-search-horizontal-scaling/261218 "2021-01-19T19:02:16Z")
We have cross cluster search running as a separate cluster with two nodes, and cross cluster search is connected to multiple remote clusters. We have an issue with cross cluster search that all search quires are getting… --- ## [Restrict the kibana user for one machine only](https://discuss.elastic.co/t/restrict-the-kibana-user-for-one-machine-only/261546)
**Author:** [@mrunalini](https://discuss.elastic.co/u/mrunalini)\ **Replies:** 2\ **Last updated:** [January 19, 2021, 3:10pm UTC](https://discuss.elastic.co/t/restrict-the-kibana-user-for-one-machine-only/261546 "2021-01-19T15:10:23Z")
HI , My basic requirement here is to restrict any Kibana user for one machine only ,so that same user can not login from different machine at the same time. is there any setting needs to modify in kibana.yml file or an… --- ## [Cron+curator+logrotate are not work?](https://discuss.elastic.co/t/cron-curator-logrotate-are-not-work/261409)
**Author:** [@Tabriz](https://discuss.elastic.co/u/Tabriz)\ **Replies:** 3\ **Last updated:** [January 19, 2021, 3:05pm UTC](https://discuss.elastic.co/t/cron-curator-logrotate-are-not-work/261409 "2021-01-19T15:05:13Z")
l made a schedule for delete old logs from elasticsearch. The command is that 30 0 \* \* \* curator delete indices --time-unit days --timestring '%Y.%m.%d' --older-than 270 but the command doesn't wokr and even the logrot… --- ## [How can I easily implement pseudo-relevance feedback in a full-text search application?](https://discuss.elastic.co/t/how-can-i-easily-implement-pseudo-relevance-feedback-in-a-full-text-search-application/261564)
**Author:** [@Soumya\_Ranjan\_Sahoo](https://discuss.elastic.co/u/Soumya_Ranjan_Sahoo)\ **Replies:** 0\ **Last updated:** [January 19, 2021, 3:01pm UTC](https://discuss.elastic.co/t/how-can-i-easily-implement-pseudo-relevance-feedback-in-a-full-text-search-application/261564 "2021-01-19T15:01:33Z")
Hi, I want to implement pseudo-relevance feedback, together with a bm25 ranking function, for a full-text search application. Is there any reference that I can look up for implementing this? I have just one field in my … --- ## [Terms Aggregation Partitioning + filter buckets](https://discuss.elastic.co/t/terms-aggregation-partitioning-filter-buckets/261550)
**Author:** [@Nam\_Nguyen](https://discuss.elastic.co/u/Nam_Nguyen)\ **Replies:** 0\ **Last updated:** [January 19, 2021, 1:40pm UTC](https://discuss.elastic.co/t/terms-aggregation-partitioning-filter-buckets/261550 "2021-01-19T13:40:05Z")
Hey :wave:, Context I am currently struggling to create a proper query for the following use case. Let's say I wanted to create a group-by aggregation for e.g. term app:.\*, I would do the following: "terms":… --- ## [Query ES cluster state size](https://discuss.elastic.co/t/query-es-cluster-state-size/261542)
**Author:** [@CanardMoussant](https://discuss.elastic.co/u/CanardMoussant)\ **Replies:** 3\ **Last updated:** [January 19, 2021, 1:36pm UTC](https://discuss.elastic.co/t/query-es-cluster-state-size/261542 "2021-01-19T13:36:37Z")
I would like to get the size of the compressed cluster state (as done in PublishClusterStateAction) using an external API. Is it something possible? I am not calling it from java so I cannot duplicate the code doing the … --- ## [Debugging extremely slow indexing](https://discuss.elastic.co/t/debugging-extremely-slow-indexing/258498)
**Author:** [@beckerr](https://discuss.elastic.co/u/beckerr)\ **Replies:** 38\ **Last updated:** [January 19, 2021, 11:48am UTC](https://discuss.elastic.co/t/debugging-extremely-slow-indexing/258498 "2021-01-19T11:48:32Z")
I have recently "inherited" an ES cluster at work that used to run just fine (since I inherited at least) but recently has been experiencing extreme performance issues around indexing. We have 20 "processor" containers i… --- ## [How to disable limitation of index fields](https://discuss.elastic.co/t/how-to-disable-limitation-of-index-fields/261347)
**Author:** [@liran1](https://discuss.elastic.co/u/liran1)\ **Replies:** 2\ **Last updated:** [January 19, 2021, 11:29am UTC](https://discuss.elastic.co/t/how-to-disable-limitation-of-index-fields/261347 "2021-01-19T11:29:21Z")
In the ElasticSearch docs, I found a way to increase the default index fields parameter (1000), I wonder if there is an option to disable the limitation for good? Thanks :slight\_smile: --- ## [Can't start elasticsearch service after enabling TLS for Detections/Alerts - Permission Denied](https://discuss.elastic.co/t/cant-start-elasticsearch-service-after-enabling-tls-for-detections-alerts-permission-denied/261330)
**Author:** [@yarooski](https://discuss.elastic.co/u/yarooski)\ **Replies:** 2\ **Last updated:** [January 19, 2021, 11:09am UTC](https://discuss.elastic.co/t/cant-start-elasticsearch-service-after-enabling-tls-for-detections-alerts-permission-denied/261330 "2021-01-19T11:09:23Z")
After following these instructions meticulously: How To Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 20.04 And then enabling TLS and setting up user permissions here: Detections prerequisites an… --- ## [Force a node to be the master](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 6\ **Last updated:** [January 19, 2021, 9:26am UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467 "2021-01-19T09:26:23Z")
Just curious, how to fix a broken cluster like this: Via official Helm release, I made a 3 nodes cluster, was working fine: node-0 / node-1 / node-2 I scaled down the STS to 0, to remove all pods/nodes. Then when I s… --- ## [Clearing cache on stack with read only indices](https://discuss.elastic.co/t/clearing-cache-on-stack-with-read-only-indices/261507)
**Author:** [@crs](https://discuss.elastic.co/u/crs)\ **Replies:** 2\ **Last updated:** [January 19, 2021, 9:15am UTC](https://discuss.elastic.co/t/clearing-cache-on-stack-with-read-only-indices/261507 "2021-01-19T09:15:43Z")
Hi, I have system (ES 7.6.2) with many read only indices (index.blocks.read\_only=true). All works fine but when I want to clear cache (curl -X POST "localhost:9200/\_cache/clear") then I'm getting a lot of errors "index … --- ## [Elasticsearch Failed with signal](https://discuss.elastic.co/t/elasticsearch-failed-with-signal/260975)
**Author:** [@Joegio](https://discuss.elastic.co/u/Joegio)\ **Replies:** 4\ **Last updated:** [January 19, 2021, 8:09am UTC](https://discuss.elastic.co/t/elasticsearch-failed-with-signal/260975 "2021-01-19T08:09:15Z")
Good Morning, i'm newbie with elasticsearch. I have installed zammad on the our server with elasticsearch. ES service running but after about one hour has been killed with result 'signal'. I have tried to change confi… --- ## [Join query type in Query DSL](https://discuss.elastic.co/t/join-query-type-in-query-dsl/261423)
**Author:** [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)\ **Replies:** 4\ **Last updated:** [January 19, 2021, 7:45am UTC](https://discuss.elastic.co/t/join-query-type-in-query-dsl/261423 "2021-01-19T07:45:10Z")
Hello, I have created a join field using below query, PUT test\_corr { "mappings": { "properties": { "my\_id": { "type": "keyword" }, "my\_join\_field": { "type": "join", "relations": { "indicators": "indicator" … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=720) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=722)