# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=230 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 231 --- ## [Periodic disconnection of same data nodes](https://discuss.elastic.co/t/periodic-disconnection-of-same-data-nodes/338197)
**Author:** [@alissan](https://discuss.elastic.co/u/alissan)\ **Replies:** 5\ **Last updated:** [July 12, 2023, 3:01pm UTC](https://discuss.elastic.co/t/periodic-disconnection-of-same-data-nodes/338197 "2023-07-12T15:01:53Z")
Hello, I have a cluster with 3 master, 40 data nodes (d1,d2,...,d40). First 5 data nodes have voting only master role. Only the following data nodes have periodic abnormal behavior: d11,d12,d13,d14,d15,d16,d17,d21,d2… --- ## [Select Timeout parameter for python helper async\_bulk](https://discuss.elastic.co/t/select-timeout-parameter-for-python-helper-async-bulk/338037)
**Author:** [@ionFreeman](https://discuss.elastic.co/u/ionFreeman)\ **Replies:** 1\ **Last updated:** [July 12, 2023, 2:53pm UTC](https://discuss.elastic.co/t/select-timeout-parameter-for-python-helper-async-bulk/338037 "2023-07-12T14:53:22Z")
Hello! Every so often, my async\_bulk load fails with a Connection Timeout. I have my timeout parameter set to 60; I had set it arbitrarily high, but it didn't pass code review. I can't just wrap the call in tenacity as I… --- ## [I want to get last record saved to Elasticsearch](https://discuss.elastic.co/t/i-want-to-get-last-record-saved-to-elasticsearch/338235)
**Author:** [@Valerie\_Barbacion](https://discuss.elastic.co/u/Valerie_Barbacion)\ **Replies:** 0\ **Last updated:** [July 12, 2023, 2:32pm UTC](https://discuss.elastic.co/t/i-want-to-get-last-record-saved-to-elasticsearch/338235 "2023-07-12T14:32:51Z")
Hi Pals, I have a question about throwing request to Elasticsearch. For example given I will going to send a first message to Elasticsearch with has a field value "Sample" and then I send again another message with diffe… --- ## [Data storage stragety](https://discuss.elastic.co/t/data-storage-stragety/338159)
**Author:** [@JasonREC](https://discuss.elastic.co/u/JasonREC)\ **Replies:** 3\ **Last updated:** [July 12, 2023, 12:04pm UTC](https://discuss.elastic.co/t/data-storage-stragety/338159 "2023-07-12T12:04:50Z")
Hi, I wonder if I already have couple indices that contains a big size of data. I wonder if using snapshot or best compression are good ways to help reduding the size of the existing indices. I have looked at the docu… --- ## [Issues with pushing packages to my own package registry](https://discuss.elastic.co/t/issues-with-pushing-packages-to-my-own-package-registry/338209)
**Author:** [@hari\_ibm](https://discuss.elastic.co/u/hari_ibm)\ **Replies:** 0\ **Last updated:** [July 12, 2023, 11:40am UTC](https://discuss.elastic.co/t/issues-with-pushing-packages-to-my-own-package-registry/338209 "2023-07-12T11:40:24Z")
I have created new package and i want to push it to my custom hosted package registry? How to update the packages list in my custom hosted package registry? --- ## [How to upgrade metricbeat from 7.17.11 to 7.17.xx or 8.1.xx?](https://discuss.elastic.co/t/how-to-upgrade-metricbeat-from-7-17-11-to-7-17-xx-or-8-1-xx/338183)
**Author:** [@Swathi12](https://discuss.elastic.co/u/Swathi12)\ **Replies:** 2\ **Last updated:** [July 12, 2023, 11:13am UTC](https://discuss.elastic.co/t/how-to-upgrade-metricbeat-from-7-17-11-to-7-17-xx-or-8-1-xx/338183 "2023-07-12T11:13:59Z")
How do i upgrade next time from 7.17.11 to 7.17.xx ? or 8.x.x Is there any command which i can use in the Kibana DEV Tool ? Or how is the possible and easy way to do it ? --- ## [How Elastic APM estimate SQL duraion](https://discuss.elastic.co/t/how-elastic-apm-estimate-sql-duraion/338198)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 1\ **Last updated:** [July 12, 2023, 10:04am UTC](https://discuss.elastic.co/t/how-elastic-apm-estimate-sql-duraion/338198 "2023-07-12T10:04:32Z")
Hi I ran Elastic APM agent with my Application that use jdbc to connect to database. Now I compare top sql duration of Elastic APM agent and database log with below query on kibana: service.target.type : "sqli" and s… --- ## [Help with query please](https://discuss.elastic.co/t/help-with-query-please/338191)
**Author:** [@lakhr034](https://discuss.elastic.co/u/lakhr034)\ **Replies:** 0\ **Last updated:** [July 12, 2023, 8:59am UTC](https://discuss.elastic.co/t/help-with-query-please/338191 "2023-07-12T08:59:04Z")
POST user\_info,user\_auth\_cards\_info/\_search { "size": 0, "query": { "bool": { "filter": \[ { "multi\_match": { "query": "test", "fields": \[ "e\_name.auto… --- ## [Provide values to the ctx.vars variables in the Painless file by extracting them from the YAML file](https://discuss.elastic.co/t/provide-values-to-the-ctx-vars-variables-in-the-painless-file-by-extracting-them-from-the-yaml-file/338172)
**Author:** [@Hardik\_Dave](https://discuss.elastic.co/u/Hardik_Dave)\ **Replies:** 0\ **Last updated:** [July 12, 2023, 5:37am UTC](https://discuss.elastic.co/t/provide-values-to-the-ctx-vars-variables-in-the-painless-file-by-extracting-them-from-the-yaml-file/338172 "2023-07-12T05:37:24Z")
I have a .painless file as mentioned below. As of now, the ctx.vars.var1 have hardcoded values in my original file, which now need to be retrieved from a YAML file. All these files are in same project, so relative path w… --- ## [Forwarding logs from Sun Solaris to ELK](https://discuss.elastic.co/t/forwarding-logs-from-sun-solaris-to-elk/338147)
**Author:** [@DKalin0789e](https://discuss.elastic.co/u/DKalin0789e)\ **Replies:** 6\ **Last updated:** [July 11, 2023, 9:04pm UTC](https://discuss.elastic.co/t/forwarding-logs-from-sun-solaris-to-elk/338147 "2023-07-11T21:04:29Z")
We need to find a workaround for forwarding logs from Sun Solaris to ELK. Any ideas - very welcome! No any vendors like Logstash, Filebeat, Vector officially support Log Forwarders on Sun Solaris. Any help? Thank you. --- ## [Cannot use terms aggregation to get the field which is injest by enrich processor](https://discuss.elastic.co/t/cannot-use-terms-aggregation-to-get-the-field-which-is-injest-by-enrich-processor/336554)
**Author:** [@JasonREC](https://discuss.elastic.co/u/JasonREC)\ **Replies:** 1\ **Last updated:** [July 11, 2023, 8:23pm UTC](https://discuss.elastic.co/t/cannot-use-terms-aggregation-to-get-the-field-which-is-injest-by-enrich-processor/336554 "2023-07-11T20:23:04Z")
I am trying to get the db\_tag field which is injested using injestpipeline with enrich processor, but it does not return anything even the field is existed in the doucment. --- ## [Hi Team, Do we have any radio button option in Kibana](https://discuss.elastic.co/t/hi-team-do-we-have-any-radio-button-option-in-kibana/336670)
**Author:** [@Abj\_Ins](https://discuss.elastic.co/u/Abj_Ins)\ **Replies:** 1\ **Last updated:** [July 11, 2023, 7:58pm UTC](https://discuss.elastic.co/t/hi-team-do-we-have-any-radio-button-option-in-kibana/336670 "2023-07-11T19:58:46Z")
Do we have any radio button option in Kibana. --- ## [ILM With Index Sorting](https://discuss.elastic.co/t/ilm-with-index-sorting/338140)
**Author:** [@Ofir\_Sudai](https://discuss.elastic.co/u/Ofir_Sudai)\ **Replies:** 0\ **Last updated:** [July 11, 2023, 4:42pm UTC](https://discuss.elastic.co/t/ilm-with-index-sorting/338140 "2023-07-11T16:42:29Z")
Hi, We have an index the represents a feed of nft related activities (listing, bid, minted, transfer, sold, etc). The index is used to return realtime feed for users in our system so latency is a priority. The index is… --- ## [Stuck "Cancelled Tasks" In ElasticSearch 8.6.2 causing Cluster failure](https://discuss.elastic.co/t/stuck-cancelled-tasks-in-elasticsearch-8-6-2-causing-cluster-failure/337490)
**Author:** [@Thomas\_Kuisel](https://discuss.elastic.co/u/Thomas_Kuisel)\ **Replies:** 18\ **Last updated:** [July 11, 2023, 4:27pm UTC](https://discuss.elastic.co/t/stuck-cancelled-tasks-in-elasticsearch-8-6-2-causing-cluster-failure/337490 "2023-07-11T16:27:41Z")
Hi - We are using Elasticsearch 8.6.2 running on Azure AKS and noticed some serious issues lately. As of last week the week of Jun30 2023, we started noticing huge unresolved search transport queues in our cluster. The… --- ## [What if difference between setting node.roles: \["data\_hot"\] vs node.attr.box\_type: hot?](https://discuss.elastic.co/t/what-if-difference-between-setting-node-roles-data-hot-vs-node-attr-box-type-hot/337766)
**Author:** [@kuldeep\_gupta](https://discuss.elastic.co/u/kuldeep_gupta)\ **Replies:** 3\ **Last updated:** [July 11, 2023, 4:27pm UTC](https://discuss.elastic.co/t/what-if-difference-between-setting-node-roles-data-hot-vs-node-attr-box-type-hot/337766 "2023-07-11T16:27:15Z")
What is the difference between node.roles:\["data\_hot"\] vs node.attr.box\_type: hot. --- ## [Starting Elasticsearch failed](https://discuss.elastic.co/t/starting-elasticsearch-failed/337616)
**Author:** [@stephane\_chan](https://discuss.elastic.co/u/stephane_chan)\ **Replies:** 4\ **Last updated:** [July 11, 2023, 2:43pm UTC](https://discuss.elastic.co/t/starting-elasticsearch-failed/337616 "2023-07-11T14:43:07Z")
Hi, Why does the elasticsearch.service status always give me this every time I start/restart my server? . ● elasticsearch.service - Elasticsearch Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled;… --- ## [Using the transform feature for summarizing APM indexes](https://discuss.elastic.co/t/using-the-transform-feature-for-summarizing-apm-indexes/338118)
**Author:** [@Mohammad\_Mousavi](https://discuss.elastic.co/u/Mohammad_Mousavi)\ **Replies:** 0\ **Last updated:** [July 11, 2023, 2:27pm UTC](https://discuss.elastic.co/t/using-the-transform-feature-for-summarizing-apm-indexes/338118 "2023-07-11T14:27:23Z")
I have elasticsearch version 7.17.1 and our applications are integrated with APM server to send metrics. The problem that we have is it generates heavy indexes, and we want to have old data as well. I thought maybe I ca… --- ## [What mapping or structure should I use for an index that will have very varying fields per document?](https://discuss.elastic.co/t/what-mapping-or-structure-should-i-use-for-an-index-that-will-have-very-varying-fields-per-document/338114)
**Author:** [@Bart\_de\_Man](https://discuss.elastic.co/u/Bart_de_Man)\ **Replies:** 0\ **Last updated:** [July 11, 2023, 2:03pm UTC](https://discuss.elastic.co/t/what-mapping-or-structure-should-i-use-for-an-index-that-will-have-very-varying-fields-per-document/338114 "2023-07-11T14:03:04Z")
Hi there! As per title; i'd like to have an index which will have very different fields per document. How should I approach this task? What does the mapping look like, if any. Short example of what i'd like to acchieve… --- ## [FluentBit not able to connect to ElasticSearch even with username password given in config](https://discuss.elastic.co/t/fluentbit-not-able-to-connect-to-elasticsearch-even-with-username-password-given-in-config/338104)
**Author:** [@Ganesh\_Kannan\_K\_S1](https://discuss.elastic.co/u/Ganesh_Kannan_K_S1)\ **Replies:** 1\ **Last updated:** [July 11, 2023, 1:58pm UTC](https://discuss.elastic.co/t/fluentbit-not-able-to-connect-to-elasticsearch-even-with-username-password-given-in-config/338104 "2023-07-11T13:58:05Z")
I am using an AWS EC2 server for running a single-node Elasticsearch instance. I have kibana installed in the same server. I am able to configure 'FluentD' to this node with security enabled, but not 'Fluent-bit'. I am … --- ## [How to archive snapshots](https://discuss.elastic.co/t/how-to-archive-snapshots/337959)
**Author:** [@jaykb77](https://discuss.elastic.co/u/jaykb77)\ **Replies:** 5\ **Last updated:** [July 11, 2023, 12:27pm UTC](https://discuss.elastic.co/t/how-to-archive-snapshots/337959 "2023-07-11T12:27:59Z")
Hi all, We have a requirement to archive snapshots for long term retention. However since the snapshot directories are incremental, we do not want to simply zip it and send to archive storage. Is there a recommended alt… --- ## [Elastic cloud and Elastic Package Registry](https://discuss.elastic.co/t/elastic-cloud-and-elastic-package-registry/336272)
**Author:** [@adrien\_moreau](https://discuss.elastic.co/u/adrien_moreau)\ **Replies:** 3\ **Last updated:** [July 11, 2023, 12:16pm UTC](https://discuss.elastic.co/t/elastic-cloud-and-elastic-package-registry/336272 "2023-07-11T12:16:43Z")
I have developed a custom integration for Elastic but I don't want to publish this integration to the public EPR repo. I am running Elastic on Elastic Cloud. I would like to know if it is possible to push my custom inte… --- ## [Transferring a writable index from one cluster to another](https://discuss.elastic.co/t/transferring-a-writable-index-from-one-cluster-to-another/337934)
**Author:** [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Replies:** 15\ **Last updated:** [July 11, 2023, 11:35am UTC](https://discuss.elastic.co/t/transferring-a-writable-index-from-one-cluster-to-another/337934 "2023-07-11T11:35:10Z")
I am trying to migrate writable indices from one cluster to another. I wanted to know what could possibly the best approach for doing this. Currently I am doing it as follows : Phase1 -\> Before taking snapshot of init… --- ## [\[ECONNREFUSED\] connect ECONNREFUSED .. Using elaticsearch connector to connect to theHive fails](https://discuss.elastic.co/t/econnrefused-connect-econnrefused-using-elaticsearch-connector-to-connect-to-thehive-fails/338096)
**Author:** [@Cone](https://discuss.elastic.co/u/Cone)\ **Replies:** 0\ **Last updated:** [July 11, 2023, 11:10am UTC](https://discuss.elastic.co/t/econnrefused-connect-econnrefused-using-elaticsearch-connector-to-connect-to-thehive-fails/338096 "2023-07-11T11:10:43Z")
When I try to connect the Elasticsearch to TheHive via webhooks i get this error shown in the image . Why is taht happenning, The Hive is up and running on that ip address and port? Thanx in advance My webhook is … --- ## [Trino connector : Elasticsearch exception \[type=search\_phase\_execution\_exception, reason=all shards failed](https://discuss.elastic.co/t/trino-connector-elasticsearch-exception-type-search-phase-execution-exception-reason-all-shards-failed/338092)
**Author:** [@subash\_k](https://discuss.elastic.co/u/subash_k)\ **Replies:** 0\ **Last updated:** [July 11, 2023, 10:39am UTC](https://discuss.elastic.co/t/trino-connector-elasticsearch-exception-type-search-phase-execution-exception-reason-all-shards-failed/338092 "2023-07-11T10:39:09Z")
Pushing logs via logstash and while retrieving data from two tables query failing with below error Using trino query engine connector.name=elasticsearch and also tried with below config but no luck elasticsearch.da… --- ## [Error : environment is not locked in 3 node Kubernetes Elastic Deployment](https://discuss.elastic.co/t/error-environment-is-not-locked-in-3-node-kubernetes-elastic-deployment/338085)
**Author:** [@brusque.sowers](https://discuss.elastic.co/u/brusque.sowers)\ **Replies:** 1\ **Last updated:** [July 11, 2023, 9:51am UTC](https://discuss.elastic.co/t/error-environment-is-not-locked-in-3-node-kubernetes-elastic-deployment/338085 "2023-07-11T09:51:10Z")
We have a 3 node Elastic deployment on Kubernetes. We encountered the following error in elastic Logs : elasticsearch.node.id":"LMHIgOg6RrWkCRWFY5QyZQ","elasticsearch.node.name":"elasticsearch-0.es-service","elasticsea… --- ## [Sysdig integration with ELK](https://discuss.elastic.co/t/sysdig-integration-with-elk/338057)
**Author:** [@pennywise01](https://discuss.elastic.co/u/pennywise01)\ **Replies:** 2\ **Last updated:** [July 11, 2023, 6:47am UTC](https://discuss.elastic.co/t/sysdig-integration-with-elk/338057 "2023-07-11T06:47:35Z")
Hi all, i am trying to intergrate sysdig with ELK stack. I am following a tutorial from a blog. I already configured logstash to put the log into elasticsearch but i got an error. \> Blockquote \[ERROR\] 2023-07-11 05:45:… --- ## [Why isElectionQuorum need lastCommitedConfiguration and lastAcceptedConfiguration all pass?](https://discuss.elastic.co/t/why-iselectionquorum-need-lastcommitedconfiguration-and-lastacceptedconfiguration-all-pass/338048)
**Author:** [@cm\_z](https://discuss.elastic.co/u/cm_z)\ **Replies:** 1\ **Last updated:** [July 11, 2023, 6:43am UTC](https://discuss.elastic.co/t/why-iselectionquorum-need-lastcommitedconfiguration-and-lastacceptedconfiguration-all-pass/338048 "2023-07-11T06:43:01Z")
Why does Elasticsearch need to check if both lastAcceptedConfiguration and lastCommitedConfiguration are over the majority threshold when deciding whether to start an election? What is the reasoning behind this, and are … --- ## [Implementing machine learning API in Rust using elasticsearch8.4.0-alpha.1 library](https://discuss.elastic.co/t/implementing-machine-learning-api-in-rust-using-elasticsearch8-4-0-alpha-1-library/337770)
**Author:** [@aniket\_mandhare](https://discuss.elastic.co/u/aniket_mandhare)\ **Replies:** 2\ **Last updated:** [July 11, 2023, 6:11am UTC](https://discuss.elastic.co/t/implementing-machine-learning-api-in-rust-using-elasticsearch8-4-0-alpha-1-library/337770 "2023-07-11T06:11:23Z")
Hey folks, I am trying to implement machine learning API in Rust programming language using elasticsearch8.4.0-alpha.1 library and now I am stuck. I want to use ML model which is already imported in Elasticsearch which c… --- ## [Failed to create outliear detection](https://discuss.elastic.co/t/failed-to-create-outliear-detection/337878)
**Author:** [@JasonREC](https://discuss.elastic.co/u/JasonREC)\ **Replies:** 4\ **Last updated:** [July 11, 2023, 1:59am UTC](https://discuss.elastic.co/t/failed-to-create-outliear-detection/337878 "2023-07-11T01:59:31Z")
Hi, I was trying to create a machine learning which is simply to identify a distinct record Below is my data: In my index, I have 20 documents, and 7 fields Codenum is a keyword A,B,C,V+,V-,P are boolean only … --- ## [CPU heavy load after indexing 1.5M vectors](https://discuss.elastic.co/t/cpu-heavy-load-after-indexing-1-5m-vectors/338002)
**Author:** [@mwon](https://discuss.elastic.co/u/mwon)\ **Replies:** 1\ **Last updated:** [July 11, 2023, 12:23am UTC](https://discuss.elastic.co/t/cpu-heavy-load-after-indexing-1-5m-vectors/338002 "2023-07-11T00:23:00Z")
Hi, I want to build an index with more than 4M vectors of dimension 768. My setup for now is a DigitalOcean droplet with 4GB and 2vCPU (planning to increase as needed). I first started to add the first million which se… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=229) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=231)