# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=353 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 354 --- ## [Seeking advice on setting up the ELK Stack](https://discuss.elastic.co/t/seeking-advice-on-setting-up-the-elk-stack/348968)
**Author:** [@Carrier99](https://discuss.elastic.co/u/Carrier99)\ **Replies:** 1\ **Last updated:** [December 12, 2023, 12:30am UTC](https://discuss.elastic.co/t/seeking-advice-on-setting-up-the-elk-stack/348968 "2023-12-12T00:30:36Z")
Hey there. I want to set up the ELK stack, and I'm wondering about a good way to set it up. I'm running Proxmox so I have the options of either VMs or LXCs (was thinking of going with LXCs). My main question is should… --- ## [Disk space measure for Elasticsearch service](https://discuss.elastic.co/t/disk-space-measure-for-elasticsearch-service/349076)
**Author:** [@kaushalshriyan](https://discuss.elastic.co/u/kaushalshriyan)\ **Replies:** 1\ **Last updated:** [December 12, 2023, 12:11am UTC](https://discuss.elastic.co/t/disk-space-measure-for-elasticsearch-service/349076 "2023-12-12T00:11:07Z")
Hi, I have provisioned 300 GB of Hard disk storage to Elastic search stack. Is there a way to measure how much storage is consumed by ES service, as I need to work on Capacity planning. For example, how much memory and … --- ## [Problems with elasticsearch container](https://discuss.elastic.co/t/problems-with-elasticsearch-container/349080)
**Author:** [@Ergo\_Proxy](https://discuss.elastic.co/u/Ergo_Proxy)\ **Replies:** 1\ **Last updated:** [December 11, 2023, 6:06pm UTC](https://discuss.elastic.co/t/problems-with-elasticsearch-container/349080 "2023-12-11T18:06:39Z")
I have some trouble making a docker with elastic. Here is the .yaml and a fragment of the terminal log result that I think shows the problem. services: setup: image: docker.elastic.co/elasticsearc… --- ## [Migrate shards from one node set to another](https://discuss.elastic.co/t/migrate-shards-from-one-node-set-to-another/349065)
**Author:** [@hashworks](https://discuss.elastic.co/u/hashworks)\ **Replies:** 1\ **Last updated:** [December 11, 2023, 5:33pm UTC](https://discuss.elastic.co/t/migrate-shards-from-one-node-set-to-another/349065 "2023-12-11T17:33:54Z")
Hi, I have three sets of nodes: A, B and C. Over time, I want to migrate all shards on A and B to C (and remove A and B from the cluster). I could do that all at once by setting the cluster routing allocation setting: … --- ## [Splitting an array of objects using Logstash](https://discuss.elastic.co/t/splitting-an-array-of-objects-using-logstash/349066)
**Author:** [@M0hsen](https://discuss.elastic.co/u/M0hsen)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 4:21pm UTC](https://discuss.elastic.co/t/splitting-an-array-of-objects-using-logstash/349066 "2023-12-11T16:21:38Z")
Hello everyone, I'm trying to split the following array of objects into multiple log events: \[ { "time": "\*", "twkMessageId": "\*", "environmentName": "\*", "virtualhostName": "default", "apiproxyNa… --- ## [TLS and Metricbeat](https://discuss.elastic.co/t/tls-and-metricbeat/349064)
**Author:** [@Jame\_M](https://discuss.elastic.co/u/Jame_M)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 3:53pm UTC](https://discuss.elastic.co/t/tls-and-metricbeat/349064 "2023-12-11T15:53:12Z")
Hello, mostly new to the Elastic Search framework, but I have a question. I have successfully deployed Metricbeat onto a set of remote servers pointing back to our Elastic Search via a IP address, and those work perfectl… --- ## [Kibana Server is not ready yet](https://discuss.elastic.co/t/kibana-server-is-not-ready-yet/349026)
**Author:** [@AbcDE](https://discuss.elastic.co/u/AbcDE)\ **Replies:** 1\ **Last updated:** [December 11, 2023, 3:55pm UTC](https://discuss.elastic.co/t/kibana-server-is-not-ready-yet/349026 "2023-12-11T15:55:50Z")
Hi I'm a new user of Kibana&Ubuntu(and forum) and i'm trying to start Kibana service for view the log of Suricata with a graphical interface, so please if you give me help be clear and precise Thanks. ok so i'm on Ubun… --- ## [Grok with custom pattern works in debugger but not in pipline](https://discuss.elastic.co/t/grok-with-custom-pattern-works-in-debugger-but-not-in-pipline/348957)
**Author:** [@helldunkel](https://discuss.elastic.co/u/helldunkel)\ **Replies:** 5\ **Last updated:** [December 11, 2023, 3:48pm UTC](https://discuss.elastic.co/t/grok-with-custom-pattern-works-in-debugger-but-not-in-pipline/348957 "2023-12-11T15:48:22Z")
Hi, I´m have a lot of problems to get a dataset in elastic. In Debugger it works. Log \<30\>2023:12:08-12:59:39 fw-swr-2 ulogd\[32373\]: grock .\*\>%{SOPHOS\_TIMESTAMP:\_tmp.timestamp} %{TEST:firewall.name} custom pattern … --- ## [Scripted upsert is failing in Elasticsearch output](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057)
**Author:** [@gshankar-elastic](https://discuss.elastic.co/u/gshankar-elastic)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 3:02pm UTC](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057 "2023-12-11T15:02:26Z")
I am using an indexed script in the output to transform the event data like this: input { kafka { bootstrap\_servers =\> "kafka.localhost.com:9092" topics =\> \["enriched"\] } } filter { json { … --- ## [Memory spikes on ILM transition](https://discuss.elastic.co/t/memory-spikes-on-ilm-transition/349008)
**Author:** [@azhurbilo](https://discuss.elastic.co/u/azhurbilo)\ **Replies:** 7\ **Last updated:** [December 11, 2023, 11:38am UTC](https://discuss.elastic.co/t/memory-spikes-on-ilm-transition/349008 "2023-12-11T11:38:24Z")
We use hot-warm-cold architecture policy { "filebeat": { "version": 39128, "modified\_date": "2023-12-11T04:05:20.404Z", "policy": { "phases": { "hot": { "min\_age": "0ms", … --- ## [Expose Elasticsearch on GKE to Dataproc (Performantly )](https://discuss.elastic.co/t/expose-elasticsearch-on-gke-to-dataproc-performantly/349056)
**Author:** [@Pat\_Humphreys](https://discuss.elastic.co/u/Pat_Humphreys)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 2:42pm UTC](https://discuss.elastic.co/t/expose-elasticsearch-on-gke-to-dataproc-performantly/349056 "2023-12-11T14:42:19Z")
What is the recomended way of exposing an elasticsearch cluster outside of the GKE cluster e.g. to use by a spark job running in dataproc (Using ES Hadoop libary) within the same VPC, without enabling the wan.only option… --- ## [Upgrade Single Node Docker Instance](https://discuss.elastic.co/t/upgrade-single-node-docker-instance/349041)
**Author:** [@litronics](https://discuss.elastic.co/u/litronics)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 2:24pm UTC](https://discuss.elastic.co/t/upgrade-single-node-docker-instance/349041 "2023-12-11T14:24:23Z")
Hi, I am trying to upgrade a single node docker instance from 8.6.0 to the latest version. My first try was to update the docker containers to the latest version and start them on the old data. Kibana upgraded well and… --- ## [Uprading Elastic search upgrade from 6.6.1 to 7.16.3](https://discuss.elastic.co/t/uprading-elastic-search-upgrade-from-6-6-1-to-7-16-3/348851)
**Author:** [@Abhishek\_Gangadharai](https://discuss.elastic.co/u/Abhishek_Gangadharai)\ **Replies:** 3\ **Last updated:** [December 11, 2023, 1:20pm UTC](https://discuss.elastic.co/t/uprading-elastic-search-upgrade-from-6-6-1-to-7-16-3/348851 "2023-12-11T13:20:36Z")
Uprading Elastic search upgrade from 6.6.1 to 7.16.3 in Bitbucket datacenter cluster nodes we are facing challenges can please help for proper documentation for Upgrading Elastic search Instructions, It will help us fo… --- ## [Write structured log but field is never recognized as 'date'](https://discuss.elastic.co/t/write-structured-log-but-field-is-never-recognized-as-date/348920)
**Author:** [@Heija](https://discuss.elastic.co/u/Heija)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 12:01pm UTC](https://discuss.elastic.co/t/write-structured-log-but-field-is-never-recognized-as-date/348920 "2023-12-11T12:01:29Z")
Hi! I have a .net application which write log entries per nlog direct to Elasticsearch. It works with strings, numbers and booleans. Now I need to transfer a 'date' as structured log entry to Elasticsearch, but it wil… --- ## [I am facing issue with apm related indices "illegal\_argument\_exception: index.lifecycle.rollover\_alias \[apm-7.9.1-transaction\] does not point to index \[apm-7.9.1-transaction-000087\]"](https://discuss.elastic.co/t/i-am-facing-issue-with-apm-related-indices-illegal-argument-exception-index-lifecycle-rollover-alias-apm-7-9-1-transaction-does-not-point-to-index-apm-7-9-1-transaction-000087/349039)
**Author:** [@vaiagr](https://discuss.elastic.co/u/vaiagr)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 11:53am UTC](https://discuss.elastic.co/t/i-am-facing-issue-with-apm-related-indices-illegal-argument-exception-index-lifecycle-rollover-alias-apm-7-9-1-transaction-does-not-point-to-index-apm-7-9-1-transaction-000087/349039 "2023-12-11T11:53:28Z")
illegal\_argument\_exception: index.lifecycle.rollover\_alias \[apm-7.9.1-transaction\] does not point to index \[apm-7.9.1-transaction-000087\] --- ## [One of our nodes is constantly leaving with "master not discovered yet"](https://discuss.elastic.co/t/one-of-our-nodes-is-constantly-leaving-with-master-not-discovered-yet/349031)
**Author:** [@coudenysj](https://discuss.elastic.co/u/coudenysj)\ **Replies:** 1\ **Last updated:** [December 11, 2023, 10:58am UTC](https://discuss.elastic.co/t/one-of-our-nodes-is-constantly-leaving-with-master-not-discovered-yet/349031 "2023-12-11T10:58:22Z")
We have a cluster with 33 nodes, and one server (always the same one) is leaving the cluster quite often. After restarting the service, it joins immediately. The exact error is: \[2023-12-11T11:23:29,293\]\[WARN \]\[o.e.c.… --- ## [Is there any way to update the \`last\_run\_metadata\_file\` in the output plugin?](https://discuss.elastic.co/t/is-there-any-way-to-update-the-last-run-metadata-file-in-the-output-plugin/348919)
**Author:** [@gayatri\_SN](https://discuss.elastic.co/u/gayatri_SN)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 10:02am UTC](https://discuss.elastic.co/t/is-there-any-way-to-update-the-last-run-metadata-file-in-the-output-plugin/348919 "2023-12-11T10:02:30Z")
My scenario is as follows: I am using the JDBC input plugin with a tracking column and last\_run\_metadata\_file, and it is working as expected. However, when the ETL host is down or unreachable, the filter API throws an er… --- ## [Elasticsearch node disconnect](https://discuss.elastic.co/t/elasticsearch-node-disconnect/349010)
**Author:** [@Farid\_Niasti](https://discuss.elastic.co/u/Farid_Niasti)\ **Replies:** 1\ **Last updated:** [December 11, 2023, 8:48am UTC](https://discuss.elastic.co/t/elasticsearch-node-disconnect/349010 "2023-12-11T08:48:47Z")
I have 3 nodes of Elasticsearch, sometimes one of my node leave cluster with bellow log: org.elasticsearch.cluster.block.ClusterBlockException: blocked by: \[SERVICE\_UNAVAILABLE/1/state not recovered / initialized\]; … --- ## [Lets Encrypt not working from console but does from firefox](https://discuss.elastic.co/t/lets-encrypt-not-working-from-console-but-does-from-firefox/349000)
**Author:** [@Donovan\_Hoare](https://discuss.elastic.co/u/Donovan_Hoare)\ **Replies:** 3\ **Last updated:** [December 11, 2023, 8:04am UTC](https://discuss.elastic.co/t/lets-encrypt-not-working-from-console-but-does-from-firefox/349000 "2023-12-11T08:04:42Z")
Good Day All. I have setup a 2-node cluster ith Elasticsearch and kibana. I took me quite some time on how to get lets-encrypt real certs to work. The cert now seems to be ok with kibana and Elasticsearch communicatio… --- ## [Tenable.sc integration not visible on the intergations tab](https://discuss.elastic.co/t/tenable-sc-integration-not-visible-on-the-intergations-tab/349019)
**Author:** [@Kotsos](https://discuss.elastic.co/u/Kotsos)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 7:48am UTC](https://discuss.elastic.co/t/tenable-sc-integration-not-visible-on-the-intergations-tab/349019 "2023-12-11T07:48:42Z")
Hello, I am trying to install the Tenable.sc integration with the ELK stack, which according to the documentation collects and parses data from the Tenable.sc APIs. However it is nowhere to be found on the "Browse inter… --- ## [Kibana Log In Error](https://discuss.elastic.co/t/kibana-log-in-error/348861)
**Author:** [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 5:31am UTC](https://discuss.elastic.co/t/kibana-log-in-error/348861 "2023-12-11T05:31:12Z")
Hi Team, When my mount point in Elasticsearch server reaches between 85-90 percent , we are not able to login the Kibana UI as per screenshot. While checking log in Kibana show disk usage issue and after adding space a… --- ## [Getting the error as "agent\_id\_status": "auth\_metadata\_missing" in the file beat data stream on elastic agent standalone deployed on k8s cluster](https://discuss.elastic.co/t/getting-the-error-as-agent-id-status-auth-metadata-missing-in-the-file-beat-data-stream-on-elastic-agent-standalone-deployed-on-k8s-cluster/349007)
**Author:** [@Subrahmanyam\_Veerank](https://discuss.elastic.co/u/Subrahmanyam_Veerank)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 5:14am UTC](https://discuss.elastic.co/t/getting-the-error-as-agent-id-status-auth-metadata-missing-in-the-file-beat-data-stream-on-elastic-agent-standalone-deployed-on-k8s-cluster/349007 "2023-12-11T05:14:04Z")
Deployed the elastic agent standalone in a kubernetes cluster and a agent policy is created with a kubernetes integration. Getting some of the metric visualizations of the pods,nodes,etc but when i checked the filebeat d… --- ## [Custom Index Creation Issue](https://discuss.elastic.co/t/custom-index-creation-issue/348979)
**Author:** [@Mani\_Manikanta](https://discuss.elastic.co/u/Mani_Manikanta)\ **Replies:** 4\ **Last updated:** [December 11, 2023, 4:01am UTC](https://discuss.elastic.co/t/custom-index-creation-issue/348979 "2023-12-11T04:01:57Z")
Hello, I am New to ELK Stack, Trying to Setup ELK Stack in Single Server following Elastic Documentation and I am Stuck here now Can Anyone Please Look into this and help me out Unable to Create a Custom Index from Lo… --- ## [CCR - auto-follow problem on data streams](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789)
**Author:** [@Wojciech\_Kwiecien](https://discuss.elastic.co/u/Wojciech_Kwiecien)\ **Replies:** 5\ **Last updated:** [December 11, 2023, 1:13am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789 "2023-12-11T01:13:10Z")
I found this topic Elastic Cross Cluster Replication of Data Stream But I do not find a solution for me. I was able to create CCR auto-follow pattern and nothing happened when I ran GET /\_ccr/stats After I follow inst… --- ## [Gave up on setting up ELK](https://discuss.elastic.co/t/gave-up-on-setting-up-elk/348984)
**Author:** [@Aamira](https://discuss.elastic.co/u/Aamira)\ **Replies:** 2\ **Last updated:** [December 10, 2023, 10:38pm UTC](https://discuss.elastic.co/t/gave-up-on-setting-up-elk/348984 "2023-12-10T22:38:45Z")
After 12 years of Linux background and infrastructure management, I accepted defeat setting up ELK for monitoring. I struggled for over 2 weeks trying every guide. but it seems that ELK is not worth the headache. The g… --- ## [Beats\_input\_raw\_event](https://discuss.elastic.co/t/beats-input-raw-event/348575)
**Author:** [@Yuval\_Algresi](https://discuss.elastic.co/u/Yuval_Algresi)\ **Replies:** 3\ **Last updated:** [December 10, 2023, 10:28pm UTC](https://discuss.elastic.co/t/beats-input-raw-event/348575 "2023-12-10T22:28:42Z")
Hello, I use winlogbeat to ship event viewer logs to my elastic stack. It first goes to logstash and from there to elastic - I use beats input plugin. Usually there is an event.original field that contains the raw eve… --- ## [What is the impact on a live system doing a change in elasticsearch.yml file to change the node.attr.storage\_term: to be none?](https://discuss.elastic.co/t/what-is-the-impact-on-a-live-system-doing-a-change-in-elasticsearch-yml-file-to-change-the-node-attr-storage-term-to-be-none/348980)
**Author:** [@mpniel](https://discuss.elastic.co/u/mpniel)\ **Replies:** 3\ **Last updated:** [December 10, 2023, 2:25pm UTC](https://discuss.elastic.co/t/what-is-the-impact-on-a-live-system-doing-a-change-in-elasticsearch-yml-file-to-change-the-node-attr-storage-term-to-be-none/348980 "2023-12-10T14:25:17Z")
Hello, I run GET /\_cat/nodeattrs?v and found that the node have storage\_term attribute as cold. What is the impact on a live system doing a change in elasticsearch.yml file to change the node.attr.storage\_term: to be… --- ## [Error: master not discovered yet , Elasticsearch cluster using docker swarm with three nodes on three separate servers](https://discuss.elastic.co/t/error-master-not-discovered-yet-elasticsearch-cluster-using-docker-swarm-with-three-nodes-on-three-separate-servers/348973)
**Author:** [@hossein\_rahmatei](https://discuss.elastic.co/u/hossein_rahmatei)\ **Replies:** 1\ **Last updated:** [December 10, 2023, 12:42pm UTC](https://discuss.elastic.co/t/error-master-not-discovered-yet-elasticsearch-cluster-using-docker-swarm-with-three-nodes-on-three-separate-servers/348973 "2023-12-10T12:42:24Z")
I want to set up an elasticsearch cluster using docker swarm with three nodes on three separate servers. But when I do the docker stack deploy command and the service comes up, but I get docker logs from the containe… --- ## [Kibana UI Dashboard Access Issue](https://discuss.elastic.co/t/kibana-ui-dashboard-access-issue/348971)
**Author:** [@Mani\_Manikanta](https://discuss.elastic.co/u/Mani_Manikanta)\ **Replies:** 11\ **Last updated:** [December 10, 2023, 8:55am UTC](https://discuss.elastic.co/t/kibana-ui-dashboard-access-issue/348971 "2023-12-10T08:55:27Z")
Unable to Access Kibana UI Dashboard Getting Below Error, Getting Please upgrade browser in Google Chrome/Edge/Firefox Kibana - 8.3.3 Elasticsearch - 8.3.3 No Domain Mapped Trying to Access using kibana-ip:5601 --- ## [How filebeat custom parse rules?](https://discuss.elastic.co/t/how-filebeat-custom-parse-rules/348878)
**Author:** [@fansehep](https://discuss.elastic.co/u/fansehep)\ **Replies:** 11\ **Last updated:** [December 10, 2023, 6:42am UTC](https://discuss.elastic.co/t/how-filebeat-custom-parse-rules/348878 "2023-12-10T06:42:11Z")
I have a project. it will output a spiecal log like: | xxx | xxx | xxx | | xxx | xxx | xxx | | xxx | xxx | xxx | I want to know it there anyway to custom self parse rules or some plugins to help me ? [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=352) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=354)