# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=114 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 115 --- ## [Filtering messages in Logstash](https://discuss.elastic.co/t/filtering-messages-in-logstash/373126)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 4\ **Last updated:** [January 14, 2025, 7:49pm UTC](https://discuss.elastic.co/t/filtering-messages-in-logstash/373126 "2025-01-14T19:49:55Z")
Hi All, The incoming feed (log) to Logstash has parameters which are delimited by ~|~ These are being mutated and split in Logstash as follows. : if \[type\] == "tv\_dmz\_access" { mutate { … --- ## [Sizing a secondary Cluster with Frozen nodes and Searchable Snapshots](https://discuss.elastic.co/t/sizing-a-secondary-cluster-with-frozen-nodes-and-searchable-snapshots/373184)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 1\ **Last updated:** [January 14, 2025, 6:53pm UTC](https://discuss.elastic.co/t/sizing-a-secondary-cluster-with-frozen-nodes-and-searchable-snapshots/373184 "2025-01-14T18:53:40Z")
Hi all, I’m working on a solution involving the use of searchable snapshots, and I need some clarification to properly size a second cluster. Here’s the context: I plan to use my production cluster (Platinum license) … --- ## [Iterating over fields with multiple filter processes](https://discuss.elastic.co/t/iterating-over-fields-with-multiple-filter-processes/373187)
**Author:** [@philhagen](https://discuss.elastic.co/u/philhagen)\ **Replies:** 2\ **Last updated:** [January 14, 2025, 6:32pm UTC](https://discuss.elastic.co/t/iterating-over-fields-with-multiple-filter-processes/373187 "2025-01-14T18:32:35Z")
I run a series of consistent filters over a series of fields that may or may not exist. For consistency and to simplify maintenance, I'd like to do this with one code block instead of multiple repetitive ones. For a sp… --- ## [Elasticsearch Performance Issues on CapCut Resource Website](https://discuss.elastic.co/t/elasticsearch-performance-issues-on-capcut-resource-website/373130)
**Author:** [@joeroot](https://discuss.elastic.co/u/joeroot)\ **Replies:** 2\ **Last updated:** [January 14, 2025, 4:46pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-issues-on-capcut-resource-website/373130 "2025-01-14T16:46:29Z")
Hi everyone, I’m running a website focused on CapCut tutorials and resources, and I’ve implemented Elasticsearch for powering the search functionality. However, I’m experiencing significant performance issues: Slow Se… --- ## [Heartbeat roles that are required](https://discuss.elastic.co/t/heartbeat-roles-that-are-required/373189)
**Author:** [@Gong](https://discuss.elastic.co/u/Gong)\ **Replies:** 0\ **Last updated:** [January 14, 2025, 4:00pm UTC](https://discuss.elastic.co/t/heartbeat-roles-that-are-required/373189 "2025-01-14T16:00:18Z")
Hi, I'm integrating Heartbeat and I've created a user but despite reading the doc's including Grant users access to secured resources | Heartbeat Reference \[8.17\] | Elastic and scribbling a page of notes, I'm still gett… --- ## [Snapshot Lifecycle Lifecyle and Managment Creating issue for rollover](https://discuss.elastic.co/t/snapshot-lifecycle-lifecyle-and-managment-creating-issue-for-rollover/373154)
**Author:** [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\ **Replies:** 1\ **Last updated:** [January 14, 2025, 2:50pm UTC](https://discuss.elastic.co/t/snapshot-lifecycle-lifecyle-and-managment-creating-issue-for-rollover/373154 "2025-01-14T14:50:53Z")
Overview i have elasticsearch 8.11 with no replicas right now, only 5 hot nodes and 2 frozen that's it. Here is index lifecycle policy is this PUT \_ilm/policy/ConrainerLogs { "policy": { "phases": { … --- ## [Hiding or Excluding Fields in Filebeat Data Ingestion via MQTT](https://discuss.elastic.co/t/hiding-or-excluding-fields-in-filebeat-data-ingestion-via-mqtt/373166)
**Author:** [@TheNewGuy123](https://discuss.elastic.co/u/TheNewGuy123)\ **Replies:** 0\ **Last updated:** [January 14, 2025, 10:38am UTC](https://discuss.elastic.co/t/hiding-or-excluding-fields-in-filebeat-data-ingestion-via-mqtt/373166 "2025-01-14T10:38:26Z")
Hi, I'm encountering some challenges while ingesting data using Filebeat with MQTT. My setup involves ingesting data from multiple MQTT topics that follow this naming structure: \/\/\ … --- ## [Startup failure - arch64 - crashes - arm64 - fedora, rocky9, centos9, macbook m4, docker](https://discuss.elastic.co/t/startup-failure-arch64-crashes-arm64-fedora-rocky9-centos9-macbook-m4-docker/371708)
**Author:** [@contactnkm](https://discuss.elastic.co/u/contactnkm)\ **Replies:** 7\ **Last updated:** [January 14, 2025, 10:35am UTC](https://discuss.elastic.co/t/startup-failure-arch64-crashes-arm64-fedora-rocky9-centos9-macbook-m4-docker/371708 "2025-01-14T10:35:49Z")
Get the following error whenever trying to install and run elasticsearch on aarch64 rocky9.5 or fedora 6.11.4 stdoout: # # A fatal error has been detected by the Java Runtime Environment: # # SIGILL (0x4) at pc=0x0000… --- ## [Nodes going down without any error logs](https://discuss.elastic.co/t/nodes-going-down-without-any-error-logs/373155)
**Author:** [@Naveen\_S1](https://discuss.elastic.co/u/Naveen_S1)\ **Replies:** 1\ **Last updated:** [January 14, 2025, 7:59am UTC](https://discuss.elastic.co/t/nodes-going-down-without-any-error-logs/373155 "2025-01-14T07:59:53Z")
I have an ES cluster hosted on-prem with around 110 nodes. Each node resides on a 2TB partition. I have to index around 5 set of 250 GB JSON data into the cluster. Each JSON file contains 10,000 docs. I am using a custo… --- ## [Can't make MongoDB connection work in Logstash Filter](https://discuss.elastic.co/t/cant-make-mongodb-connection-work-in-logstash-filter/373144)
**Author:** [@mdschoe](https://discuss.elastic.co/u/mdschoe)\ **Replies:** 2\ **Last updated:** [January 14, 2025, 5:26am UTC](https://discuss.elastic.co/t/cant-make-mongodb-connection-work-in-logstash-filter/373144 "2025-01-14T05:26:14Z")
I've been fighting this issue for days and keep circling back to the same problem: \[2025-01-13T21:24:32,037\]\[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline error {:pipeline\_id=\>"main", :exception=\>#\*\*\ **Author:** [@Byungsoo\_Kim](https://discuss.elastic.co/u/Byungsoo_Kim)\ **Replies:** 8\ **Last updated:** [January 14, 2025, 12:52am UTC](https://discuss.elastic.co/t/get-security-api-key-does-not-show-role-descriptors/373029 "2025-01-14T00:52:48Z") I am using ES cloud and create api\_key through "API Console" UI on ES cloud. The following is an example of the payload to create an api\_key POST /\_security/api\_key { "name": "midtier-2025", "role\_descriptors": { … --- ## [Index relocation failing at 100% bytes](https://discuss.elastic.co/t/index-relocation-failing-at-100-bytes/372934)
**Author:** [@nisow95612](https://discuss.elastic.co/u/nisow95612)\ **Replies:** 5\ **Last updated:** [January 13, 2025, 11:09pm UTC](https://discuss.elastic.co/t/index-relocation-failing-at-100-bytes/372934 "2025-01-13T23:09:54Z")
After upgrade from 7.17.23 to 7.17.26 I started seeing patterns like this: ILM starts relocating from warm tier to cold in /\_cat/allocation transfer goes to 100% bp nothing in log of source node error in log of target … --- ## [Tips for managing spikes in data sources](https://discuss.elastic.co/t/tips-for-managing-spikes-in-data-sources/373143)
**Author:** [@erikg](https://discuss.elastic.co/u/erikg)\ **Replies:** 0\ **Last updated:** [January 13, 2025, 9:27pm UTC](https://discuss.elastic.co/t/tips-for-managing-spikes-in-data-sources/373143 "2025-01-13T21:27:12Z")
Hello, I was wondering what others do to manage a huge increase in events from a particular data source (spikes). I know there's a rate lime processor on elastic agent but is there other mechanisms used? I have heard o… --- ## [Filebeat8.12 cisco module for ise logs is supported or not?](https://discuss.elastic.co/t/filebeat8-12-cisco-module-for-ise-logs-is-supported-or-not/373043)
**Author:** [@Priyanka\_chauhan](https://discuss.elastic.co/u/Priyanka_chauhan)\ **Replies:** 4\ **Last updated:** [January 13, 2025, 8:28pm UTC](https://discuss.elastic.co/t/filebeat8-12-cisco-module-for-ise-logs-is-supported-or-not/373043 "2025-01-13T20:28:35Z")
I have setup filebeat 8.12 and set kafka input in filebeat input file , since cisco ise logs are coming at kafka topic , ingest pipeline is created for cisco module and filebeat index is created ,ise logs are coming in f… --- ## [Thread Pool Search Size and Query Cache](https://discuss.elastic.co/t/thread-pool-search-size-and-query-cache/373137)
**Author:** [@kdwolf](https://discuss.elastic.co/u/kdwolf)\ **Replies:** 0\ **Last updated:** [January 13, 2025, 4:57pm UTC](https://discuss.elastic.co/t/thread-pool-search-size-and-query-cache/373137 "2025-01-13T16:57:57Z")
We use default values for both search size and query cache. Providing we decide to increase the query cache looking to improve search experience, would one recommend to increase the search size of the thread pool as well… --- ## [Issue with \_prefer\_nodes behavior in Elasticsearch 8.15.1](https://discuss.elastic.co/t/issue-with-prefer-nodes-behavior-in-elasticsearch-8-15-1/373125)
**Author:** [@GabM](https://discuss.elastic.co/u/GabM)\ **Replies:** 0\ **Last updated:** [January 13, 2025, 1:49pm UTC](https://discuss.elastic.co/t/issue-with-prefer-nodes-behavior-in-elasticsearch-8-15-1/373125 "2025-01-13T13:49:24Z")
Hello, I am encountering what seems to be unexpected behavior with the preference option in the /\_search API when using the \_prefer\_nodes value. Here's the setup and the problem: Cluster Configuration: Elasticsearch v… --- ## [Kibana not starting Invalid or unexpected token](https://discuss.elastic.co/t/kibana-not-starting-invalid-or-unexpected-token/357843)
**Author:** [@aafreddy](https://discuss.elastic.co/u/aafreddy)\ **Replies:** 1\ **Last updated:** [January 13, 2025, 12:54pm UTC](https://discuss.elastic.co/t/kibana-not-starting-invalid-or-unexpected-token/357843 "2025-01-13T12:54:45Z")
Hello, this is a simple one server ELK stack. After a server crash kibana is not starting. This is the error from kibana log \[2024-04-20T17:33:03.132+02:00\]\[DEBUG\]\[plugins-discovery\] Successfully discovered plugin "ur… --- ## [ILM Policy Reset for Indices Created by Kubernetes Integration](https://discuss.elastic.co/t/ilm-policy-reset-for-indices-created-by-kubernetes-integration/373119)
**Author:** [@KarimOthman](https://discuss.elastic.co/u/KarimOthman)\ **Replies:** 0\ **Last updated:** [January 13, 2025, 10:00am UTC](https://discuss.elastic.co/t/ilm-policy-reset-for-indices-created-by-kubernetes-integration/373119 "2025-01-13T10:00:12Z")
We are leveraging the Kubernetes integration in Elastic Agents to collect data from multiple Kubernetes clusters. By default, indices created by the Kubernetes integration are assigned the metrics@lifecycle ILM policy, w… --- ## [How to Parsing Nested Json](https://discuss.elastic.co/t/how-to-parsing-nested-json/372879)
**Author:** [@zerratriani](https://discuss.elastic.co/u/zerratriani)\ **Replies:** 3\ **Last updated:** [January 13, 2025, 8:15am UTC](https://discuss.elastic.co/t/how-to-parsing-nested-json/372879 "2025-01-13T08:15:48Z")
So i have log that alreary parse in json. But i want to parsing again that nested? This the log i manipulated cause is sensitive data. And i want to parse again is it inside responsePayload.data {"cif":"XXX","gram":"X… --- ## [Docker ELK stack not moving after licence validation and fails to authenticate with the the .env password](https://discuss.elastic.co/t/docker-elk-stack-not-moving-after-licence-validation-and-fails-to-authenticate-with-the-the-env-password/373053)
**Author:** [@forabraham1](https://discuss.elastic.co/u/forabraham1)\ **Replies:** 3\ **Last updated:** [January 13, 2025, 5:21am UTC](https://discuss.elastic.co/t/docker-elk-stack-not-moving-after-licence-validation-and-fails-to-authenticate-with-the-the-env-password/373053 "2025-01-13T05:21:54Z")
Hi Team, I trying to setup ELK stack in local using docker. I'm having some hard time in bringing Elasticsearch up. Actually elasticsearch starts up but as per logs it didn't move after licence validation and stuck in … --- ## [Memory leak in Kibana 7.10](https://discuss.elastic.co/t/memory-leak-in-kibana-7-10/371070)
**Author:** [@satyam](https://discuss.elastic.co/u/satyam)\ **Replies:** 4\ **Last updated:** [November 29, 2024, 12:59pm UTC](https://discuss.elastic.co/t/memory-leak-in-kibana-7-10/371070 "2024-11-29T12:59:01Z")
We use KIbana dashboard but the older version 7.10. On discover plugin when we keep manipulating filters, after certain time browser crashes with out of memory error. I did my investigation and found out that cell.html … --- ## [SMTP and ServiceDesk integration with Elastic Basic License](https://discuss.elastic.co/t/smtp-and-servicedesk-integration-with-elastic-basic-license/373102)
**Author:** [@Eshwar\_K](https://discuss.elastic.co/u/Eshwar_K)\ **Replies:** 1\ **Last updated:** [January 12, 2025, 7:53pm UTC](https://discuss.elastic.co/t/smtp-and-servicedesk-integration-with-elastic-basic-license/373102 "2025-01-12T19:53:29Z")
Hi Community, I would like to integrate SMTP and Servicedesk for alerts so my question is does Elastic basic license supports integrations with SMTP and Servicedesk? if yes, kindly suggest! Much appreciated for your re… --- ## [Parse single-line json with logstash](https://discuss.elastic.co/t/parse-single-line-json-with-logstash/373096)
**Author:** [@Resort](https://discuss.elastic.co/u/Resort)\ **Replies:** 3\ **Last updated:** [January 12, 2025, 2:48pm UTC](https://discuss.elastic.co/t/parse-single-line-json-with-logstash/373096 "2025-01-12T14:48:21Z")
Hello, I'm looking for assistance with my attempt of passing logs of .json type to Elasticsearch using Logstash. The tricky moment is that the .json file contains one single valid data and is being ignored by Logstash. … --- ## [Restoring snapshot (filesystem) - empty](https://discuss.elastic.co/t/restoring-snapshot-filesystem-empty/372987)
**Author:** [@e-lastic](https://discuss.elastic.co/u/e-lastic)\ **Replies:** 5\ **Last updated:** [January 12, 2025, 9:25am UTC](https://discuss.elastic.co/t/restoring-snapshot-filesystem-empty/372987 "2025-01-12T09:25:44Z")
Hi, TL;DR; Tried to restore a snapshot I did before re-installing the machine but ended up with elasticsearch complaining the snapshot is "empty" What happened: I had to set up a machine from scratch (single instance… --- ## [AttributeError: 'BM25RetrievalStrategy' object has no attribute 'query\_model\_id'](https://discuss.elastic.co/t/attributeerror-bm25retrievalstrategy-object-has-no-attribute-query-model-id/373085)
**Author:** [@yonglie](https://discuss.elastic.co/u/yonglie)\ **Replies:** 0\ **Last updated:** [January 11, 2025, 12:34am UTC](https://discuss.elastic.co/t/attributeerror-bm25retrievalstrategy-object-has-no-attribute-query-model-id/373085 "2025-01-11T00:34:39Z")
I want to run elasticsearch-lab examples chatbot-with-bm25-only-example.ipynb but i got the wrong info ,help me AttributeError: 'BM25RetrievalStrategy' object has no attribute 'query\_model\_id' --- ## [The documentation does not seem to document the minimum JDK runtime requirement for \`org.elasticsearch.plugin:x-pack-sql-jdbc\`](https://discuss.elastic.co/t/the-documentation-does-not-seem-to-document-the-minimum-jdk-runtime-requirement-for-org-elasticsearch-plugin-x-pack-sql-jdbc/372078)
**Author:** [@linghengqian](https://discuss.elastic.co/u/linghengqian)\ **Replies:** 4\ **Last updated:** [January 10, 2025, 9:01pm UTC](https://discuss.elastic.co/t/the-documentation-does-not-seem-to-document-the-minimum-jdk-runtime-requirement-for-org-elasticsearch-plugin-x-pack-sql-jdbc/372078 "2025-01-10T21:01:21Z")
The documentation does not seem to document the minimum JDK runtime requirement for org.elasticsearch.plugin:x-pack-sql-jdbc. I am of course aware that using the JDBC driver requires a trial or paid license for the Elas… --- ## [Elastic Agent Panic with nil pointer on \`null\` processor](https://discuss.elastic.co/t/elastic-agent-panic-with-nil-pointer-on-null-processor/373078)
**Author:** [@BenB196](https://discuss.elastic.co/u/BenB196)\ **Replies:** 0\ **Last updated:** [January 10, 2025, 7:55pm UTC](https://discuss.elastic.co/t/elastic-agent-panic-with-nil-pointer-on-null-processor/373078 "2025-01-10T19:55:08Z")
Hi, I was wondering if anyone else has hit this issue, if I define a null processor in an integration policy, the Elastic Agent will immediately enter a crashloop with the error: panic: runtime error: invalid memory add… --- ## [Moving indices from Hot tier to Cold does not reduce hot tier size](https://discuss.elastic.co/t/moving-indices-from-hot-tier-to-cold-does-not-reduce-hot-tier-size/373001)
**Author:** [@Oolong](https://discuss.elastic.co/u/Oolong)\ **Replies:** 16\ **Last updated:** [January 10, 2025, 6:28pm UTC](https://discuss.elastic.co/t/moving-indices-from-hot-tier-to-cold-does-not-reduce-hot-tier-size/373001 "2025-01-10T18:28:47Z")
Hi Everyone, I have 200GB storage for HOT tier and 500GB for cold. I have so far used up 150GB on the HOT tier and have moved the indices that make up that 150 to Cold tier. My problem is that the Cold tier has increase… --- ## [Parse a log file from esp32](https://discuss.elastic.co/t/parse-a-log-file-from-esp32/373020)
**Author:** [@s0mbra](https://discuss.elastic.co/u/s0mbra)\ **Replies:** 2\ **Last updated:** [January 10, 2025, 5:41pm UTC](https://discuss.elastic.co/t/parse-a-log-file-from-esp32/373020 "2025-01-10T17:41:57Z")
Hi everone! Im starting to develop a project with the ESP32. This project generates a web interface to control outputs on the ESP32. I want to collect the logs and process them through the ELK stack, but I am not being … --- ## [Can I install license in a cluster that has no internet enabled](https://discuss.elastic.co/t/can-i-install-license-in-a-cluster-that-has-no-internet-enabled/373046)
**Author:** [@searchwithme](https://discuss.elastic.co/u/searchwithme)\ **Replies:** 1\ **Last updated:** [January 10, 2025, 5:22pm UTC](https://discuss.elastic.co/t/can-i-install-license-in-a-cluster-that-has-no-internet-enabled/373046 "2025-01-10T17:22:12Z")
Do I need internet to install a license in my ECK environment? there is no mentioned about this in the documentation: Manage licenses in ECK | Elastic Cloud on Kubernetes \[2.16\] | Elastic [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=113) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=115)