# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=20 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 21 --- ## [Thousands of filebeat ingest pipelines / cleanup](https://discuss.elastic.co/t/thousands-of-filebeat-ingest-pipelines-cleanup/381421)
**Author:** [@idl0r](https://discuss.elastic.co/u/idl0r)\ **Replies:** 4\ **Last updated:** [August 29, 2025, 1:38pm UTC](https://discuss.elastic.co/t/thousands-of-filebeat-ingest-pipelines-cleanup/381421 "2025-08-29T13:38:52Z")
Hey, I’m not very familiar yet with ELK/SIEM and I‘m also not sure about the proper category here. So we have like thousands of filebeat ingest pipelines. I assume they’re auto created during updates. Is it safe to del… --- ## [Python: ImportError: cannot import name 'E' from 'elasticsearch.esql'](https://discuss.elastic.co/t/python-importerror-cannot-import-name-e-from-elasticsearch-esql/381204)
**Author:** [@peter9](https://discuss.elastic.co/u/peter9)\ **Replies:** 3\ **Last updated:** [August 29, 2025, 1:02pm UTC](https://discuss.elastic.co/t/python-importerror-cannot-import-name-e-from-elasticsearch-esql/381204 "2025-08-29T13:02:01Z")
I am reading the docs at ES|QL Query Builder | Python There are examples that start with: from elasticsearch.esql import ESQL, E When I try these, I get: ImportError: cannot import name 'E' from 'elasticsearch.esql' … --- ## [No such indices in ElasticSearch](https://discuss.elastic.co/t/no-such-indices-in-elasticsearch/381434)
**Author:** [@levi19](https://discuss.elastic.co/u/levi19)\ **Replies:** 0\ **Last updated:** [August 29, 2025, 4:01am UTC](https://discuss.elastic.co/t/no-such-indices-in-elasticsearch/381434 "2025-08-29T04:01:31Z")
We are currently running an Elasticsearch & Kibana (8.14.3) cluster with: 3 master nodes and 6 data nodes Also, we are using Metricbeat ver 8.14.3 for monitoring the cluster ( we are enabled xpack.monitoring featur… --- ## [Integrate Keycloak OIDC authentication with Elasticsearch and Kibana](https://discuss.elastic.co/t/integrate-keycloak-oidc-authentication-with-elasticsearch-and-kibana/381370)
**Author:** [@M311ow](https://discuss.elastic.co/u/M311ow)\ **Replies:** 1\ **Last updated:** [August 29, 2025, 1:37am UTC](https://discuss.elastic.co/t/integrate-keycloak-oidc-authentication-with-elasticsearch-and-kibana/381370 "2025-08-29T01:37:38Z")
Hi, I’m trying to integrate Keycloak OIDC with Elasticsearch and Kibana. From what I understand, Elasticsearch has three types of nodes Master node Data node Ingest node If we want to set up Keycloak, on which node s… --- ## [Closing connection Netty4HttpChannel Error](https://discuss.elastic.co/t/closing-connection-netty4httpchannel-error/381108)
**Author:** [@NikoCosmico01](https://discuss.elastic.co/u/NikoCosmico01)\ **Replies:** 2\ **Last updated:** [August 28, 2025, 12:41pm UTC](https://discuss.elastic.co/t/closing-connection-netty4httpchannel-error/381108 "2025-08-28T12:41:03Z")
Hi, I have an ELK on-premise setup deployed with docker (x3 nodes, x1 Kibana, 1x fleet-server). Recently, I decided to generate self-signed certificated to ensure SSL and HTTPS communications. The only certificate sign… --- ## [Adding volumeAttributesClassName is forbidden](https://discuss.elastic.co/t/adding-volumeattributesclassname-is-forbidden/381124)
**Author:** [@jackchi](https://discuss.elastic.co/u/jackchi)\ **Replies:** 1\ **Last updated:** [August 28, 2025, 12:02am UTC](https://discuss.elastic.co/t/adding-volumeattributesclassname-is-forbidden/381124 "2025-08-28T00:02:25Z")
After the cluster’s nodeSets are created. I want to specific a volumeAttributesClassName however, the eck operator’s admission webhook is preventing me. Failed to save resource: (Forbidden) admission webhook "elastic-e… --- ## [Kibana Discover highligth fragment size](https://discuss.elastic.co/t/kibana-discover-highligth-fragment-size/381309)
**Author:** [@rodrigopaulodecastro](https://discuss.elastic.co/u/rodrigopaulodecastro)\ **Replies:** 2\ **Last updated:** [August 27, 2025, 11:47pm UTC](https://discuss.elastic.co/t/kibana-discover-highligth-fragment-size/381309 "2025-08-27T23:47:56Z")
Hi there, when using Elasticsearch API it is possible to define de highlight fragment size to have a better visualization of the highlight portion, example: { "query": { "match": { "text\_field": "search ter… --- ## [Serverless project APIs returning 401's](https://discuss.elastic.co/t/serverless-project-apis-returning-401s/381402)
**Author:** [@danN](https://discuss.elastic.co/u/danN)\ **Replies:** 1\ **Last updated:** [August 27, 2025, 8:14pm UTC](https://discuss.elastic.co/t/serverless-project-apis-returning-401s/381402 "2025-08-27T20:14:59Z")
Hello, I created a serverless project in elastic the other day as I wanted to run some API queries. I then clicked on the ? icon in the top right of my portal and selected “Connection Details”, copied the elasticsearch … --- ## [Issue with kibana/es after update to 9.1.0](https://discuss.elastic.co/t/issue-with-kibana-es-after-update-to-9-1-0/381265)
**Author:** [@kim.ae](https://discuss.elastic.co/u/kim.ae)\ **Replies:** 4\ **Last updated:** [August 27, 2025, 3:48pm UTC](https://discuss.elastic.co/t/issue-with-kibana-es-after-update-to-9-1-0/381265 "2025-08-27T15:48:48Z")
Hello Community and Elastic team, We are using the ECK community version Installed in a AKS with k8s version 1.32.6. We just updated the installation to 9.1.0 from 8.15.x. No we are unable to create index templates, it… --- ## [400-Bad request](https://discuss.elastic.co/t/400-bad-request/381344)
**Author:** [@tarun-ghcp](https://discuss.elastic.co/u/tarun-ghcp)\ **Replies:** 2\ **Last updated:** [August 27, 2025, 9:35am UTC](https://discuss.elastic.co/t/400-bad-request/381344 "2025-08-27T09:35:03Z")
Below is my payload but it works only when query\_body is empty. I couldn’t figure out whats wrong inside it. I have a Docker container for the MCP server and it throws Bad Request every time. payload = { "jsonrpc": "2.… --- ## [Using the same snapshot repository for SLM and Frozen Tier](https://discuss.elastic.co/t/using-the-same-snapshot-repository-for-slm-and-frozen-tier/381345)
**Author:** [@gueguet57](https://discuss.elastic.co/u/gueguet57)\ **Replies:** 1\ **Last updated:** [August 27, 2025, 7:55am UTC](https://discuss.elastic.co/t/using-the-same-snapshot-repository-for-slm-and-frozen-tier/381345 "2025-08-27T07:55:30Z")
Hello all, In my Elastic setup (ECK), I’ve already created a snapshot repository that I use for the Frozen data tier and searchable snapshots. Now, I’d like to configure Snapshot Lifecycle Management (SLM) to automatic… --- ## [pyspark.sql.Observation.get blocking on pyspark.sql.Dataframe.writer.save()](https://discuss.elastic.co/t/pyspark-sql-observation-get-blocking-on-pyspark-sql-dataframe-writer-save/381362)
**Author:** [@DaveLiddy](https://discuss.elastic.co/u/DaveLiddy)\ **Replies:** 0\ **Last updated:** [August 27, 2025, 4:43am UTC](https://discuss.elastic.co/t/pyspark-sql-observation-get-blocking-on-pyspark-sql-dataframe-writer-save/381362 "2025-08-27T04:43:26Z")
I’m trying to add some metrics to a pyspark pipeline feeding an index. I’ve found that the get operation blocks on the write, which I can workaround by adding in another request on the dataframe, such as the count below.… --- ## [400 Bad request](https://discuss.elastic.co/t/400-bad-request/381357)
**Author:** [@tarun-ghcp](https://discuss.elastic.co/u/tarun-ghcp)\ **Replies:** 1\ **Last updated:** [August 27, 2025, 3:31am UTC](https://discuss.elastic.co/t/400-bad-request/381357 "2025-08-27T03:31:14Z")
I am using Docker container for MCP server. When running my Python script to do a knn search, the server throws 400 Bad request. But when request is sent without anything inside query\_body, it gives me the result. I coul… --- ## ["unknown field \[pivot\]" error](https://discuss.elastic.co/t/unknown-field-pivot-error/381330)
**Author:** [@meatwad](https://discuss.elastic.co/u/meatwad)\ **Replies:** 1\ **Last updated:** [August 26, 2025, 6:13am UTC](https://discuss.elastic.co/t/unknown-field-pivot-error/381330 "2025-08-26T06:13:34Z")
Hi there, I’m trying to follow along with this post to try and enrich FortiGate logs: But when I do the initial POST \_transform/fortivpntunnels/\_update, I get the following error: { "error": { "root\_cause": \[ … --- ## [Ingest Pipeline Parsing](https://discuss.elastic.co/t/ingest-pipeline-parsing/381238)
**Author:** [@Ankita\_Pachauri](https://discuss.elastic.co/u/Ankita_Pachauri)\ **Replies:** 7\ **Last updated:** [August 26, 2025, 3:59am UTC](https://discuss.elastic.co/t/ingest-pipeline-parsing/381238 "2025-08-26T03:59:23Z")
Hi Folks, Can someone help me parsing the below logs using ingest pipeline? //Ankita --- ## [Ingest Pipeline KV Processor](https://discuss.elastic.co/t/ingest-pipeline-kv-processor/367252)
**Author:** [@Ankita\_Pachauri](https://discuss.elastic.co/u/Ankita_Pachauri)\ **Replies:** 9\ **Last updated:** [August 26, 2025, 1:38am UTC](https://discuss.elastic.co/t/ingest-pipeline-kv-processor/367252 "2025-08-26T01:38:26Z")
Hi Team, I am using custom logs integration to receive data via an elastic agent(ELK Version 8.13.4). The data looks something like this. "message": " URL : https://www.google.com\\n Action Type … --- ## [Auto-scaling the number of replicas](https://discuss.elastic.co/t/auto-scaling-the-number-of-replicas/381318)
**Author:** [@Travis\_Andelin](https://discuss.elastic.co/u/Travis_Andelin)\ **Replies:** 4\ **Last updated:** [August 25, 2025, 8:02pm UTC](https://discuss.elastic.co/t/auto-scaling-the-number-of-replicas/381318 "2025-08-25T20:02:45Z")
We typically provision our elasticsearch clusters with a lot of indices, and one replica. The search traffic can vary based on the index. Sometimes we get a burst of search traffic on one index (usually a node gets CPU … --- ## [How to delete old system indices?](https://discuss.elastic.co/t/how-to-delete-old-system-indices/381146)
**Author:** [@garethhumphriesgkc](https://discuss.elastic.co/u/garethhumphriesgkc)\ **Replies:** 2\ **Last updated:** [August 25, 2025, 9:04pm UTC](https://discuss.elastic.co/t/how-to-delete-old-system-indices/381146 "2025-08-25T21:04:51Z")
I have a bunch of old indices from old versions I’d like to get rid of to free up disk and shards, e.g.: .kibana\_task\_manager\_7.15.2\_001 .kibana\_task\_manager\_7.17.5\_001 .kibana\_task\_manager\_8.3.3\_001 .kibana\_task\_manag… --- ## [Upgraded to a new PC and now cant see indicies version 9 - Windows](https://discuss.elastic.co/t/upgraded-to-a-new-pc-and-now-cant-see-indicies-version-9-windows/381296)
**Author:** [@Nate9872](https://discuss.elastic.co/u/Nate9872)\ **Replies:** 1\ **Last updated:** [August 25, 2025, 8:12am UTC](https://discuss.elastic.co/t/upgraded-to-a-new-pc-and-now-cant-see-indicies-version-9-windows/381296 "2025-08-25T08:12:02Z")
Hello, As the title suggests, I copied over my entire data folder (9TB) along with my elasticsearch and kibana folders to a new machine. Nothing else changed, all drive letters and folders are still the same, however, n… --- ## [Log4j2 won't send logs to external log server after upgrade to 8.18 (Java 24)](https://discuss.elastic.co/t/log4j2-wont-send-logs-to-external-log-server-after-upgrade-to-8-18-java-24/381300)
**Author:** [@baneinc](https://discuss.elastic.co/u/baneinc)\ **Replies:** 0\ **Last updated:** [August 25, 2025, 7:18am UTC](https://discuss.elastic.co/t/log4j2-wont-send-logs-to-external-log-server-after-upgrade-to-8-18-java-24/381300 "2025-08-25T07:18:39Z")
Hi Folks, for Elasticsearch logging we use log4j2, which is configured to send logs to log server. We had issues, that we fixed implementing .java.policy file (as decribed in this topic: Log4j framework can no longer r… --- ## [Thread Pool stats show queued items despite not being full](https://discuss.elastic.co/t/thread-pool-stats-show-queued-items-despite-not-being-full/381298)
**Author:** [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Replies:** 0\ **Last updated:** [August 25, 2025, 6:39am UTC](https://discuss.elastic.co/t/thread-pool-stats-show-queued-items-despite-not-being-full/381298 "2025-08-25T06:39:47Z")
Good day, in our monitoring data I can see that one cluster with search load (dozens of queries per second) shows slightly confusing thread pool stats. I can regularly see searches being queued while the thread pool si… --- ## [Sizing for ElasticStack in on-Prem environment](https://discuss.elastic.co/t/sizing-for-elasticstack-in-on-prem-environment/381269)
**Author:** [@Eshwar\_K](https://discuss.elastic.co/u/Eshwar_K)\ **Replies:** 13\ **Last updated:** [August 25, 2025, 5:14am UTC](https://discuss.elastic.co/t/sizing-for-elasticstack-in-on-prem-environment/381269 "2025-08-25T05:14:23Z")
Hi Elastic Community, I am in a preparation of hardware sizing for one of my customer for log analysis for below requirement: Current Data size: 400GB Daily data ingestion: 20GB Retention Period: 30days So, as per t… --- ## [Shared file system repository and Roles](https://discuss.elastic.co/t/shared-file-system-repository-and-roles/381283)
**Author:** [@GiorgioS13](https://discuss.elastic.co/u/GiorgioS13)\ **Replies:** 6\ **Last updated:** [August 23, 2025, 5:58pm UTC](https://discuss.elastic.co/t/shared-file-system-repository-and-roles/381283 "2025-08-23T17:58:05Z")
Hi I have a cold node with the appropriate roles, but it does not include the data role. My goal is to set up a Shared File System repository. I have a dedicated disk attached to the cold node, but I cannot create the … --- ## [Make a duration field based of field's value](https://discuss.elastic.co/t/make-a-duration-field-based-of-fields-value/381058)
**Author:** [@erikg](https://discuss.elastic.co/u/erikg)\ **Replies:** 3\ **Last updated:** [August 23, 2025, 5:48am UTC](https://discuss.elastic.co/t/make-a-duration-field-based-of-fields-value/381058 "2025-08-23T05:48:16Z")
Hello, I am ingesting a field that changes value and I would like to capture a duration of the value, is this possible with Elastic? For example: 13:00 - 1 document - field1: “Offline” , newfield\_duration: 0min 13:0… --- ## [Elasticsearch index creation from Python, results in error, for known mapping](https://discuss.elastic.co/t/elasticsearch-index-creation-from-python-results-in-error-for-known-mapping/381271)
**Author:** [@dave\_espinosa\_qs](https://discuss.elastic.co/u/dave_espinosa_qs)\ **Replies:** 1\ **Last updated:** [August 23, 2025, 3:18am UTC](https://discuss.elastic.co/t/elasticsearch-index-creation-from-python-results-in-error-for-known-mapping/381271 "2025-08-23T03:18:04Z")
I am trying to create an index from a Python script, but I am getting an error. Find the whole description here. (IMPORTANT! I have not uploaded anything in this website, as every time I attempted, the attachments kept … --- ## [Lifecycle Policy](https://discuss.elastic.co/t/lifecycle-policy/381211)
**Author:** [@GiorgioS13](https://discuss.elastic.co/u/GiorgioS13)\ **Replies:** 10\ **Last updated:** [August 22, 2025, 10:08pm UTC](https://discuss.elastic.co/t/lifecycle-policy/381211 "2025-08-22T22:08:22Z")
I created a lifecycle policy in Kibana and attached it to a data stream, for example: logs-system.syslog-default However, the indices it created, such as: .ds-logs-system.syslog-default-2025.08.21-000001 are still sh… --- ## [How to query a sum and it's percentage in the same query?](https://discuss.elastic.co/t/how-to-query-a-sum-and-its-percentage-in-the-same-query/381229)
**Author:** [@khat33b](https://discuss.elastic.co/u/khat33b)\ **Replies:** 4\ **Last updated:** [August 22, 2025, 2:49pm UTC](https://discuss.elastic.co/t/how-to-query-a-sum-and-its-percentage-in-the-same-query/381229 "2025-08-22T14:49:44Z")
I am writing an ES|QL to find out the sum and of a field grouped by another field and also it’s percentage by other sums of that field. How do I write the query? FROM test\_index | WHERE start\_time \>= DATE\_PARSE("yyyy-M… --- ## [Inconsistent behavior when querying host.ip field: ORDER BY ASC and WHERE IS NOT NULL fail with "Arrays not supported"](https://discuss.elastic.co/t/inconsistent-behavior-when-querying-host-ip-field-order-by-asc-and-where-is-not-null-fail-with-arrays-not-supported/381187)
**Author:** [@wwxb](https://discuss.elastic.co/u/wwxb)\ **Replies:** 5\ **Last updated:** [August 22, 2025, 3:16am UTC](https://discuss.elastic.co/t/inconsistent-behavior-when-querying-host-ip-field-order-by-asc-and-where-is-not-null-fail-with-arrays-not-supported/381187 "2025-08-22T03:16:56Z")
Hello Elasticsearch team, I encountered an unexpected issue when querying a field in my index using the SQL API. Index: xuji\_test Mapping excerpt (simplified): POST /\_sql?format=json { "query": """ DESCRIBE xuji\_… --- ## [Missing documents](https://discuss.elastic.co/t/missing-documents/380946)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 32\ **Last updated:** [August 21, 2025, 7:34pm UTC](https://discuss.elastic.co/t/missing-documents/380946 "2025-08-21T19:34:27Z")
I have following setup and I am missing documents. it is very hard to pinpoint at this time. that is why posting here to get some idea 1000 machine running metricbeat sending metric every min to HA proxy Ha Proxy then … --- ## [Fleet server issue in cluster](https://discuss.elastic.co/t/fleet-server-issue-in-cluster/381183)
**Author:** [@GiorgioS13](https://discuss.elastic.co/u/GiorgioS13)\ **Replies:** 4\ **Last updated:** [August 21, 2025, 12:13pm UTC](https://discuss.elastic.co/t/fleet-server-issue-in-cluster/381183 "2025-08-21T12:13:29Z")
Hello, I need your assistance. I set up an Elastic cluster using the enrollment token method, including Kibana, and everything is working correctly the data nodes have their respective roles, etc. The issue is with th… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=19) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=21)