# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=428 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 429 --- ## [Help me - How can I configure firebeat to read log files from log folder](https://discuss.elastic.co/t/help-me-how-can-i-configure-firebeat-to-read-log-files-from-log-folder/342599)
**Author:** [@Huy\_Hoang\_Le](https://discuss.elastic.co/u/Huy_Hoang_Le)\ **Replies:** 3\ **Last updated:** [September 11, 2023, 3:49am UTC](https://discuss.elastic.co/t/help-me-how-can-i-configure-firebeat-to-read-log-files-from-log-folder/342599 "2023-09-11T03:49:04Z")
I have an app that write log to .log file to a log folder that mounted from container to host. I'm very new to ELK overall so I want filebeat to read .log files generated in that log folder. Here is my filebeat config f… --- ## [Generate monthly report from elastic index](https://discuss.elastic.co/t/generate-monthly-report-from-elastic-index/342691)
**Author:** [@JasonREC](https://discuss.elastic.co/u/JasonREC)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 2:56am UTC](https://discuss.elastic.co/t/generate-monthly-report-from-elastic-index/342691 "2023-09-11T02:56:54Z")
Hi, I want to generate a monthly report from my elastic index Currently, I have an index which contains time series data from July to now(Sep) after September, I want to generate the summary report of July, Aug And Se… --- ## [Kql search bar not applying](https://discuss.elastic.co/t/kql-search-bar-not-applying/342687)
**Author:** [@Farah\_Bhr](https://discuss.elastic.co/u/Farah_Bhr)\ **Replies:** 0\ **Last updated:** [September 10, 2023, 10:13pm UTC](https://discuss.elastic.co/t/kql-search-bar-not-applying/342687 "2023-09-10T22:13:24Z")
Hello , I am developing a cutomized kibana plugin and I added the kql search bar on the top , but my problem is that it is not applying the search bar on the visualized hits Here is my try const doAsyncSearch = async … --- ## [How to Insert 50Million documents per 30sec in elasticsearch cluster?](https://discuss.elastic.co/t/how-to-insert-50million-documents-per-30sec-in-elasticsearch-cluster/342669)
**Author:** [@coldcoder8502](https://discuss.elastic.co/u/coldcoder8502)\ **Replies:** 6\ **Last updated:** [September 10, 2023, 3:52pm UTC](https://discuss.elastic.co/t/how-to-insert-50million-documents-per-30sec-in-elasticsearch-cluster/342669 "2023-09-10T15:52:11Z")
Hi all, Iam facing difficulty to insert 50million documents per 30 sec from source to Elasticsearch cluster. I have 7 sources so total = 350million documents per 30 sec. I have one machine with 500GB Ram, 500Tb storage … --- ## [How does Logstash know the host.hostname field?](https://discuss.elastic.co/t/how-does-logstash-know-the-host-hostname-field/342666)
**Author:** [@Poubelle\_Dirty](https://discuss.elastic.co/u/Poubelle_Dirty)\ **Replies:** 4\ **Last updated:** [September 10, 2023, 3:17pm UTC](https://discuss.elastic.co/t/how-does-logstash-know-the-host-hostname-field/342666 "2023-09-10T15:17:26Z")
Hello, I don't find the information and I think it's about ECS. I have a really simple config on logtash that uses syslog input, grok parsing and inject into elastic cluster. input { syslog { host =\> "0.0.0.0" … --- ## [Elasitc search date issue](https://discuss.elastic.co/t/elasitc-search-date-issue/342682)
**Author:** [@varun\_bisht](https://discuss.elastic.co/u/varun_bisht)\ **Replies:** 0\ **Last updated:** [September 10, 2023, 1:26pm UTC](https://discuss.elastic.co/t/elasitc-search-date-issue/342682 "2023-09-10T13:26:48Z")
Hi i am using this curl to create mapping - curl --cacert http\_ca.crt -u elastic:$ELASTIC\_PASSWORD -X PUT "https://DNS:9200/elasticdate-6?pretty=" -H 'content-type: application/json' -H 'user-agent: -lContent-Type: app… --- ## [Best way to write from Apache Spark to ECK](https://discuss.elastic.co/t/best-way-to-write-from-apache-spark-to-eck/342480)
**Author:** [@krezno](https://discuss.elastic.co/u/krezno)\ **Replies:** 2\ **Last updated:** [September 9, 2023, 9:26pm UTC](https://discuss.elastic.co/t/best-way-to-write-from-apache-spark-to-eck/342480 "2023-09-09T21:26:04Z")
Hello I have a lot of batch processes that write large batches of data to elastic in scheduled intervals. Currently we are writing to elastic using the es-hadoop library. From what I understand when writing to an ECK in… --- ## [Security\_exception: missing authentication credentials for REST request](https://discuss.elastic.co/t/security-exception-missing-authentication-credentials-for-rest-request/342664)
**Author:** [@agonzalez](https://discuss.elastic.co/u/agonzalez)\ **Replies:** 0\ **Last updated:** [September 9, 2023, 6:03pm UTC](https://discuss.elastic.co/t/security-exception-missing-authentication-credentials-for-rest-request/342664 "2023-09-09T18:03:09Z")
After upgrading elastic pods using docker images from 8.4.1 to 8.9.1 i am getting this error in kibana always. Any idea why? and how to solve? \[2023-09-09T17:58:21.908+00:00\]\[WARN \]\[plugins.security.authenticator\] Sess… --- ## [Endpoint \_cat/indices doesn't work when "license expired"?](https://discuss.elastic.co/t/endpoint-cat-indices-doesnt-work-when-license-expired/342644)
**Author:** [@Mathemaphysics](https://discuss.elastic.co/u/Mathemaphysics)\ **Replies:** 4\ **Last updated:** [September 9, 2023, 3:42pm UTC](https://discuss.elastic.co/t/endpoint-cat-indices-doesnt-work-when-license-expired/342644 "2023-09-09T15:42:32Z")
One day, no countdown, I was told after adding data to my ES server through kibana that I can't view indices. What do I have to do just to use the \_cat/indices endpoint? This is an incredible mess I'm in because of this… --- ## [Estimating max. search throughput that can be achieved from a cluster](https://discuss.elastic.co/t/estimating-max-search-throughput-that-can-be-achieved-from-a-cluster/342661)
**Author:** [@Dhineshkumar\_R](https://discuss.elastic.co/u/Dhineshkumar_R)\ **Replies:** 0\ **Last updated:** [September 9, 2023, 3:25pm UTC](https://discuss.elastic.co/t/estimating-max-search-throughput-that-can-be-achieved-from-a-cluster/342661 "2023-09-09T15:25:55Z")
Hello Folks, I'm trying to compute the max. search throughput I can achieve from a given cluster. Following is my cluster settings 27 data nodes(Mem:384GB and 48 vCPUs) each having 17 shards with 0 replica attached t… --- ## [BadRequestError(400, 'search\_phase\_execution\_exception', 'runtime error') when trying to do l2 similarity search](https://discuss.elastic.co/t/badrequesterror-400-search-phase-execution-exception-runtime-error-when-trying-to-do-l2-similarity-search/342649)
**Author:** [@longlegs](https://discuss.elastic.co/u/longlegs)\ **Replies:** 0\ **Last updated:** [September 9, 2023, 8:25am UTC](https://discuss.elastic.co/t/badrequesterror-400-search-phase-execution-exception-runtime-error-when-trying-to-do-l2-similarity-search/342649 "2023-09-09T08:25:15Z")
Hello, Im trying to implement a vector similarity search for face recognition using Elasticsearch in python. This is my mapping for creating an index: mapping = { "mappings": { "properties": { "… --- ## [ElasticSearch + Kibana + Logstash Config in Windows](https://discuss.elastic.co/t/elasticsearch-kibana-logstash-config-in-windows/342550)
**Author:** [@dfir](https://discuss.elastic.co/u/dfir)\ **Replies:** 1\ **Last updated:** [September 8, 2023, 10:10pm UTC](https://discuss.elastic.co/t/elasticsearch-kibana-logstash-config-in-windows/342550 "2023-09-08T22:10:09Z")
I am working on configuring Elasticsearch with Kibana and Logstash. My goal is to be able to import AWS S3 Bucket Access logs, cloudtrail and other aws logs into Elastic either by manual import or via logstash. When I … --- ## [Fix to libbeats to split bulk requests which are too large, not working in Elastic Agent 8.9.0?](https://discuss.elastic.co/t/fix-to-libbeats-to-split-bulk-requests-which-are-too-large-not-working-in-elastic-agent-8-9-0/342136)
**Author:** [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\ **Replies:** 6\ **Last updated:** [September 8, 2023, 8:45pm UTC](https://discuss.elastic.co/t/fix-to-libbeats-to-split-bulk-requests-which-are-too-large-not-working-in-elastic-agent-8-9-0/342136 "2023-09-08T20:45:22Z")
In Beats, I see that this commit was merged in March 2023: " Split large batches on error instead of dropping them" PR 34911 I think that PR 34911 changed the Publish() logic: If I look here: func (client \*Client) … --- ## [Install Winlogbeat with user account](https://discuss.elastic.co/t/install-winlogbeat-with-user-account/342642)
**Author:** [@JJ007](https://discuss.elastic.co/u/JJ007)\ **Replies:** 0\ **Last updated:** [September 8, 2023, 8:29pm UTC](https://discuss.elastic.co/t/install-winlogbeat-with-user-account/342642 "2023-09-08T20:29:22Z")
Hi, I am trying to install winlogbeat and run as user who is not an Adminsitrator Saw a link - request for enhancement (\[Winlogbeat\] Document minimum permissions for Windows service user · Issue #15773 · elastic/beats … --- ## [Logstash 7.17 keystore - get value by variable key](https://discuss.elastic.co/t/logstash-7-17-keystore-get-value-by-variable-key/342446)
**Author:** [@bonyolult](https://discuss.elastic.co/u/bonyolult)\ **Replies:** 4\ **Last updated:** [September 8, 2023, 8:24pm UTC](https://discuss.elastic.co/t/logstash-7-17-keystore-get-value-by-variable-key/342446 "2023-09-08T20:24:59Z")
Hello, long story short: i must process logs that are encryped with EC keys. The customer requires the keys' passwords to be stored in Logstash keystore where the key is the p12's serial and the value is the password. T… --- ## [Filebeat filtering incoming syslogs?](https://discuss.elastic.co/t/filebeat-filtering-incoming-syslogs/342458)
**Author:** [@artschooldropout](https://discuss.elastic.co/u/artschooldropout)\ **Replies:** 5\ **Last updated:** [September 8, 2023, 7:16pm UTC](https://discuss.elastic.co/t/filebeat-filtering-incoming-syslogs/342458 "2023-09-08T19:16:01Z")
I'm setting up Filebeat (8.9) on an Elasticsearch (8.9) instance, and it looks like Filebeat is filtering logs from external hosts. Here's the relevant section from my Filebeat config: - type: syslog format: auto p… --- ## [What happens when Redis output plugin can't deliver a message?](https://discuss.elastic.co/t/what-happens-when-redis-output-plugin-cant-deliver-a-message/342630)
**Author:** [@noobiewan](https://discuss.elastic.co/u/noobiewan)\ **Replies:** 2\ **Last updated:** [September 8, 2023, 6:49pm UTC](https://discuss.elastic.co/t/what-happens-when-redis-output-plugin-cant-deliver-a-message/342630 "2023-09-08T18:49:17Z")
Hello there, I'm trying to understand what happens when an output plugin can't deliver a message. We are using logstash-output-redis to send batched messages to Redis and I would like to understand what happens if Redis… --- ## [Deploying Elasticsearch latest version in Anthos on-prem K8s cluster](https://discuss.elastic.co/t/deploying-elasticsearch-latest-version-in-anthos-on-prem-k8s-cluster/342636)
**Author:** [@Esakki](https://discuss.elastic.co/u/Esakki)\ **Replies:** 0\ **Last updated:** [September 8, 2023, 5:30pm UTC](https://discuss.elastic.co/t/deploying-elasticsearch-latest-version-in-anthos-on-prem-k8s-cluster/342636 "2023-09-08T17:30:03Z")
Hello Experts, I'm new to Elasticsearch, I want to deploy elasticsearch v 8.9.1 into Anthos on-pre mK8s cluster, and I used the attached .yaml files for the same, I have the below questions. Is the attached yaml files… --- ## [Grok pattern to account for #](https://discuss.elastic.co/t/grok-pattern-to-account-for/342616)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 1\ **Last updated:** [September 8, 2023, 4:46pm UTC](https://discuss.elastic.co/t/grok-pattern-to-account-for/342616 "2023-09-08T16:46:26Z")
I have log messages coming in and a few of them have one of three special characters: @, -, or #. How can I get grok to ignore the hashtag without removing it from the new field? Below is the code snippet I'm using. It… --- ## [ECK 8.7 Fleet managed Agents, Create Policy, Add Integrations & Agent enrollment via code](https://discuss.elastic.co/t/eck-8-7-fleet-managed-agents-create-policy-add-integrations-agent-enrollment-via-code/342626)
**Author:** [@mayur.kadam](https://discuss.elastic.co/u/mayur.kadam)\ **Replies:** 0\ **Last updated:** [September 8, 2023, 3:26pm UTC](https://discuss.elastic.co/t/eck-8-7-fleet-managed-agents-create-policy-add-integrations-agent-enrollment-via-code/342626 "2023-09-08T15:26:33Z")
We have ECK (2.7) installed on our AKS cluster and used Elastic agents for policy orchestration managed by fleet server. The Policy creation and addition of Integration to the Policy is currently done via UI console and … --- ## [Remove event.original from logstash as it comes in every document of logstash version 8.8.2(ECS)](https://discuss.elastic.co/t/remove-event-original-from-logstash-as-it-comes-in-every-document-of-logstash-version-8-8-2-ecs/342612)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 2\ **Last updated:** [September 8, 2023, 3:12pm UTC](https://discuss.elastic.co/t/remove-event-original-from-logstash-as-it-comes-in-every-document-of-logstash-version-8-8-2-ecs/342612 "2023-09-08T15:12:18Z")
Hello All, I am facing major issue with logstash after migration from 7.9.1 to 8.8.2 version. Elasticsearch/Logstash 8.X version has ECS compatibility enabled by default.This adds new field event.original in every do… --- ## [Elastic Agent Standalone: Does agent reload when items in inputs.d/ are updated?](https://discuss.elastic.co/t/elastic-agent-standalone-does-agent-reload-when-items-in-inputs-d-are-updated/342621)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 0\ **Last updated:** [September 8, 2023, 3:08pm UTC](https://discuss.elastic.co/t/elastic-agent-standalone-does-agent-reload-when-items-in-inputs-d-are-updated/342621 "2023-09-08T15:08:42Z")
A quick question. I'm deploying Elastic Agent standalone to our Kubernetes environment. I am saving the inputs as individual files in input.d/ and would like to mount that directory from a secret, since secrets automatic… --- ## [Heartbeat No matching indices found for index pattern heartbeat\*](https://discuss.elastic.co/t/heartbeat-no-matching-indices-found-for-index-pattern-heartbeat/342345)
**Author:** [@BenKenobi](https://discuss.elastic.co/u/BenKenobi)\ **Replies:** 3\ **Last updated:** [September 8, 2023, 2:24pm UTC](https://discuss.elastic.co/t/heartbeat-no-matching-indices-found-for-index-pattern-heartbeat/342345 "2023-09-08T14:24:21Z")
Hi, I have installed and configured heartbeat according your documentation. I have created a cfg-file tcpdemo.yml and it looks like this: Elastic and Kibana run on the same cluster and heartbeat is installed on anoth… --- ## [Granular access to diferent queries in the index](https://discuss.elastic.co/t/granular-access-to-diferent-queries-in-the-index/342615)
**Author:** [@Mihai-CMM](https://discuss.elastic.co/u/Mihai-CMM)\ **Replies:** 2\ **Last updated:** [September 8, 2023, 2:18pm UTC](https://discuss.elastic.co/t/granular-access-to-diferent-queries-in-the-index/342615 "2023-09-08T14:18:14Z")
Hello all, Can you please help with this question (saw some variants Setup access dashboard per user) but not quite what i want or maybe google did not liked my question enough: I have a VectorDev that sends all k8s l… --- ## [Kibana 6.8.13 User Read Only Access](https://discuss.elastic.co/t/kibana-6-8-13-user-read-only-access/342613)
**Author:** [@Jinn](https://discuss.elastic.co/u/Jinn)\ **Replies:** 1\ **Last updated:** [September 8, 2023, 1:50pm UTC](https://discuss.elastic.co/t/kibana-6-8-13-user-read-only-access/342613 "2023-09-08T13:50:13Z")
We use Amazon OpenSearch Service to configure user access to Kibana 6.8.13. I am trying to assign user read only access. When user is assigned with "ESHttpGet" only action or "ESHttpGet" and "ESHttpHead" actions (which … --- ## [EsHadoopIllegalArgumentException: Cannot detect ES version](https://discuss.elastic.co/t/eshadoopillegalargumentexception-cannot-detect-es-version/342593)
**Author:** [@Piyush\_Jain](https://discuss.elastic.co/u/Piyush_Jain)\ **Replies:** 1\ **Last updated:** [September 8, 2023, 1:33pm UTC](https://discuss.elastic.co/t/eshadoopillegalargumentexception-cannot-detect-es-version/342593 "2023-09-08T13:33:00Z")
I'm getting error "EsHadoopIllegalArgumentException: Cannot detect ES version-typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setti… --- ## [S3 Intelligent-Tiering class with Deep Archive Access tier for snapshots is working?](https://discuss.elastic.co/t/s3-intelligent-tiering-class-with-deep-archive-access-tier-for-snapshots-is-working/342122)
**Author:** [@mihai1](https://discuss.elastic.co/u/mihai1)\ **Replies:** 6\ **Last updated:** [September 8, 2023, 12:41pm UTC](https://discuss.elastic.co/t/s3-intelligent-tiering-class-with-deep-archive-access-tier-for-snapshots-is-working/342122 "2023-09-08T12:41:56Z")
We're currently utilizing Elasticsearch version 8.8.1 and storing our daily snapshots in S3 using the 'Intelligent\_Tiering' storage class. In an effort to optimize costs, based on AWS's documentation on Intelligent Tieri… --- ## [Kibanan 8.9.2. Not loading after upgrade from 8.9.1. because its MIME type ('text/html') is not executable, and strict MIME type checking is enabled](https://discuss.elastic.co/t/kibanan-8-9-2-not-loading-after-upgrade-from-8-9-1-because-its-mime-type-text-html-is-not-executable-and-strict-mime-type-checking-is-enabled/342607)
**Author:** [@deepfusion](https://discuss.elastic.co/u/deepfusion)\ **Replies:** 0\ **Last updated:** [September 8, 2023, 12:23pm UTC](https://discuss.elastic.co/t/kibanan-8-9-2-not-loading-after-upgrade-from-8-9-1-because-its-mime-type-text-html-is-not-executable-and-strict-mime-type-checking-is-enabled/342607 "2023-09-08T12:23:23Z")
Hi, I have just upgraded on my ubuntu 22.04 server all elastic stack to 8.9.2 from 8.9.1 and now I can no longer log-in because I get these errors: Refused to execute script from 'https://kibana.\*\*\*\*\*\*\*/login?next=%2F6… --- ## [Removing fields from logstash](https://discuss.elastic.co/t/removing-fields-from-logstash/341782)
**Author:** [@bharti](https://discuss.elastic.co/u/bharti)\ **Replies:** 66\ **Last updated:** [September 8, 2023, 11:06am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782 "2023-09-08T11:06:15Z")
input { file { path =\> "/var/log/abc.log" } beats { port =\> 5044 } } filter { mutate { remove\_field =\> \[ "agent.version.keyword" \] } } --- ## [Logstash output Elastic upsert](https://discuss.elastic.co/t/logstash-output-elastic-upsert/341549)
**Author:** [@mathur7vidit](https://discuss.elastic.co/u/mathur7vidit)\ **Replies:** 6\ **Last updated:** [September 8, 2023, 11:08am UTC](https://discuss.elastic.co/t/logstash-output-elastic-upsert/341549 "2023-09-08T11:08:55Z")
Hi Team, i am looking for clarity on logstash's Elasticsearch output attribute docs\_as\_upsert and action =\> update. Now for this action to work properly and update the existing document, does the document should be on … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=427) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=429)