# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=211 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 212 --- ## [Enrich policy working only on some documents](https://discuss.elastic.co/t/enrich-policy-working-only-on-some-documents/362326)
**Author:** [@gomay](https://discuss.elastic.co/u/gomay)\ **Replies:** 2\ **Last updated:** [July 3, 2024, 2:31am UTC](https://discuss.elastic.co/t/enrich-policy-working-only-on-some-documents/362326 "2024-07-03T02:31:30Z")
hello guys!! I did the simulate pipeline with two documents, one that gets enriched and one that is not. This is the pipeline { "peruchannelspm-ingest-pipeline": { "processors": \[ { "enrich": { … --- ## [Better cluster configuration for 63 terrabyes of data](https://discuss.elastic.co/t/better-cluster-configuration-for-63-terrabyes-of-data/362347)
**Author:** [@paladin\_paterson](https://discuss.elastic.co/u/paladin_paterson)\ **Replies:** 5\ **Last updated:** [July 2, 2024, 9:00pm UTC](https://discuss.elastic.co/t/better-cluster-configuration-for-63-terrabyes-of-data/362347 "2024-07-02T21:00:59Z")
Hey gang, out team has an Elasticsearch cluster filled with HTML documents. Here is some data abour our current configuration: It consists of 10 data nodes, each with a heap.size of 32GB It contains a total of 600 shar… --- ## [Elastic-Agent errors "failed to publish events" errors when sending Auditd logs to Logstash](https://discuss.elastic.co/t/elastic-agent-errors-failed-to-publish-events-errors-when-sending-auditd-logs-to-logstash/362402)
**Author:** [@julia.navarro](https://discuss.elastic.co/u/julia.navarro)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 7:18pm UTC](https://discuss.elastic.co/t/elastic-agent-errors-failed-to-publish-events-errors-when-sending-auditd-logs-to-logstash/362402 "2024-07-02T19:18:38Z")
We keep getting the following errors and warnings when trying to send auditd logs to logstash. We assume it is an ssl issue but how can we get logstash to ignore ssl verification when receiving data from a standalone ela… --- ## [Logstash HTTP API port throwing errors when metricbeat is run](https://discuss.elastic.co/t/logstash-http-api-port-throwing-errors-when-metricbeat-is-run/362378)
**Author:** [@akasegaonkar](https://discuss.elastic.co/u/akasegaonkar)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 5:31pm UTC](https://discuss.elastic.co/t/logstash-http-api-port-throwing-errors-when-metricbeat-is-run/362378 "2024-07-02T17:31:01Z")
Hi! I hope this is the right forum to ask this question. I have a metricbeat -\> logstash -\> elasticsearch pipeline. My logstash setup consists of multiple pipelines. When I run metricbeat (when logstash/elasticsearch a… --- ## [How to calc login,logout duration inside special time span](https://discuss.elastic.co/t/how-to-calc-login-logout-duration-inside-special-time-span/362201)
**Author:** [@moryaden](https://discuss.elastic.co/u/moryaden)\ **Replies:** 4\ **Last updated:** [July 2, 2024, 4:30pm UTC](https://discuss.elastic.co/t/how-to-calc-login-logout-duration-inside-special-time-span/362201 "2024-07-02T16:30:54Z")
Guys, I already struggled from es docs for days, but not a better solution could be found. Need your professional opinions!! I have these recorded data lines: {"uid":1, "login": "2024-06-28 17:00:00", "logout":"2024-… --- ## [Encrypt Password In Filebeat Yml](https://discuss.elastic.co/t/encrypt-password-in-filebeat-yml/361743)
**Author:** [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)\ **Replies:** 5\ **Last updated:** [July 2, 2024, 3:37pm UTC](https://discuss.elastic.co/t/encrypt-password-in-filebeat-yml/361743 "2024-07-02T15:37:13Z")
Hi Team, Could you please help me how we can encrypt the plain text password which we are mentioning in filebeat.yml file. Thanks, Debasis --- ## [Change in dense\_vector field indexation default in ES 8.14 - Existing indices behaviour?](https://discuss.elastic.co/t/change-in-dense-vector-field-indexation-default-in-es-8-14-existing-indices-behaviour/362385)
**Author:** [@clemsau](https://discuss.elastic.co/u/clemsau)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 4:13pm UTC](https://discuss.elastic.co/t/change-in-dense-vector-field-indexation-default-in-es-8-14-existing-indices-behaviour/362385 "2024-07-02T16:13:04Z")
Hello, In the ES 8.14 patch notes, we can see that now, the dense\_vector field type used for vector search will now be indexed by default with the quantized hnsw\_int8 graphs. Prior to that, hnsw\_int8 was the graph used… --- ## [Elastic Agent not working on RAspberry Pi5](https://discuss.elastic.co/t/elastic-agent-not-working-on-raspberry-pi5/362390)
**Author:** [@Toony](https://discuss.elastic.co/u/Toony)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 2:53pm UTC](https://discuss.elastic.co/t/elastic-agent-not-working-on-raspberry-pi5/362390 "2024-07-02T14:53:31Z")
After the "elastic-agent-8.12.2-linux-arm64.tar.gz" agent installation, the agent binary execution is systematically failing with this message: ./elastic-agent: cannot execute: required file not found Is this a bug or … --- ## [Sending Email for each hit of Alert](https://discuss.elastic.co/t/sending-email-for-each-hit-of-alert/360868)
**Author:** [@MaximH](https://discuss.elastic.co/u/MaximH)\ **Replies:** 2\ **Last updated:** [July 2, 2024, 2:40pm UTC](https://discuss.elastic.co/t/sending-email-for-each-hit-of-alert/360868 "2024-07-02T14:40:13Z")
Hello there! I have the use case that we store k8s logs in Elasticsearch, and now want to send alerts containing the logmessage from said logs with kibana alerts. I have now created an alert firing once a loglevel equa… --- ## [How can i parse my logs using custom log integration](https://discuss.elastic.co/t/how-can-i-parse-my-logs-using-custom-log-integration/362287)
**Author:** [@sandraimmaculate](https://discuss.elastic.co/u/sandraimmaculate)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 1:20pm UTC](https://discuss.elastic.co/t/how-can-i-parse-my-logs-using-custom-log-integration/362287 "2024-07-02T13:20:14Z")
i want to parse my custom logs symfony which is an application logs. As it doesn't have integration so trying to use custom integration yes the log is getting stored in elasticsearch but not parsing it i mean the whole l… --- ## [Logstash HTTP output NDJSON with json\_batch](https://discuss.elastic.co/t/logstash-http-output-ndjson-with-json-batch/359779)
**Author:** [@gunlomboy](https://discuss.elastic.co/u/gunlomboy)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 1:14pm UTC](https://discuss.elastic.co/t/logstash-http-output-ndjson-with-json-batch/359779 "2024-07-02T13:14:44Z")
Hi, I am trying to output batched newline delimited JSON using the logstash http output. Can the json\_batch format be used together with the json\_lines codec to achieve this? A similar query was posted here, but nobod… --- ## [java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 80 During pipeline creation](https://discuss.elastic.co/t/java-lang-arrayindexoutofboundsexception-index-1-out-of-bounds-for-length-80-during-pipeline-creation/362379)
**Author:** [@sasikiranvaddi](https://discuss.elastic.co/u/sasikiranvaddi)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 12:55pm UTC](https://discuss.elastic.co/t/java-lang-arrayindexoutofboundsexception-index-1-out-of-bounds-for-length-80-during-pipeline-creation/362379 "2024-07-02T12:55:49Z")
We observe at times ArrayIndexOutofBoundsException during creating of pipeline. Here we use log4j2 pattern as PatternLayout for processing the logs. From the logs it is unclear, is it something with log event not match… --- ## [Logstash Ruby filter not saving when fields change](https://discuss.elastic.co/t/logstash-ruby-filter-not-saving-when-fields-change/362167)
**Author:** [@emas](https://discuss.elastic.co/u/emas)\ **Replies:** 3\ **Last updated:** [July 2, 2024, 11:55am UTC](https://discuss.elastic.co/t/logstash-ruby-filter-not-saving-when-fields-change/362167 "2024-07-02T11:55:58Z")
I am currently trying to parse some XML with the XML plugin, and then getting the attributes with a Ruby script. When it works, the XML I am parsing looks like this \ \ **Author:** [@taichi](https://discuss.elastic.co/u/taichi)\ **Replies:** 8\ **Last updated:** [July 2, 2024, 11:19am UTC](https://discuss.elastic.co/t/optimal-shard-strategy-for-high-search-load-with-elasticsearch-cluster/362242 "2024-07-02T11:19:25Z") Hi, I am working on optimizing an Elasticsearch cluster and am seeking advice on the best shard strategy. Here are the specifics of my setup: Node Details: 4 data nodes running on AWS r6g.large instances, with 3 maste… --- ## [Running multiple instances of Elasticsearch on the same host - Allocated processors setting](https://discuss.elastic.co/t/running-multiple-instances-of-elasticsearch-on-the-same-host-allocated-processors-setting/362372)
**Author:** [@avnere](https://discuss.elastic.co/u/avnere)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 10:50am UTC](https://discuss.elastic.co/t/running-multiple-instances-of-elasticsearch-on-the-same-host-allocated-processors-setting/362372 "2024-07-02T10:50:44Z")
Hi, I have machine with 86 cores and I want to run multiple instances of Elasticsearch on it. I want to isolate the number of cores foreach node. I saw in \[multiple instances of Elasticsearch on the same host\]( Thread… --- ## [Logstash SNMP plugin](https://discuss.elastic.co/t/logstash-snmp-plugin/360825)
**Author:** [@Viktor\_Movita](https://discuss.elastic.co/u/Viktor_Movita)\ **Replies:** 7\ **Last updated:** [July 2, 2024, 9:59am UTC](https://discuss.elastic.co/t/logstash-snmp-plugin/360825 "2024-07-02T09:59:15Z")
Hello everyone, Can I install a plugin on my Logstash server to receive SNMP messages when my Logstash server is completely disconnected from the internet? --- ## [Showing path / directory tree (or taxonomy) in Kibana](https://discuss.elastic.co/t/showing-path-directory-tree-or-taxonomy-in-kibana/362357)
**Author:** [@Cekir](https://discuss.elastic.co/u/Cekir)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 9:14am UTC](https://discuss.elastic.co/t/showing-path-directory-tree-or-taxonomy-in-kibana/362357 "2024-07-02T09:14:46Z")
Hi, I have an index containing information about files. One of the fields is the path of the file. Now, I need to show a directory tree and show how many files are there in the directory and all sub-directories, and pot… --- ## [Looking for document or technical article which explains the Memory Utilization of Java.exe when Elasticsearch is running on Windows server](https://discuss.elastic.co/t/looking-for-document-or-technical-article-which-explains-the-memory-utilization-of-java-exe-when-elasticsearch-is-running-on-windows-server/362354)
**Author:** [@Kalidastate](https://discuss.elastic.co/u/Kalidastate)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 9:09am UTC](https://discuss.elastic.co/t/looking-for-document-or-technical-article-which-explains-the-memory-utilization-of-java-exe-when-elasticsearch-is-running-on-windows-server/362354 "2024-07-02T09:09:49Z")
During extensive testing of Elasticsearch's search functionality, we noticed that the memory utilization of java.exe spikes, causing overall system memory utilization to reach 100%. Despite this, no search failures were … --- ## [Logstash grok divides the log message into segments](https://discuss.elastic.co/t/logstash-grok-divides-the-log-message-into-segments/362170)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 1\ **Last updated:** [June 27, 2024, 5:32pm UTC](https://discuss.elastic.co/t/logstash-grok-divides-the-log-message-into-segments/362170 "2024-06-27T17:32:24Z")
The message is chopped from the log file and grok fails for few of the events. The below is chopped one al:9092/alp.incident.raw/group0/0\\\\\\",\\\\\\"cen://kafka-china-qa-1a-0.kafka-china-qa-1a-headless.china-qa-posdatabus… --- ## [Logstash skips string in the beginning of the event and ends up with json parse error](https://discuss.elastic.co/t/logstash-skips-string-in-the-beginning-of-the-event-and-ends-up-with-json-parse-error/362303)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 0\ **Last updated:** [July 1, 2024, 3:30pm UTC](https://discuss.elastic.co/t/logstash-skips-string-in-the-beginning-of-the-event-and-ends-up-with-json-parse-error/362303 "2024-07-01T15:30:02Z")
\[2024-06-27T07:38:58,131\]\[ERROR\]\[logstash.codecs.json \]\[main\]\[f606102f50879b5c431b527abd5f18d638386c850d0f511237fefcb5bd81e725\] JSON parse error, original data now in message field {:message=\>"incompatible json objec… --- ## [Issues utilizing Top Hit aggregation](https://discuss.elastic.co/t/issues-utilizing-top-hit-aggregation/362349)
**Author:** [@Konrad\_Tarczynski](https://discuss.elastic.co/u/Konrad_Tarczynski)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 7:28am UTC](https://discuss.elastic.co/t/issues-utilizing-top-hit-aggregation/362349 "2024-07-02T07:28:01Z")
Hello! I am looking to visualise a table containing info regarding logon locations of devices by recognizing IP patterns. The important bit I'm missing now is getting the table to only show the most recent entry for eac… --- ## [Renaming the segments of pie chart and associated data value](https://discuss.elastic.co/t/renaming-the-segments-of-pie-chart-and-associated-data-value/362311)
**Author:** [@timadsit](https://discuss.elastic.co/u/timadsit)\ **Replies:** 1\ **Last updated:** [July 2, 2024, 8:21am UTC](https://discuss.elastic.co/t/renaming-the-segments-of-pie-chart-and-associated-data-value/362311 "2024-07-02T08:21:36Z")
Hello, I am mapping logs to a severity of numbers 1-4. For the life of me I cannot find anywhere that talks about renaming the names of the slices to something readable. Any help in renaming the numbers to text? Th… --- ## [Highlighting issue with fuzzy query with edge\_ngram tokens](https://discuss.elastic.co/t/highlighting-issue-with-fuzzy-query-with-edge-ngram-tokens/362345)
**Author:** [@lschnei](https://discuss.elastic.co/u/lschnei)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 7:08am UTC](https://discuss.elastic.co/t/highlighting-issue-with-fuzzy-query-with-edge-ngram-tokens/362345 "2024-07-02T07:08:35Z")
I couldn’t find a relevant example, so here’s my issue: I’m developing a search-as-you-type feature using edge\_ngram tokens, which allows for spelling mistakes (fuzzy queries) while highlighting the current edge\_ngram t… --- ## [Slow Data Collection and Limited Event Count in Filebeat](https://discuss.elastic.co/t/slow-data-collection-and-limited-event-count-in-filebeat/362339)
**Author:** [@lkh14341](https://discuss.elastic.co/u/lkh14341)\ **Replies:** 0\ **Last updated:** [July 2, 2024, 6:23am UTC](https://discuss.elastic.co/t/slow-data-collection-and-limited-event-count-in-filebeat/362339 "2024-07-02T06:23:35Z")
I am experiencing an issue with Filebeat where the number of events collected at a time is limited to either 50 or 100, and the data collection speed to Elasticsearch is noticeably slow. This problem doesn't occur all t… --- ## [Looking up values from a diiferent index to the one being searched](https://discuss.elastic.co/t/looking-up-values-from-a-diiferent-index-to-the-one-being-searched/362111)
**Author:** [@xef](https://discuss.elastic.co/u/xef)\ **Replies:** 0\ **Last updated:** [June 26, 2024, 5:33pm UTC](https://discuss.elastic.co/t/looking-up-values-from-a-diiferent-index-to-the-one-being-searched/362111 "2024-06-26T17:33:53Z")
Let's say we have an index for Jobs to be carried out called "jobs". In this index there is a field called localityId which is a string that represnets where the job location is. This string is defined in another index c… --- ## [Dynamic index creation](https://discuss.elastic.co/t/dynamic-index-creation/362280)
**Author:** [@siakc](https://discuss.elastic.co/u/siakc)\ **Replies:** 4\ **Last updated:** [July 1, 2024, 2:07pm UTC](https://discuss.elastic.co/t/dynamic-index-creation/362280 "2024-07-01T14:07:11Z")
From what I understand ES tries to create index by some rules from incoming logs (which I think we call documents). Now I have too many fields in my docs which ES creates index for. What I need is to stop it from includ… --- ## [Calculating duration using timestamps from multiple documents](https://discuss.elastic.co/t/calculating-duration-using-timestamps-from-multiple-documents/362296)
**Author:** [@Bjorn\_Ottenheim](https://discuss.elastic.co/u/Bjorn_Ottenheim)\ **Replies:** 0\ **Last updated:** [July 1, 2024, 1:05pm UTC](https://discuss.elastic.co/t/calculating-duration-using-timestamps-from-multiple-documents/362296 "2024-07-01T13:05:43Z")
Hi.. I'm using Elastic Cloud v 8.13.2 I have a service which logs timestamps at the start and the end of the process and does that in separate documents: For example: { "\_index": "dev-2024.06.02-000002", "\_id": "MP… --- ## [Documents in being deleted after indexed](https://discuss.elastic.co/t/documents-in-being-deleted-after-indexed/362307)
**Author:** [@gomay](https://discuss.elastic.co/u/gomay)\ **Replies:** 1\ **Last updated:** [July 1, 2024, 5:09pm UTC](https://discuss.elastic.co/t/documents-in-being-deleted-after-indexed/362307 "2024-07-01T17:09:59Z")
Some of the documents of an index are being deleted, the input is logstash jdbc plugin. No id's comes from the source. the settings of my index are as following { "settings": { "index": { "routing": { … --- ## [Kubernetes integration wrong node name](https://discuss.elastic.co/t/kubernetes-integration-wrong-node-name/362306)
**Author:** [@pk92](https://discuss.elastic.co/u/pk92)\ **Replies:** 0\ **Last updated:** [July 1, 2024, 4:07pm UTC](https://discuss.elastic.co/t/kubernetes-integration-wrong-node-name/362306 "2024-07-01T16:07:22Z")
Hi, Let's say I have a Kubernetes cluster with three nodes: host1 host2 host3 now when I get container logs from these nodes, all of them have host1 in their node.name and node.hostname field, though the host.hostname… --- ## [Limiting role to a space removes search access?](https://discuss.elastic.co/t/limiting-role-to-a-space-removes-search-access/362172)
**Author:** [@fwalling](https://discuss.elastic.co/u/fwalling)\ **Replies:** 3\ **Last updated:** [July 1, 2024, 3:44pm UTC](https://discuss.elastic.co/t/limiting-role-to-a-space-removes-search-access/362172 "2024-07-01T15:44:55Z")
I've installed elastic and kibana 8.14.1 on Linux via the tar.gz method. I started both without any parameters, and used the enrollment token to connect them. I logged in to kibana using the generated elastic user and pa… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=210) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=212)