# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=817 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 818 --- ## [I need help using elastic agent to collect and transform logs](https://discuss.elastic.co/t/i-need-help-using-elastic-agent-to-collect-and-transform-logs/312981)
**Author:** [@shshsh](https://discuss.elastic.co/u/shshsh)\ **Replies:** 0\ **Last updated:** [August 26, 2022, 3:04am UTC](https://discuss.elastic.co/t/i-need-help-using-elastic-agent-to-collect-and-transform-logs/312981 "2022-08-26T03:04:15Z")
Hi, Im new to elasticsearch and its agents, and I'm having trouble understanding how to use the agents properly, especially ever since its trying to replace the beats family. I'm trying to use elastic agents managed by… --- ## [How is log4j-1.2.17 jar being installed on my system](https://discuss.elastic.co/t/how-is-log4j-1-2-17-jar-being-installed-on-my-system/312971)
**Author:** [@brilong](https://discuss.elastic.co/u/brilong)\ **Replies:** 1\ **Last updated:** [August 26, 2022, 1:10am UTC](https://discuss.elastic.co/t/how-is-log4j-1-2-17-jar-being-installed-on-my-system/312971 "2022-08-26T01:10:16Z")
I am not very familiar with logstash but I am trying to determine why the file /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-log4j-3.1.3-java/vendor/jar-dependencies/runtime-jars/log4j-1.2.17.jar exis… --- ## [Filebeat how to set a custom indice](https://discuss.elastic.co/t/filebeat-how-to-set-a-custom-indice/312969)
**Author:** [@dimmthewitted1](https://discuss.elastic.co/u/dimmthewitted1)\ **Replies:** 2\ **Last updated:** [August 25, 2022, 11:13pm UTC](https://discuss.elastic.co/t/filebeat-how-to-set-a-custom-indice/312969 "2022-08-25T23:13:15Z")
I have never been more frustrated in a product than I am with the ELK stack. Its needlessly complicated, feels almost intentional as if to push people towards Fleet and Elastic Cloud. I have given up on the Apache inte… --- ## [How do I match a newline in logstash grok](https://discuss.elastic.co/t/how-do-i-match-a-newline-in-logstash-grok/311596)
**Author:** [@yts85205107](https://discuss.elastic.co/u/yts85205107)\ **Replies:** 5\ **Last updated:** [August 25, 2022, 10:50pm UTC](https://discuss.elastic.co/t/how-do-i-match-a-newline-in-logstash-grok/311596 "2022-08-25T22:50:48Z")
this is my log sample as below: \[2022-08-07T15:57:54+08:00\] 9.9.9.9 "Request-Method-URL: GET XXX Sex - Free Porn Videos on XXX.com" "Status-Code: 200" "Request-Length: 1" "Request-Time: 1.23" "Upstream-Server: 1.1.1.1:1… --- ## [Script score vector search performance](https://discuss.elastic.co/t/script-score-vector-search-performance/312341)
**Author:** [@zyyme](https://discuss.elastic.co/u/zyyme)\ **Replies:** 2\ **Last updated:** [August 25, 2022, 9:21pm UTC](https://discuss.elastic.co/t/script-score-vector-search-performance/312341 "2022-08-25T21:21:20Z")
Hello! I use Elasticsearch 7.14 and I have a mapping like this: { "mappings": { "properties": { "vector": { "type": "dense\_vector", "dims": 512 }, "category": { "type": "… --- ## [Elastic search match Query string query (blue OR red OR yellow) AND cake](https://discuss.elastic.co/t/elastic-search-match-query-string-query-blue-or-red-or-yellow-and-cake/312704)
**Author:** [@qub123](https://discuss.elastic.co/u/qub123)\ **Replies:** 4\ **Last updated:** [August 25, 2022, 8:13pm UTC](https://discuss.elastic.co/t/elastic-search-match-query-string-query-blue-or-red-or-yellow-and-cake/312704 "2022-08-25T20:13:27Z")
Hi, I am trying to create a query to request all the "cake" with blue or red or yellow color, and I guess the condition similar to (blue OR red OR yellow) AND cake. And I have look at the Elasticsearch Query String q… --- ## [We need a currency type in Elastic String validators and converters to currencies of other countries. We must store currency values in the native country currency and we need field validators to remove currency formatting characters](https://discuss.elastic.co/t/we-need-a-currency-type-in-elastic-string-validators-and-converters-to-currencies-of-other-countries-we-must-store-currency-values-in-the-native-country-currency-and-we-need-field-validators-to-remove-currency-formatting-characters/312965)
**Author:** [@jmkdev](https://discuss.elastic.co/u/jmkdev)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 8:04pm UTC](https://discuss.elastic.co/t/we-need-a-currency-type-in-elastic-string-validators-and-converters-to-currencies-of-other-countries-we-must-store-currency-values-in-the-native-country-currency-and-we-need-field-validators-to-remove-currency-formatting-characters/312965 "2022-08-25T20:04:09Z")
Continuing the discussion from How to store currency datatype in elasticsearch?: --- ## [LDAP configuration for elastic](https://discuss.elastic.co/t/ldap-configuration-for-elastic/312874)
**Author:** [@irivas95](https://discuss.elastic.co/u/irivas95)\ **Replies:** 4\ **Last updated:** [August 25, 2022, 6:30pm UTC](https://discuss.elastic.co/t/ldap-configuration-for-elastic/312874 "2022-08-25T18:30:54Z")
Hi, I'm a bit new to security configurations that don't involve native realm. The case is that I would like to configure elasticsearch authentication with ldap and the configuration is not very clear about some things. … --- ## [Search in Discover](https://discuss.elastic.co/t/search-in-discover/312958)
**Author:** [@papilocheg1992](https://discuss.elastic.co/u/papilocheg1992)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 6:13pm UTC](https://discuss.elastic.co/t/search-in-discover/312958 "2022-08-25T18:13:46Z")
we have some value (word) that we want to find in an index with a large number of fields. How can this be done using the Kibana Discovery interface? --- ## [The filestream input](https://discuss.elastic.co/t/the-filestream-input/312948)
**Author:** [@morad\_della3](https://discuss.elastic.co/u/morad_della3)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 6:07pm UTC](https://discuss.elastic.co/t/the-filestream-input/312948 "2022-08-25T18:07:35Z")
what's the role of the the filestream input ??? filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input spe… --- ## [Failed to install template](https://discuss.elastic.co/t/failed-to-install-template/312949)
**Author:** [@Queren\_Santos](https://discuss.elastic.co/u/Queren_Santos)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 5:58pm UTC](https://discuss.elastic.co/t/failed-to-install-template/312949 "2022-08-25T17:58:07Z")
Can anyone help? My template in Elasticsearch is ready, but Logstash does not send the index to which it is already created: ERROR\]\[logstash.outputs.elasticsearch\]\[main\] Failed to install template {:message=\>"Got respon… --- ## [Metricbeat http module cant parse query key with period in it](https://discuss.elastic.co/t/metricbeat-http-module-cant-parse-query-key-with-period-in-it/312540)
**Author:** [@Nimar\_Arora](https://discuss.elastic.co/u/Nimar_Arora)\ **Replies:** 13\ **Last updated:** [August 25, 2022, 5:30pm UTC](https://discuss.elastic.co/t/metricbeat-http-module-cant-parse-query-key-with-period-in-it/312540 "2022-08-25T17:30:48Z")
query key-values such as resource.kafka.id: XXXX are being encoded as resource=map%5Bkafka%3Amap%5Bid%3AXXXX%5D%5D instead of resource.kafka.id=XXXX This is my metricbeat http module configuration: - module: http met… --- ## [Logstash only logs the word 'log' on completion](https://discuss.elastic.co/t/logstash-only-logs-the-word-log-on-completion/312916)
**Author:** [@usman1](https://discuss.elastic.co/u/usman1)\ **Replies:** 3\ **Last updated:** [August 25, 2022, 5:03pm UTC](https://discuss.elastic.co/t/logstash-only-logs-the-word-log-on-completion/312916 "2022-08-25T17:03:14Z")
I am using the following input configurations for deleting the files once read and log the information. input { file { path =\> "C:/.log" mode =\> "read" file\_completed\_action =\> "log\_and\_delete" file\_co… --- ## [Invalid Index Name Exception](https://discuss.elastic.co/t/invalid-index-name-exception/312908)
**Author:** [@hnf](https://discuss.elastic.co/u/hnf)\ **Replies:** 2\ **Last updated:** [August 25, 2022, 4:42pm UTC](https://discuss.elastic.co/t/invalid-index-name-exception/312908 "2022-08-25T16:42:29Z")
Hello everyone, I have an issue when I try to make an index a field created using add\_field by referencing them when outputing. I create three .conf files that begin by a number , like 1file.conf, 2file.conf and 99outp… --- ## [GPG key fails to import for filebeat on Centos9](https://discuss.elastic.co/t/gpg-key-fails-to-import-for-filebeat-on-centos9/312946)
**Author:** [@sbradley](https://discuss.elastic.co/u/sbradley)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 4:28pm UTC](https://discuss.elastic.co/t/gpg-key-fails-to-import-for-filebeat-on-centos9/312946 "2022-08-25T16:28:50Z")
Following the instructions here Repositories for APT and YUM | Filebeat Reference \[8.4\] | Elastic I attempt to import the GPG key and receive the following error: error: https://packages.elastic.co/GPG-KEY-elasticsearc… --- ## [Dynamic filter according to the presence of a field](https://discuss.elastic.co/t/dynamic-filter-according-to-the-presence-of-a-field/312944)
**Author:** [@arnoz](https://discuss.elastic.co/u/arnoz)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 4:10pm UTC](https://discuss.elastic.co/t/dynamic-filter-according-to-the-presence-of-a-field/312944 "2022-08-25T16:10:55Z")
I have for example this type of documents. { "name": "sylvie dupont", "age": 25, "accounts": \[ { "name": "BankOfAmerica", "amount": 33 }, { "name": "GoldBank", "amount": 578 … --- ## [Logstash IndexOutOfBoundsException: writerIndex + minWritableBytes exceeds maxCapacity](https://discuss.elastic.co/t/logstash-indexoutofboundsexception-writerindex-minwritablebytes-exceeds-maxcapacity/312943)
**Author:** [@cotjoey](https://discuss.elastic.co/u/cotjoey)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 3:37pm UTC](https://discuss.elastic.co/t/logstash-indexoutofboundsexception-writerindex-minwritablebytes-exceeds-maxcapacity/312943 "2022-08-25T15:37:17Z")
Hello, Lately my logstash instance has been throwing the following exceptions intermittently. I cannot find a solution anywhere. All the posts in this forum with the string "writerIndex" with similar issues are left una… --- ## [Logstash http input plugin and azure load balancer](https://discuss.elastic.co/t/logstash-http-input-plugin-and-azure-load-balancer/312940)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 3:25pm UTC](https://discuss.elastic.co/t/logstash-http-input-plugin-and-azure-load-balancer/312940 "2022-08-25T15:25:25Z")
I am not able to get the http input plugin to connect to our azure load balancer. The input input { http { host =\> "igemsng-eastus2-nprd-elkcluster.test.att.com" port =\> 8808 password =\> XXXX… --- ## [Why does Kibana stops after 10seconds?](https://discuss.elastic.co/t/why-does-kibana-stops-after-10seconds/312860)
**Author:** [@Vinyl\_Down](https://discuss.elastic.co/u/Vinyl_Down)\ **Replies:** 12\ **Last updated:** [August 25, 2022, 3:08pm UTC](https://discuss.elastic.co/t/why-does-kibana-stops-after-10seconds/312860 "2022-08-25T15:08:59Z")
Dear all, I really didn't want to post a question here, and i am trying for a week to fix my issue, but i can't seem to figure this out. I am trying to do some tests on Elastic Search Endpoint Security (using elastic a… --- ## [Failed to install template {:message=\>"Got response code '400' contacting Elasticsearch at URL](https://discuss.elastic.co/t/failed-to-install-template-message-got-response-code-400-contacting-elasticsearch-at-url/312939)
**Author:** [@Queren\_Santos](https://discuss.elastic.co/u/Queren_Santos)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 3:05pm UTC](https://discuss.elastic.co/t/failed-to-install-template-message-got-response-code-400-contacting-elasticsearch-at-url/312939 "2022-08-25T15:05:48Z")
Can anyone help? My template in Elasticsearch is ready, but Logstash does not send the index to which it is already created: ERROR\]\[logstash.outputs.elasticsearch\]\[main\] Failed to install template {:message=\>"Got respon… --- ## [Mapper\_parsing\_exception: failed to parse ( Dataframe to Elastic - batch process)](https://discuss.elastic.co/t/mapper-parsing-exception-failed-to-parse-dataframe-to-elastic-batch-process/312930)
**Author:** [@visit2siva](https://discuss.elastic.co/u/visit2siva)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 2:10pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-failed-to-parse-dataframe-to-elastic-batch-process/312930 "2022-08-25T14:10:38Z")
We are adding documents through a pyspark dataframe to elastic. df.write.mode('append').format('org.elasticsearch.spark.sql') .option('es.nodes', 'https://xyz.com') .option('es.net.http.auth.user','userid') .optio… --- ## [Can't see the Snapshot features on Kibana UI](https://discuss.elastic.co/t/cant-see-the-snapshot-features-on-kibana-ui/312912)
**Author:** [@CountryBall\_Tube](https://discuss.elastic.co/u/CountryBall_Tube)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 2:55pm UTC](https://discuss.elastic.co/t/cant-see-the-snapshot-features-on-kibana-ui/312912 "2022-08-25T14:55:31Z")
Hey i installed ES and Kibana on Docker but under the stack managament i can't see Stack Management \> Snapshot and Restore option, why? and how to fix it? --- ## [MongoDB to Elasticsearch synchronisation using Logstash JDBC Input plugin](https://discuss.elastic.co/t/mongodb-to-elasticsearch-synchronisation-using-logstash-jdbc-input-plugin/312934)
**Author:** [@hrasheed90](https://discuss.elastic.co/u/hrasheed90)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 2:27pm UTC](https://discuss.elastic.co/t/mongodb-to-elasticsearch-synchronisation-using-logstash-jdbc-input-plugin/312934 "2022-08-25T14:27:53Z")
I have defined several pipelines (each pipeline represents one to one relation with mongodb collection to elasticsearch index i.e. each mongodb collection will have its own index in elasticsearch). all my pipelines look… --- ## [Unable to upgrade from elastic-agent 8.3.3 to 8.4.0, or install 8.4.0 cleanly](https://discuss.elastic.co/t/unable-to-upgrade-from-elastic-agent-8-3-3-to-8-4-0-or-install-8-4-0-cleanly/312928)
**Author:** [@Curtis\_Ruck](https://discuss.elastic.co/u/Curtis_Ruck)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 1:57pm UTC](https://discuss.elastic.co/t/unable-to-upgrade-from-elastic-agent-8-3-3-to-8-4-0-or-install-8-4-0-cleanly/312928 "2022-08-25T13:57:22Z")
On Ubuntu 22.04.1, system had Elastic Agent 8.3.3 installed, doing daily apt upgrade -y and the 8.4.0 upgrade fails. Removing elastic-agent and reinstalling also fails. It fails during package configuration running ela… --- ## [Filebeat keeps file handle on deleted logfile](https://discuss.elastic.co/t/filebeat-keeps-file-handle-on-deleted-logfile/312608)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 6\ **Last updated:** [August 25, 2022, 1:29pm UTC](https://discuss.elastic.co/t/filebeat-keeps-file-handle-on-deleted-logfile/312608 "2022-08-25T13:29:43Z")
Hi, I have following issue and hope you can provide a solution. We are using elastic stack for monitoring our application. Priority 1: application must run Priority 2: monitoring should run to support and monitor the… --- ## [How to calculate % growth for a sliding time window with a yearly offset?](https://discuss.elastic.co/t/how-to-calculate-growth-for-a-sliding-time-window-with-a-yearly-offset/312880)
**Author:** [@tastyminerals](https://discuss.elastic.co/u/tastyminerals)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 7:58am UTC](https://discuss.elastic.co/t/how-to-calculate-growth-for-a-sliding-time-window-with-a-yearly-offset/312880 "2022-08-25T07:58:58Z")
I would like to create a timelion visualizaiton which shows a % growth of received documents within a range of 1 week against the same week but a year ago. I am not sure if the below timelion expression does exactly that… --- ## [Deleting processed objects from S3 when using Beats input (and Filebeat with aws-s3 input)](https://discuss.elastic.co/t/deleting-processed-objects-from-s3-when-using-beats-input-and-filebeat-with-aws-s3-input/312922)
**Author:** [@G.E](https://discuss.elastic.co/u/G.E)\ **Replies:** 0\ **Last updated:** [August 25, 2022, 1:15pm UTC](https://discuss.elastic.co/t/deleting-processed-objects-from-s3-when-using-beats-input-and-filebeat-with-aws-s3-input/312922 "2022-08-25T13:15:37Z")
Hi, Can anyone suggest a (hopefully uncomplicated) way to delete processed log files from S3 when using Filebeat and Logstash with SQS and S3? I had previously used the Logstash S3 plugin alone which does support this … --- ## [Error 1053 when starting winlogbeat or auditbeat](https://discuss.elastic.co/t/error-1053-when-starting-winlogbeat-or-auditbeat/312917)
**Author:** [@mammodde](https://discuss.elastic.co/u/mammodde)\ **Replies:** 1\ **Last updated:** [August 25, 2022, 1:12pm UTC](https://discuss.elastic.co/t/error-1053-when-starting-winlogbeat-or-auditbeat/312917 "2022-08-25T13:12:17Z")
Goodmorning, when i try to Start-Service winlogbeat i receive the error 1053m same thing for auditbeat. for the yml files i followed the getting started guide, so there should not be any error. i have a keystore and it… --- ## [Unable to run ElasticSearch after setting up SSL certificate](https://discuss.elastic.co/t/unable-to-run-elasticsearch-after-setting-up-ssl-certificate/312774)
**Author:** [@RonH8](https://discuss.elastic.co/u/RonH8)\ **Replies:** 2\ **Last updated:** [August 25, 2022, 9:34am UTC](https://discuss.elastic.co/t/unable-to-run-elasticsearch-after-setting-up-ssl-certificate/312774 "2022-08-25T09:34:09Z")
Hey I've followed the following guide to setup the SSL certificate for my Elasticsearch which is running on Ubuntu 22.04.1 LTS: https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html … --- ## [Unable to install Elastic Agent 7.17.5 (Windows)](https://discuss.elastic.co/t/unable-to-install-elastic-agent-7-17-5-windows/311284)
**Author:** [@sirineb](https://discuss.elastic.co/u/sirineb)\ **Replies:** 2\ **Last updated:** [August 25, 2022, 12:19pm UTC](https://discuss.elastic.co/t/unable-to-install-elastic-agent-7-17-5-windows/311284 "2022-08-25T12:19:35Z")
Hi everyone, I have an issue trying to enroll elastic agent 7.17.5 in Windows. (My fleet server is 7.17.5 too) The first time I installed it, it worked perfectly but due to a storage issue on the machine that hosts ELK,… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=816) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=818)