# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=519 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 520 --- ## [Auditbeat \>=8, logstash, and elasticsearch data stream](https://discuss.elastic.co/t/auditbeat-8-logstash-and-elasticsearch-data-stream/335357)
**Author:** [@Mike\_Williams](https://discuss.elastic.co/u/Mike_Williams)\ **Replies:** 2\ **Last updated:** [June 7, 2023, 9:37am UTC](https://discuss.elastic.co/t/auditbeat-8-logstash-and-elasticsearch-data-stream/335357 "2023-06-07T09:37:50Z")
Hey, I'm preparing to upgrade a set of auditbeat agents from 7.17 to 8.something. Clients are not allowed to talk directly to elasticsearch, all messages go through logstash. More than happy with the requirement to us… --- ## [Can't (yet) decode flowset id 256 from source id 0, because no template to decode it with has been received. This message will usually go away after 1 minute on logstash 7.17 and elasticsearch 7.17](https://discuss.elastic.co/t/cant-yet-decode-flowset-id-256-from-source-id-0-because-no-template-to-decode-it-with-has-been-received-this-message-will-usually-go-away-after-1-minute-on-logstash-7-17-and-elasticsearch-7-17/335421)
**Author:** [@Hanginium65](https://discuss.elastic.co/u/Hanginium65)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 9:32am UTC](https://discuss.elastic.co/t/cant-yet-decode-flowset-id-256-from-source-id-0-because-no-template-to-decode-it-with-has-been-received-this-message-will-usually-go-away-after-1-minute-on-logstash-7-17-and-elasticsearch-7-17/335421 "2023-06-07T09:32:33Z")
Hi, my config file for logstash looks like this: input { snmp { hosts =\> \[{host =\> "udp:192.168.56.3/161" version =\> "3"}\] get =\> \["1.3.6.1.2.1.25.3.3.1.2.1", "1.3.6.1.2.1.25.2.3.1.5.65536", "1.3.6.1.2.1.25.2… --- ## [Multiple filter for query not working as expected](https://discuss.elastic.co/t/multiple-filter-for-query-not-working-as-expected/335388)
**Author:** [@Atul\_Chadha](https://discuss.elastic.co/u/Atul_Chadha)\ **Replies:** 3\ **Last updated:** [June 7, 2023, 9:11am UTC](https://discuss.elastic.co/t/multiple-filter-for-query-not-working-as-expected/335388 "2023-06-07T09:11:41Z")
We are using elasticsearch as backend for our Wazuh cluster, i am trying to filter our results which contain values from "filter 01" and exclude results from "filter 02" however it looks its not working and showing resu… --- ## [JDBC INPUT plugin not syncing all eligible records from postgres db to elasticsearch](https://discuss.elastic.co/t/jdbc-input-plugin-not-syncing-all-eligible-records-from-postgres-db-to-elasticsearch/335409)
**Author:** [@Gio\_Vanni](https://discuss.elastic.co/u/Gio_Vanni)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 8:43am UTC](https://discuss.elastic.co/t/jdbc-input-plugin-not-syncing-all-eligible-records-from-postgres-db-to-elasticsearch/335409 "2023-06-07T08:43:00Z")
Hi I have an issue whereby logstash doesn't update all records that are returned by the jdbc-input query to Elasticsearch.As a result we always have to restart logstash to force through the updates. input plugin config: … --- ## [Can't get text on a START\_OBJECT at 1:34](https://discuss.elastic.co/t/cant-get-text-on-a-start-object-at-1-34/335399)
**Author:** [@hackercat](https://discuss.elastic.co/u/hackercat)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 7:40am UTC](https://discuss.elastic.co/t/cant-get-text-on-a-start-object-at-1-34/335399 "2023-06-07T07:40:52Z")
Hi everyone, I recently upgraded ELK from 7 to 8 and it was working fine for v7, but since v8, it continuously gave me the below error. Jun 07 16:48:00 gitlab-logger logstash\[115245\]: \[2023-06-07T16:48:00,346\]\[WARN \]\[l… --- ## [How to run multiple geo\_distance filter to get result for each filter separately?](https://discuss.elastic.co/t/how-to-run-multiple-geo-distance-filter-to-get-result-for-each-filter-separately/335286)
**Author:** [@maulik\_trapasiya](https://discuss.elastic.co/u/maulik_trapasiya)\ **Replies:** 3\ **Last updated:** [June 7, 2023, 7:20am UTC](https://discuss.elastic.co/t/how-to-run-multiple-geo-distance-filter-to-get-result-for-each-filter-separately/335286 "2023-06-07T07:20:13Z")
I need result documents based on geo\_distance query for points A(lat=3,long=101) and B(lat=5,long=102) separately. one result docs I need corresponding to filter A(lat=3,long=101) and other result set I need correspondin… --- ## [Certificate error when installing logstash plugin](https://discuss.elastic.co/t/certificate-error-when-installing-logstash-plugin/335391)
**Author:** [@fsaa](https://discuss.elastic.co/u/fsaa)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 5:23am UTC](https://discuss.elastic.co/t/certificate-error-when-installing-logstash-plugin/335391 "2023-06-07T05:23:11Z")
I'm using a VPN because I'm using a company laptop, when I run the command RUN bin/logstash-plugin install logstash-input-sftp.zip in the DockerFile I get this error: =\> \[6/7\] RUN zip -r logstash-input-sftp.zip logstash… --- ## [How to stop index from getting throttled?](https://discuss.elastic.co/t/how-to-stop-index-from-getting-throttled/334923)
**Author:** [@hjazz6](https://discuss.elastic.co/u/hjazz6)\ **Replies:** 18\ **Last updated:** [June 7, 2023, 3:26am UTC](https://discuss.elastic.co/t/how-to-stop-index-from-getting-throttled/334923 "2023-06-07T03:26:32Z")
Hi, I am ingesting netflow data into my ES 8.3.3 node at a very high rate. As I increase the ingest to ES where the index rate is about 30+K/s, I started to get the messages below: \[INFO\] \[o.e.i.e.I.EngineMergeSchedule… --- ## [Not able to to overwrite elasticsearch.keystore in elasticsearch 8.8.0](https://discuss.elastic.co/t/not-able-to-to-overwrite-elasticsearch-keystore-in-elasticsearch-8-8-0/335289)
**Author:** [@prathibha](https://discuss.elastic.co/u/prathibha)\ **Replies:** 0\ **Last updated:** [June 6, 2023, 7:12am UTC](https://discuss.elastic.co/t/not-able-to-to-overwrite-elasticsearch-keystore-in-elasticsearch-8-8-0/335289 "2023-06-06T07:12:23Z")
Hello Everyone, I have a single node elastic running on a centos vm . This elastic is running as a docker container and I am trying to upgrade from 7.16-\> 7.17-\> 8.8.0. I have multile configuration files mounted on to c… --- ## [Buffer Options does not contain a definition for ConcurrentConsumers](https://discuss.elastic.co/t/buffer-options-does-not-contain-a-definition-for-concurrentconsumers/335371)
**Author:** [@Marcos\_Ivan\_Robles\_H](https://discuss.elastic.co/u/Marcos_Ivan_Robles_H)\ **Replies:** 0\ **Last updated:** [June 6, 2023, 5:11pm UTC](https://discuss.elastic.co/t/buffer-options-does-not-contain-a-definition-for-concurrentconsumers/335371 "2023-06-06T17:11:23Z")
Hello I am trying to follow the example in the following nuget package: Elastic.Serilog.Sinks I got the following error: Buffer Options does not contain a definition for ConcurrentConsumers this is the code in progra… --- ## [Elastic agent](https://discuss.elastic.co/t/elastic-agent/335353)
**Author:** [@kafikone](https://discuss.elastic.co/u/kafikone)\ **Replies:** 1\ **Last updated:** [June 6, 2023, 11:27pm UTC](https://discuss.elastic.co/t/elastic-agent/335353 "2023-06-06T23:27:29Z")
J'ai deployé elasticsearch,kibana et logstash sur une machine virtuel CentOS 7, tout fonctionne correctement. puis j'ai créé un serveur fleet et installé un agent elastic sur une Vm windows 11. Le satatut de mon agent es… --- ## [Show only results that are relevent to my shopping history](https://discuss.elastic.co/t/show-only-results-that-are-relevent-to-my-shopping-history/333149)
**Author:** [@searchwithme](https://discuss.elastic.co/u/searchwithme)\ **Replies:** 1\ **Last updated:** [June 6, 2023, 11:05pm UTC](https://discuss.elastic.co/t/show-only-results-that-are-relevent-to-my-shopping-history/333149 "2023-06-06T23:05:38Z")
I want to understand if elasticsearch + knn could be used to accomplish this ask: a search on an item should only show relevent results with my shopping history. eg) search on dress should only show red or black or full… --- ## [Logstash not pulling data fast enough from Kafka](https://discuss.elastic.co/t/logstash-not-pulling-data-fast-enough-from-kafka/334377)
**Author:** [@Francisco\_Yanez](https://discuss.elastic.co/u/Francisco_Yanez)\ **Replies:** 3\ **Last updated:** [June 6, 2023, 10:43pm UTC](https://discuss.elastic.co/t/logstash-not-pulling-data-fast-enough-from-kafka/334377 "2023-06-06T22:43:48Z")
I have a huge problem. My kafka is on a different DC and we are using logstash to pull data. Our Elastic stack is running in kubernetes but our data is getting pulled very slow. How can I optimize logstash to pull data f… --- ## [Unable to see the "Available fields column" in the logs forwarding for Devbyok cluster](https://discuss.elastic.co/t/unable-to-see-the-available-fields-column-in-the-logs-forwarding-for-devbyok-cluster/334807)
**Author:** [@subagh](https://discuss.elastic.co/u/subagh)\ **Replies:** 3\ **Last updated:** [June 6, 2023, 10:15pm UTC](https://discuss.elastic.co/t/unable-to-see-the-available-fields-column-in-the-logs-forwarding-for-devbyok-cluster/334807 "2023-06-06T22:15:09Z")
Pic 1 - Does not shows any available fields in Dashboard Can anyone please tell me why I cannot see the relevant "string fields" tab to select options from DevByok cluster but with similar configuration, I am able to se… --- ## [How to "join" two different types of documents on the closest value of a common integer key](https://discuss.elastic.co/t/how-to-join-two-different-types-of-documents-on-the-closest-value-of-a-common-integer-key/333226)
**Author:** [@Mathemaphysics](https://discuss.elastic.co/u/Mathemaphysics)\ **Replies:** 3\ **Last updated:** [June 6, 2023, 9:27pm UTC](https://discuss.elastic.co/t/how-to-join-two-different-types-of-documents-on-the-closest-value-of-a-common-integer-key/333226 "2023-06-06T21:27:32Z")
I have a problem. I've inherited legacy code for which ELK stack is now being used to capture and detect problems. Logstash + Filebeat are being used and an index template is being used to correctly map WGS84 points. I … --- ## [Palo Alto Networks - Logstash \> Elastic \> Kibana](https://discuss.elastic.co/t/palo-alto-networks-logstash-elastic-kibana/335380)
**Author:** [@thunt](https://discuss.elastic.co/u/thunt)\ **Replies:** 7\ **Last updated:** [June 6, 2023, 9:16pm UTC](https://discuss.elastic.co/t/palo-alto-networks-logstash-elastic-kibana/335380 "2023-06-06T21:16:01Z")
Hello Everyone - Hoping I have a simple solution. Testing out Elastic Stack with Palo Alto syslogs, and running into issues with GeoIP's and combining the lon/lat to use Maps in Kibana. Not sure what else needs to be d… --- ## [Can heartbeat parse the html response of a URL?](https://discuss.elastic.co/t/can-heartbeat-parse-the-html-response-of-a-url/335374)
**Author:** [@Patr123](https://discuss.elastic.co/u/Patr123)\ **Replies:** 1\ **Last updated:** [June 6, 2023, 9:03pm UTC](https://discuss.elastic.co/t/can-heartbeat-parse-the-html-response-of-a-url/335374 "2023-06-06T21:03:12Z")
I am trying to parse the html response body of a URL in http monitor type and this is my config: - type: http enabled: true id: narvar name: Narvar urls: \["https://status.narvar.com/"\] schedule: '@every 10s' … --- ## [Unable to exclude metricbeat metrics with drop\_events](https://discuss.elastic.co/t/unable-to-exclude-metricbeat-metrics-with-drop-events/335378)
**Author:** [@dfinn](https://discuss.elastic.co/u/dfinn)\ **Replies:** 0\ **Last updated:** [June 6, 2023, 7:49pm UTC](https://discuss.elastic.co/t/unable-to-exclude-metricbeat-metrics-with-drop-events/335378 "2023-06-06T19:49:31Z")
I've tried quite a few different syntax and I'm not able to get metricbeat to exclude metrics. I'm not getting any errors on startup either. Here is my config: setup: template: enabled: true … --- ## [Grok multi-line mode](https://discuss.elastic.co/t/grok-multi-line-mode/335101)
**Author:** [@M\_D](https://discuss.elastic.co/u/M_D)\ **Replies:** 7\ **Last updated:** [June 6, 2023, 7:44pm UTC](https://discuss.elastic.co/t/grok-multi-line-mode/335101 "2023-06-06T19:44:27Z")
I am using (?ms) in my grok filter, but got an error (see RegexpError: undefined). What should be the right way to lookup multiple lines using grok? in regular regex i amd doing (?sm)(?\\[0-9\]{4}-\[0-9\]{2}-\[0-9… --- ## [Returning count of buckets from aggregation terms search](https://discuss.elastic.co/t/returning-count-of-buckets-from-aggregation-terms-search/335373)
**Author:** [@pocketcolin](https://discuss.elastic.co/u/pocketcolin)\ **Replies:** 0\ **Last updated:** [June 6, 2023, 6:48pm UTC](https://discuss.elastic.co/t/returning-count-of-buckets-from-aggregation-terms-search/335373 "2023-06-06T18:48:17Z")
Before anyone suggests it, I am trying to use the terms aggregation with a very large size value to get a more exact number as opposed to using cardinality. I realize it's less efficient but I'm only searching around 75k… --- ## [Log4j2 vulnerability mitigation - JndiLookup Removal](https://discuss.elastic.co/t/log4j2-vulnerability-mitigation-jndilookup-removal/335356)
**Author:** [@JosephAnis](https://discuss.elastic.co/u/JosephAnis)\ **Replies:** 1\ **Last updated:** [June 6, 2023, 6:43pm UTC](https://discuss.elastic.co/t/log4j2-vulnerability-mitigation-jndilookup-removal/335356 "2023-06-06T18:43:49Z")
Hi All, We are working on mitigating the Log4j2 vulnerability by removing the JndiLookup class as described here: We are using version 7.9.2 for all ELK components and currently we can't upgrade to newer version. My … --- ## [Cluster takes too long to apply cluster state](https://discuss.elastic.co/t/cluster-takes-too-long-to-apply-cluster-state/328407)
**Author:** [@Vadym](https://discuss.elastic.co/u/Vadym)\ **Replies:** 26\ **Last updated:** [June 6, 2023, 6:14pm UTC](https://discuss.elastic.co/t/cluster-takes-too-long-to-apply-cluster-state/328407 "2023-06-06T18:14:51Z")
Hi guys, We have some 1Tb+ indices and it takes more than a minute to drop these indices when we rotate them. During the deletion cluster takes too long to apply cluster state and master nodes start to kick data nodes o… --- ## [How do you set up the user account to run Elasticsearch service on Linux?](https://discuss.elastic.co/t/how-do-you-set-up-the-user-account-to-run-elasticsearch-service-on-linux/335368)
**Author:** [@Latitude](https://discuss.elastic.co/u/Latitude)\ **Replies:** 5\ **Last updated:** [June 6, 2023, 5:22pm UTC](https://discuss.elastic.co/t/how-do-you-set-up-the-user-account-to-run-elasticsearch-service-on-linux/335368 "2023-06-06T17:22:28Z")
Hello, I'm new to my organization and to Elasticsearch. I'm the new server administrator for Liferay 7.4 DXP which uses Elasticsearch 7.17.x. I'm developing our migration procedure as we're migrating to Liferay 7.4 DXP … --- ## [Getting strange errors after enabling kubernetes metadata in filebeat](https://discuss.elastic.co/t/getting-strange-errors-after-enabling-kubernetes-metadata-in-filebeat/335306)
**Author:** [@sunil\_s](https://discuss.elastic.co/u/sunil_s)\ **Replies:** 12\ **Last updated:** [June 6, 2023, 4:33pm UTC](https://discuss.elastic.co/t/getting-strange-errors-after-enabling-kubernetes-metadata-in-filebeat/335306 "2023-06-06T16:33:46Z")
Getting below errors when running filebeat evel":"error","@timestamp":"2023-06-06T08:37:11.110Z","log.logger":"kubernetes","log.origin":{"file.name":"add\_kubernetes\_metadata/matchers.go","file.line":95},"message":"Error… --- ## [Logstash unable to collect logs from filebeat due to protocol mismatch](https://discuss.elastic.co/t/logstash-unable-to-collect-logs-from-filebeat-due-to-protocol-mismatch/335269)
**Author:** [@Chel\_Db](https://discuss.elastic.co/u/Chel_Db)\ **Replies:** 16\ **Last updated:** [June 6, 2023, 4:31pm UTC](https://discuss.elastic.co/t/logstash-unable-to-collect-logs-from-filebeat-due-to-protocol-mismatch/335269 "2023-06-06T16:31:33Z")
I've installed filebeat in our k8s following official elastic document (kubernetes/filebeat-kubernetes.yaml ) to collect logs of our microservices and push it to the Logstash which is installed in a different VM as a co… --- ## [Kubernetes annotation - array value declaration](https://discuss.elastic.co/t/kubernetes-annotation-array-value-declaration/335304)
**Author:** [@Vijayakumar\_Kannan](https://discuss.elastic.co/u/Vijayakumar_Kannan)\ **Replies:** 3\ **Last updated:** [June 6, 2023, 1:39pm UTC](https://discuss.elastic.co/t/kubernetes-annotation-array-value-declaration/335304 "2023-06-06T13:39:11Z")
How do we convert the following filebeat config into kubernetes pod annotation level. processors: - decode\_json\_fields: fields: \["message","msg"\] target: "qrapp" add\_error\_key: true kuber… --- ## [Docker-compose instructions lead to "unable to authenticate user \[elastic\]"](https://discuss.elastic.co/t/docker-compose-instructions-lead-to-unable-to-authenticate-user-elastic/333060)
**Author:** [@Pinch](https://discuss.elastic.co/u/Pinch)\ **Replies:** 27\ **Last updated:** [June 6, 2023, 1:09pm UTC](https://discuss.elastic.co/t/docker-compose-instructions-lead-to-unable-to-authenticate-user-elastic/333060 "2023-06-06T13:09:10Z")
Following these official instructions: Brings me to a state of "Kibana server is not ready yet." in the browser. Inspecting the logs I can see from the Kibana container that kibana\_system is not authenticated. Then c… --- ## [Rest API client](https://discuss.elastic.co/t/rest-api-client/335323)
**Author:** [@mhr](https://discuss.elastic.co/u/mhr)\ **Replies:** 1\ **Last updated:** [June 6, 2023, 12:41pm UTC](https://discuss.elastic.co/t/rest-api-client/335323 "2023-06-06T12:41:42Z")
Do i need to upgrade REST API client version 7.2.1 also after ES up-gradation from version 7.8 to 7.17. --- ## [Best Practice: Update metadata on larger documents](https://discuss.elastic.co/t/best-practice-update-metadata-on-larger-documents/335311)
**Author:** [@Hiketas](https://discuss.elastic.co/u/Hiketas)\ **Replies:** 5\ **Last updated:** [June 6, 2023, 12:21pm UTC](https://discuss.elastic.co/t/best-practice-update-metadata-on-larger-documents/335311 "2023-06-06T12:21:51Z")
I have a question about best practice in the following scenario: I have documents with some meta fields among others with a full text field which can be up to 10 MB in size. We currently do not use parent/child relation… --- ## [Displaying Latest Image with filter from Log Message](https://discuss.elastic.co/t/displaying-latest-image-with-filter-from-log-message/335160)
**Author:** [@witwit](https://discuss.elastic.co/u/witwit)\ **Replies:** 4\ **Last updated:** [June 6, 2023, 11:19am UTC](https://discuss.elastic.co/t/displaying-latest-image-with-filter-from-log-message/335160 "2023-06-06T11:19:25Z")
Hi everyone. So currently, I'm getting logs from various services. I manage to tag these services as a field when it's ingested into elasticsearch via logstash. I'm currently stump with one part where i'm trying to disp… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=518) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=520)