# Latest **URL:** https://discuss.elastic.co/latest.md?page=649 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 650 --- ## [How to construct the customized fields from the fluentd output](https://discuss.elastic.co/t/how-to-construct-the-customized-fields-from-the-fluentd-output/335474)
**Author:** [@Chel\_Db](https://discuss.elastic.co/u/Chel_Db)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 7:22pm UTC](https://discuss.elastic.co/t/how-to-construct-the-customized-fields-from-the-fluentd-output/335474 "2023-06-07T19:22:34Z")
I'm capturing the logs from fluentd output onto Logstash using a basic config. input { http { port =\> 8080 } } output { elasticsearch { hosts =\> \["\<%= @ipaddress%\>:9200"\] index =\> "fluentd-%{+YYYY… --- ## [I cannot search by telephone (part)](https://discuss.elastic.co/t/i-cannot-search-by-telephone-part/333353)
**Author:** [@mg85](https://discuss.elastic.co/u/mg85)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 7:21pm UTC](https://discuss.elastic.co/t/i-cannot-search-by-telephone-part/333353 "2023-06-07T19:21:37Z")
Im trying to create an autocomplete, this is my index creation: curl -X PUT "localhost:9200/backoffice\_clients-com" -H 'Content-Type: application/json' -d' { "settings": { "analysis": { "analyzer": { … --- ## [QueryString vs multiple wildcards](https://discuss.elastic.co/t/querystring-vs-multiple-wildcards/335382)
**Author:** [@Ortiga\_Abdo](https://discuss.elastic.co/u/Ortiga_Abdo)\ **Replies:** 5\ **Last updated:** [June 7, 2023, 6:23pm UTC](https://discuss.elastic.co/t/querystring-vs-multiple-wildcards/335382 "2023-06-07T18:23:01Z")
I can't find any documentations that talks about queries and their performance/comparison I'm wondering which is better performance/faster multiple wildcard filter or a string\_query? "query": { "bool" : { "mu… --- ## [Drop old values from group by in metric threshold rule](https://discuss.elastic.co/t/drop-old-values-from-group-by-in-metric-threshold-rule/335456)
**Author:** [@mgordon](https://discuss.elastic.co/u/mgordon)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 5:58pm UTC](https://discuss.elastic.co/t/drop-old-values-from-group-by-in-metric-threshold-rule/335456 "2023-06-07T17:58:08Z")
I have a metric threshold rule that's grouped by two values (server and app pool name). When I permanently remove an app pool from a server, how do I 'refresh' the values so that one stops alerting that it's missing? --- ## [How are logstash grok definitions updated?](https://discuss.elastic.co/t/how-are-logstash-grok-definitions-updated/335452)
**Author:** [@lreger](https://discuss.elastic.co/u/lreger)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 5:35pm UTC](https://discuss.elastic.co/t/how-are-logstash-grok-definitions-updated/335452 "2023-06-07T17:35:41Z")
How do I find out what my current version of logstash core patterns are running on my logstash cluster? I am running 7.17.1, but I suspect I am not running grok core patterns 4.34 ecsv1. I would like to have access to s… --- ## [Fetching all external IP address from firewall logs using logstash](https://discuss.elastic.co/t/fetching-all-external-ip-address-from-firewall-logs-using-logstash/334934)
**Author:** [@libinmath](https://discuss.elastic.co/u/libinmath)\ **Replies:** 5\ **Last updated:** [June 7, 2023, 4:27pm UTC](https://discuss.elastic.co/t/fetching-all-external-ip-address-from-firewall-logs-using-logstash/334934 "2023-06-07T16:27:27Z")
I am working with fortinet firewall logs, trying to get all external IP address from the fields srcip and dstip into a text file. I am new to writing filters for the logstash. The sample documents are as follow but I am… --- ## [Failed to start elastic search service after upgrade from version 8.2 to 8.8](https://discuss.elastic.co/t/failed-to-start-elastic-search-service-after-upgrade-from-version-8-2-to-8-8/335081)
**Author:** [@JonathanDSSOUZA](https://discuss.elastic.co/u/JonathanDSSOUZA)\ **Replies:** 5\ **Last updated:** [June 7, 2023, 4:17pm UTC](https://discuss.elastic.co/t/failed-to-start-elastic-search-service-after-upgrade-from-version-8-2-to-8-8/335081 "2023-06-07T16:17:49Z")
Hello community, after updating a cluster that contains 3 master nodes and 3 data nodes (ingest), the master nodes work normally, but the ingest nodes do not start the elasticsearch service, activating the DEBUG mode, re… --- ## [Https://discuss.elastic.co/t/possible-to-highlight-inner-hits-in-percolate-query/91926](https://discuss.elastic.co/t/https-discuss-elastic-co-t-possible-to-highlight-inner-hits-in-percolate-query-91926/335261)
**Author:** [@marufrahman](https://discuss.elastic.co/u/marufrahman)\ **Replies:** 4\ **Last updated:** [June 7, 2023, 3:35pm UTC](https://discuss.elastic.co/t/https-discuss-elastic-co-t-possible-to-highlight-inner-hits-in-percolate-query-91926/335261 "2023-06-07T15:35:48Z")
Is this currently supported? --- ## [How to set \`index.codec: best\_compression\` as the default for all future indices?](https://discuss.elastic.co/t/how-to-set-index-codec-best-compression-as-the-default-for-all-future-indices/335383)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 2\ **Last updated:** [June 7, 2023, 3:22pm UTC](https://discuss.elastic.co/t/how-to-set-index-codec-best-compression-as-the-default-for-all-future-indices/335383 "2023-06-07T15:22:32Z")
Pretty much what the subject says. How to I turn on best\_compression as the default for all new indices? The docs explain how to do it per index. But I haven't found anything on setting it as the default. Nor has googl… --- ## [Problems creating a ILM correctly](https://discuss.elastic.co/t/problems-creating-a-ilm-correctly/335365)
**Author:** [@blacar](https://discuss.elastic.co/u/blacar)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 2:50pm UTC](https://discuss.elastic.co/t/problems-creating-a-ilm-correctly/335365 "2023-06-07T14:50:39Z")
I am having problems creating an ILM that fits my needs. I have it done in another cluster but even trying to replicate it piece by piece ends in errors. Context: This is on Elastic Cloud using latest ES version I wi… --- ## [How to queue ECS formatted logs through RabbitMQ](https://discuss.elastic.co/t/how-to-queue-ecs-formatted-logs-through-rabbitmq/335105)
**Author:** [@bvoros](https://discuss.elastic.co/u/bvoros)\ **Replies:** 3\ **Last updated:** [June 7, 2023, 1:48pm UTC](https://discuss.elastic.co/t/how-to-queue-ecs-formatted-logs-through-rabbitmq/335105 "2023-06-07T13:48:16Z")
Hello all, Our logging infrastructure is the following: log shippers -\> logstash -\> rabbitmq -\> logstash -\> elasticsearch I am trying to start using ECS, have the template set up. However, when the first logstash plac… --- ## [How can I delete documents 3 months older?](https://discuss.elastic.co/t/how-can-i-delete-documents-3-months-older/335351)
**Author:** [@Mary2022](https://discuss.elastic.co/u/Mary2022)\ **Replies:** 5\ **Last updated:** [June 7, 2023, 1:41pm UTC](https://discuss.elastic.co/t/how-can-i-delete-documents-3-months-older/335351 "2023-06-07T13:41:09Z")
I have Elasticsearch and Kibana 8.6 and I have an index with a size of 115GB. I would like to query by @timestamp and delete documents older than April 1, 2023. How can I do that? I am new to the query part and not sure … --- ## [How can I increase the JVM via API or CLI](https://discuss.elastic.co/t/how-can-i-increase-the-jvm-via-api-or-cli/335443)
**Author:** [@c.j.t](https://discuss.elastic.co/u/c.j.t)\ **Replies:** 2\ **Last updated:** [June 7, 2023, 1:27pm UTC](https://discuss.elastic.co/t/how-can-i-increase-the-jvm-via-api-or-cli/335443 "2023-06-07T13:27:12Z")
I've an instance on AWS Opensearch Service that is has a circuit breaking exception, from reading I think increasing the jvm should work but all the examples seem to tell me to edit the yml - which I can't do - I can use… --- ## [Elastic Agent, Custom API Integration - GET Next URL from JSON response](https://discuss.elastic.co/t/elastic-agent-custom-api-integration-get-next-url-from-json-response/335104)
**Author:** [@Mark\_Campbell](https://discuss.elastic.co/u/Mark_Campbell)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 1:24pm UTC](https://discuss.elastic.co/t/elastic-agent-custom-api-integration-get-next-url-from-json-response/335104 "2023-06-07T13:24:35Z")
I'm using ES, Kibana and Agent version 8.8.0. I can use the Custom API Integration to get the JSON response from the API. API URL: https://example.com/api/data/?page=1 JSON Response: { "data": \[ { "attri… --- ## [Compatibility rabbitmq 3.11.9 with metricbeat 7.17.6](https://discuss.elastic.co/t/compatibility-rabbitmq-3-11-9-with-metricbeat-7-17-6/334943)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 1:17pm UTC](https://discuss.elastic.co/t/compatibility-rabbitmq-3-11-9-with-metricbeat-7-17-6/334943 "2023-06-07T13:17:50Z")
Hello, The metricbeat (v7.17.6) rabbitmq modules works fine with my rabbitMq 3.7.3. I plan to upgrade RabbitMQ to 3.11.9 version but I have faced a problem with the node module : ERROR module/wrapper.go:259 Error fetch… --- ## [Special characters handling](https://discuss.elastic.co/t/special-characters-handling/335294)
**Author:** [@aetius](https://discuss.elastic.co/u/aetius)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 12:01pm UTC](https://discuss.elastic.co/t/special-characters-handling/335294 "2023-06-07T12:01:53Z")
Hi Folks I was fixing a bug which came through the upgrade from Spring 2.5 to Spring 3.0. Now before in Spring 2.5 we had custom method from a repo that extended the ElasticsearchRepository interface, and in the method… --- ## [Timeout on Kibana](https://discuss.elastic.co/t/timeout-on-kibana/334444)
**Author:** [@mihai.radulescu](https://discuss.elastic.co/u/mihai.radulescu)\ **Replies:** 8\ **Last updated:** [June 7, 2023, 12:01pm UTC](https://discuss.elastic.co/t/timeout-on-kibana/334444 "2023-06-07T12:01:00Z")
Hello, Getting this error on Kibana graph is I select the period higher than 5 days (probably too many datapoints): Tried increasing elasticsearch.requestTimeout: 900000 (and restarted kibana service) but this messa… --- ## [Add\_docker\_metadata cannot process containers that already exited](https://discuss.elastic.co/t/add-docker-metadata-cannot-process-containers-that-already-exited/335435)
**Author:** [@Maciej\_Piasecki](https://discuss.elastic.co/u/Maciej_Piasecki)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 11:46am UTC](https://discuss.elastic.co/t/add-docker-metadata-cannot-process-containers-that-already-exited/335435 "2023-06-07T11:46:29Z")
For the input type container if the log file is discovered after the container is stopped, the add\_metadata\_processor reports {"file.name":"add\_docker\_metadata/add\_docker\_metadata.go","file.line":213},"message":"Contain… --- ## [Rerunning markdown in intervals](https://discuss.elastic.co/t/rerunning-markdown-in-intervals/335345)
**Author:** [@witwit](https://discuss.elastic.co/u/witwit)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 11:33am UTC](https://discuss.elastic.co/t/rerunning-markdown-in-intervals/335345 "2023-06-07T11:33:08Z")
Hi everyone, So I've been trying to get Kibana to load up images on dashboard right now. What I did is basically create a python api for Markdown in Kibana to get url to image to put on dashboard. Anyways, what happen… --- ## [How to define routing allocation of apm server index template](https://discuss.elastic.co/t/how-to-define-routing-allocation-of-apm-server-index-template/335244)
**Author:** [@Kay\_Khan](https://discuss.elastic.co/u/Kay_Khan)\ **Replies:** 5\ **Last updated:** [June 7, 2023, 10:55am UTC](https://discuss.elastic.co/t/how-to-define-routing-allocation-of-apm-server-index-template/335244 "2023-06-07T10:55:30Z")
We would like to define the routing allocation of all index templates created by the apm server so that they are stored on our dedicated nodeset group "monitoring" This is what we have attempted but unfortunately we fin… --- ## [Vega visualization](https://discuss.elastic.co/t/vega-visualization/335334)
**Author:** [@Alice\_Ionescu](https://discuss.elastic.co/u/Alice_Ionescu)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 10:46am UTC](https://discuss.elastic.co/t/vega-visualization/335334 "2023-06-07T10:46:01Z")
Hello, I am trying to display records from a document in a table, horizontally, by using vega. I have this: retrieves data from index pattern users-\*, and displays the userName and the manager { "$schema": "https://… --- ## [Schema Registry integration with Logstash kafka input plugin](https://discuss.elastic.co/t/schema-registry-integration-with-logstash-kafka-input-plugin/335431)
**Author:** [@Hemanth\_Gowda](https://discuss.elastic.co/u/Hemanth_Gowda)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 10:41am UTC](https://discuss.elastic.co/t/schema-registry-integration-with-logstash-kafka-input-plugin/335431 "2023-06-07T10:41:34Z")
Hi All, We are trying to setup Kafka Schema registry integration with Logstash. However we have below questions to understand before we start with. Can someone please help with this. We have multiple dynamic schemas … --- ## [Recommended RDMS ingestion approach can lead to lost updates](https://discuss.elastic.co/t/recommended-rdms-ingestion-approach-can-lead-to-lost-updates/332664)
**Author:** [@Alex\_McAusland](https://discuss.elastic.co/u/Alex_McAusland)\ **Replies:** 4\ **Last updated:** [June 7, 2023, 10:09am UTC](https://discuss.elastic.co/t/recommended-rdms-ingestion-approach-can-lead-to-lost-updates/332664 "2023-06-07T10:09:59Z")
The official RDMS ingestion docs recommend an approach based on tracking row modification time in the sql\_last\_value of the jdbc plugin. However this does not seem to account for database transactions; a row's modificat… --- ## [Logstash mysql](https://discuss.elastic.co/t/logstash-mysql/335400)
**Author:** [@adimi\_worou](https://discuss.elastic.co/u/adimi_worou)\ **Replies:** 4\ **Last updated:** [June 7, 2023, 9:56am UTC](https://discuss.elastic.co/t/logstash-mysql/335400 "2023-06-07T09:56:29Z")
Hi, i’ve the same problem. Logstash can’t load data from mysql db to elasticsearch. I use docker. Thanks for your help --- ## [Issue with sending apache logs to elasticsearch with different indices](https://discuss.elastic.co/t/issue-with-sending-apache-logs-to-elasticsearch-with-different-indices/335424)
**Author:** [@Akshaychdev](https://discuss.elastic.co/u/Akshaychdev)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 9:49am UTC](https://discuss.elastic.co/t/issue-with-sending-apache-logs-to-elasticsearch-with-different-indices/335424 "2023-06-07T09:49:21Z")
I am new to ELK and I want to use filebeat to fetch and transfer apache access and error logs to elasticsearch index directly. However, I need to send the logs to different indices (rather than the default filebeat\* inde… --- ## [Event.type field in system module logs not ECS compliant](https://discuss.elastic.co/t/event-type-field-in-system-module-logs-not-ecs-compliant/333579)
**Author:** [@Lorygold](https://discuss.elastic.co/u/Lorygold)\ **Replies:** 0\ **Last updated:** [May 16, 2023, 1:59pm UTC](https://discuss.elastic.co/t/event-type-field-in-system-module-logs-not-ecs-compliant/333579 "2023-05-16T13:59:40Z")
Good morning, I activated the system module of Filebeat (version 8.7.1) in order to collect the ssh logins on an Ubuntu VM. I can see them on Kibana, but the event.type field is info event if it is an authentication log… --- ## [Adding Lookup field to Kibana dataview](https://discuss.elastic.co/t/adding-lookup-field-to-kibana-dataview/335389)
**Author:** [@nitinkapoor](https://discuss.elastic.co/u/nitinkapoor)\ **Replies:** 1\ **Last updated:** [June 7, 2023, 9:44am UTC](https://discuss.elastic.co/t/adding-lookup-field-to-kibana-dataview/335389 "2023-06-07T09:44:22Z")
I have a data-view build using a transaction details index , I am showing transaction details which also has customer id , Can I also add lookup field to get customer name from customer index on the basis of customer id … --- ## [Auditbeat \>=8, logstash, and elasticsearch data stream](https://discuss.elastic.co/t/auditbeat-8-logstash-and-elasticsearch-data-stream/335357)
**Author:** [@Mike\_Williams](https://discuss.elastic.co/u/Mike_Williams)\ **Replies:** 2\ **Last updated:** [June 7, 2023, 9:37am UTC](https://discuss.elastic.co/t/auditbeat-8-logstash-and-elasticsearch-data-stream/335357 "2023-06-07T09:37:50Z")
Hey, I'm preparing to upgrade a set of auditbeat agents from 7.17 to 8.something. Clients are not allowed to talk directly to elasticsearch, all messages go through logstash. More than happy with the requirement to us… --- ## [Can't (yet) decode flowset id 256 from source id 0, because no template to decode it with has been received. This message will usually go away after 1 minute on logstash 7.17 and elasticsearch 7.17](https://discuss.elastic.co/t/cant-yet-decode-flowset-id-256-from-source-id-0-because-no-template-to-decode-it-with-has-been-received-this-message-will-usually-go-away-after-1-minute-on-logstash-7-17-and-elasticsearch-7-17/335421)
**Author:** [@Hanginium65](https://discuss.elastic.co/u/Hanginium65)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 9:32am UTC](https://discuss.elastic.co/t/cant-yet-decode-flowset-id-256-from-source-id-0-because-no-template-to-decode-it-with-has-been-received-this-message-will-usually-go-away-after-1-minute-on-logstash-7-17-and-elasticsearch-7-17/335421 "2023-06-07T09:32:33Z")
Hi, my config file for logstash looks like this: input { snmp { hosts =\> \[{host =\> "udp:192.168.56.3/161" version =\> "3"}\] get =\> \["1.3.6.1.2.1.25.3.3.1.2.1", "1.3.6.1.2.1.25.2.3.1.5.65536", "1.3.6.1.2.1.25.2… --- ## [Multiple filter for query not working as expected](https://discuss.elastic.co/t/multiple-filter-for-query-not-working-as-expected/335388)
**Author:** [@Atul\_Chadha](https://discuss.elastic.co/u/Atul_Chadha)\ **Replies:** 3\ **Last updated:** [June 7, 2023, 9:11am UTC](https://discuss.elastic.co/t/multiple-filter-for-query-not-working-as-expected/335388 "2023-06-07T09:11:41Z")
We are using elasticsearch as backend for our Wazuh cluster, i am trying to filter our results which contain values from "filter 01" and exclude results from "filter 02" however it looks its not working and showing resu… [Previous page](https://discuss.elastic.co/latest.md?page=648) [Next page](https://discuss.elastic.co/latest.md?page=650)