# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=126 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 127 --- ## [Transforming (aggregating) network logs 14 million records every 5 min](https://discuss.elastic.co/t/transforming-aggregating-network-logs-14-million-records-every-5-min/371560)
**Author:** [@sid\_shah](https://discuss.elastic.co/u/sid_shah)\ **Replies:** 8\ **Last updated:** [December 7, 2024, 2:39pm UTC](https://discuss.elastic.co/t/transforming-aggregating-network-logs-14-million-records-every-5-min/371560 "2024-12-07T14:39:33Z")
I have 14 million records arriving every 5 minutes from Filebeat into a data stream (NetFlow data). My goal is to aggregate this data based on certain rules, such as summing source bytes and destination bytes for sp… --- ## [Nested data for Kibana analyzis](https://discuss.elastic.co/t/nested-data-for-kibana-analyzis/371640)
**Author:** [@leogior](https://discuss.elastic.co/u/leogior)\ **Replies:** 0\ **Last updated:** [December 7, 2024, 12:33pm UTC](https://discuss.elastic.co/t/nested-data-for-kibana-analyzis/371640 "2024-12-07T12:33:32Z")
Hi all, I am new to Kibana and I am trying to create a dashboard with nested data. The idea would be to create live orderbook dashboard. The kind of document I create is : \[{'index': {'\_index': 'live-prices-binance\_',… --- ## [Elasticsearch ingestion : Geo point and numerical fields not recognize](https://discuss.elastic.co/t/elasticsearch-ingestion-geo-point-and-numerical-fields-not-recognize/371623)
**Author:** [@Phildefer](https://discuss.elastic.co/u/Phildefer)\ **Replies:** 12\ **Last updated:** [December 7, 2024, 3:28am UTC](https://discuss.elastic.co/t/elasticsearch-ingestion-geo-point-and-numerical-fields-not-recognize/371623 "2024-12-07T03:28:30Z")
Hi all, I'm trying to ingest into an index my data (json) with a HTTP POST. Everything works fine but I have a geo point like this { "point\_location" : "11.0500,-0.2500" } which is not recognized as a geo point but like… --- ## [Searches never distribute across nodes in different AWS Availability Zones](https://discuss.elastic.co/t/searches-never-distribute-across-nodes-in-different-aws-availability-zones/371491)
**Author:** [@hatertot](https://discuss.elastic.co/u/hatertot)\ **Replies:** 1\ **Last updated:** [December 6, 2024, 11:01pm UTC](https://discuss.elastic.co/t/searches-never-distribute-across-nodes-in-different-aws-availability-zones/371491 "2024-12-06T23:01:41Z")
I have a cluster of about 19 nodes on ES 7.17.1. They are all definitely part of the same cluster. Each node has an entire copy of every index. 11 are in us-east-1f, 6 are in us-east-1d, 1 is in 1c and 1 is in 1b. … --- ## [Inference on an image](https://discuss.elastic.co/t/inference-on-an-image/368908)
**Author:** [@martin-k](https://discuss.elastic.co/u/martin-k)\ **Replies:** 3\ **Last updated:** [December 6, 2024, 10:51pm UTC](https://discuss.elastic.co/t/inference-on-an-image/368908 "2024-12-06T22:51:34Z")
Hello all, Is it possible to get a vector embedding for an image from an inference endpoint using the sentence-transformers\_\_clip-vit-b-32-multilingual-v1 model? And if so, what format should the input be? The document… --- ## [How can I compare similar indexes](https://discuss.elastic.co/t/how-can-i-compare-similar-indexes/370011)
**Author:** [@mortenb123](https://discuss.elastic.co/u/mortenb123)\ **Replies:** 3\ **Last updated:** [December 6, 2024, 4:06pm UTC](https://discuss.elastic.co/t/how-can-i-compare-similar-indexes/370011 "2024-12-06T16:06:51Z")
Events show up in kibana: but in Elastic dev console I cant see anything: if I just change the index it works for fido2.trk.events-test12\* but not fido2.demo.events-test12\* I've used stack management to check i… --- ## [Extracting and mapping fields from event.original in Trend Micro Vision One Integration](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206)
**Author:** [@cyberm](https://discuss.elastic.co/u/cyberm)\ **Replies:** 12\ **Last updated:** [December 6, 2024, 3:16pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206 "2024-12-06T15:16:06Z")
Hi everyone, I’m trying to properly configure the Vision One integration in Kibana to collect and manage logs. Here’s what I’ve done so far: I’ve set up the Vision One integration in Kibana. I’ve configured the policy… --- ## [Custom fields.yml not loaded](https://discuss.elastic.co/t/custom-fields-yml-not-loaded/371583)
**Author:** [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\ **Replies:** 2\ **Last updated:** [December 6, 2024, 2:54pm UTC](https://discuss.elastic.co/t/custom-fields-yml-not-loaded/371583 "2024-12-06T14:54:23Z")
I have changed the mapping of a field in fields.yml and saved it as fields-patched.yml in C:\\Program Files\\Winlogbeat\\. In winlogbeat.yml, I have set this: setup.template.fields: "${path.config}/fields-patched.yml" I … --- ## [Http output and curl eqvalent](https://discuss.elastic.co/t/http-output-and-curl-eqvalent/371547)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 6\ **Last updated:** [December 6, 2024, 2:05pm UTC](https://discuss.elastic.co/t/http-output-and-curl-eqvalent/371547 "2024-12-06T14:05:07Z")
Good Day, I am trying to use the logstash http output command to replicate the following curl command? curl -k PATCH https://etp.us.example.com/api/v1/policies/123455/deny -H "Content-Type: application/json" -H "x-exam… --- ## [ILM Policy Update via filebeat.yml not functioning](https://discuss.elastic.co/t/ilm-policy-update-via-filebeat-yml-not-functioning/371294)
**Author:** [@developer\_cloud](https://discuss.elastic.co/u/developer_cloud)\ **Replies:** 1\ **Last updated:** [December 6, 2024, 12:09pm UTC](https://discuss.elastic.co/t/ilm-policy-update-via-filebeat-yml-not-functioning/371294 "2024-12-06T12:09:08Z")
Hello, I have a filebeat.yml thats sending logs from my server to kibana. For the input streams configured in the filebeat.yml, there is also an ILM Policy enabled. # ================================= ILM Configure ====… --- ## [Export via API of Exception List returns {}](https://discuss.elastic.co/t/export-via-api-of-exception-list-returns/371595)
**Author:** [@amoser](https://discuss.elastic.co/u/amoser)\ **Replies:** 0\ **Last updated:** [December 6, 2024, 11:21am UTC](https://discuss.elastic.co/t/export-via-api-of-exception-list-returns/371595 "2024-12-06T11:21:28Z")
Hello there, I'm trying to export an exception list via the Kibana API according to Export exception list. I'm testing with Console at Dev Tools: POST kbn:/api/exception\_lists/\_export?id=81039bc7-8b28-4435-8976-77a41d… --- ## [Date mapping issue](https://discuss.elastic.co/t/date-mapping-issue/371575)
**Author:** [@rmdevheart](https://discuss.elastic.co/u/rmdevheart)\ **Replies:** 3\ **Last updated:** [December 6, 2024, 11:12am UTC](https://discuss.elastic.co/t/date-mapping-issue/371575 "2024-12-06T11:12:26Z")
HI all, can't fight with it anymore, need help. I have an input 'localtime' field with text formatted as '2024-12-06 08:40:36.000404894'. Index mapping for this: "localtime": { "type": "date", "f… --- ## [Compare two fields in Kibana](https://discuss.elastic.co/t/compare-two-fields-in-kibana/369399)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 5\ **Last updated:** [December 6, 2024, 11:12am UTC](https://discuss.elastic.co/t/compare-two-fields-in-kibana/369399 "2024-12-06T11:12:04Z")
Hi, regarding is this currenly possible to compare values of 2 fields? like whether the content of field X is different/the same as content of field Y --- ## [Sflow Codec Plugin is not being installed locally](https://discuss.elastic.co/t/sflow-codec-plugin-is-not-being-installed-locally/371593)
**Author:** [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\ **Replies:** 0\ **Last updated:** [December 6, 2024, 10:59am UTC](https://discuss.elastic.co/t/sflow-codec-plugin-is-not-being-installed-locally/371593 "2024-12-06T10:59:47Z")
We have logstash logstash-8.15.2 locally but on production its not 8.7 and we don't have internet access, so i have dowloaded the gem file from \[logstash-codec-sflow | RubyGems.org | your community gem host\] (logstash… --- ## [How to pass curl argument in the filebeat.yml file](https://discuss.elastic.co/t/how-to-pass-curl-argument-in-the-filebeat-yml-file/371586)
**Author:** [@saxena\_shk7](https://discuss.elastic.co/u/saxena_shk7)\ **Replies:** 0\ **Last updated:** [December 6, 2024, 10:18am UTC](https://discuss.elastic.co/t/how-to-pass-curl-argument-in-the-filebeat-yml-file/371586 "2024-12-06T10:18:46Z")
I have an API end point which fetches traffic statistics from a Network Packet Broker, its curl equivalent command is: curl -s -k -X 'POST' 'https://10.0.0.1:8000/stats' -H ' accept:application/json' -H 'Authorization: … --- ## [Issue between Fleet-managed Elastic agent on external server and Fleet in k8s](https://discuss.elastic.co/t/issue-between-fleet-managed-elastic-agent-on-external-server-and-fleet-in-k8s/358911)
**Author:** [@LazyAgent](https://discuss.elastic.co/u/LazyAgent)\ **Replies:** 2\ **Last updated:** [December 6, 2024, 10:14am UTC](https://discuss.elastic.co/t/issue-between-fleet-managed-elastic-agent-on-external-server-and-fleet-in-k8s/358911 "2024-12-06T10:14:26Z")
Hi, My fleet-managed agent can't send logs (they are always dropped) to ES, but does send the metrics. The error message seen in the agent's status is a 504 Gateway Time-out. Additionally, we see this message in the ag… --- ## [Filebeat SSL logs reporting 'tls: failed to send closeNotify alert'](https://discuss.elastic.co/t/filebeat-ssl-logs-reporting-tls-failed-to-send-closenotify-alert/361346)
**Author:** [@wcraven](https://discuss.elastic.co/u/wcraven)\ **Replies:** 1\ **Last updated:** [December 6, 2024, 10:07am UTC](https://discuss.elastic.co/t/filebeat-ssl-logs-reporting-tls-failed-to-send-closenotify-alert/361346 "2024-12-06T10:07:44Z")
I have managed to implement an SSL connection between a FileBeat (8.13) agent and a LogStash (8.13) but have noticed in the FileBeat syslog logs that the SSL connections to the LogStash server are getting reset and these… --- ## [What is the maximum allowed length for any search string with \* wildcard?](https://discuss.elastic.co/t/what-is-the-maximum-allowed-length-for-any-search-string-with-wildcard/371581)
**Author:** [@priyankaMS](https://discuss.elastic.co/u/priyankaMS)\ **Replies:** 1\ **Last updated:** [December 6, 2024, 10:03am UTC](https://discuss.elastic.co/t/what-is-the-maximum-allowed-length-for-any-search-string-with-wildcard/371581 "2024-12-06T10:03:20Z")
In my Elasticsearch environment, for different asterisk (\*) wildcard positions in string I am getting below error: ES Exception: \[HTTP Status Code: \[400\] ORIGINAL\_EXCEPTION: \[Elasticsearch.Net.ElasticsearchClientExcep… --- ## [How should I write this statement?](https://discuss.elastic.co/t/how-should-i-write-this-statement/371584)
**Author:** [@nIxedoahz](https://discuss.elastic.co/u/nIxedoahz)\ **Replies:** 5\ **Last updated:** [December 6, 2024, 9:56am UTC](https://discuss.elastic.co/t/how-should-i-write-this-statement/371584 "2024-12-06T09:56:48Z")
Create an index ··· PUT /new\_index { "mappings": { "properties": { "data": { "type": "keyword" }, "hash": { "type": "integer" }, "hash\_mapping": { "type": "keyword" }, "task\_id": { "type": "keyword" } } … --- ## [ESET Protect CLOUD logs send to Logstash](https://discuss.elastic.co/t/eset-protect-cloud-logs-send-to-logstash/369803)
**Author:** [@Roverboy](https://discuss.elastic.co/u/Roverboy)\ **Replies:** 7\ **Last updated:** [December 6, 2024, 9:55am UTC](https://discuss.elastic.co/t/eset-protect-cloud-logs-send-to-logstash/369803 "2024-12-06T09:55:04Z")
Hi All, I have a problem with the logs received by my ESET POTECT CLOUD Console. This is what i received : "\\u0000\\u0000\\u0011\\u0000\\u000F\\u0000\\u0000\\f82.64.237.88\\u0000\\v\\u0000\\u0004\\u0003\\u0000\\u0001\\u0002\\u0000\\n" … --- ## [How to delete alerting logs](https://discuss.elastic.co/t/how-to-delete-alerting-logs/371573)
**Author:** [@FJT](https://discuss.elastic.co/u/FJT)\ **Replies:** 0\ **Last updated:** [December 6, 2024, 8:44am UTC](https://discuss.elastic.co/t/how-to-delete-alerting-logs/371573 "2024-12-06T08:44:01Z")
I create an alert rule in my kibana and it has interval like 1m so it will also generate logs every 1m. How can I delete the logs of my alerting? Thank you! --- ## [Need matching element from array objects in aggregation like below](https://discuss.elastic.co/t/need-matching-element-from-array-objects-in-aggregation-like-below/371444)
**Author:** [@krishna\_kishore](https://discuss.elastic.co/u/krishna_kishore)\ **Replies:** 1\ **Last updated:** [December 6, 2024, 8:23am UTC](https://discuss.elastic.co/t/need-matching-element-from-array-objects-in-aggregation-like-below/371444 "2024-12-06T08:23:32Z")
URL: /products/\_doc/\_search?routing=DEFAULT&filter\_path=took,hits.hits.\_id,aggregations.filters..buckets.key,aggregations.filters. .buckets.key\_as\_string,aggregations.filters..buckets.doc\_count,hits.hits.\_source,aggregat… --- ## [Elasticsearch Unable to Fetch Client Secret from Keystore for OIDC Configuration (Version 7.17/8.x)](https://discuss.elastic.co/t/elasticsearch-unable-to-fetch-client-secret-from-keystore-for-oidc-configuration-version-7-17-8-x/371530)
**Author:** [@Krish\_Shewani](https://discuss.elastic.co/u/Krish_Shewani)\ **Replies:** 0\ **Last updated:** [December 5, 2024, 11:42am UTC](https://discuss.elastic.co/t/elasticsearch-unable-to-fetch-client-secret-from-keystore-for-oidc-configuration-version-7-17-8-x/371530 "2024-12-05T11:42:52Z")
Hello, I am attempting to enable Single Sign-On (SSO) in Elasticsearch using Keycloak as the identity provider, by configuring OIDC (OpenID Connect) authentication. Following the official Elasticsearch documentation for… --- ## [Elastic Agent in Fleet producing 1000s of log entries per hours about CA certs](https://discuss.elastic.co/t/elastic-agent-in-fleet-producing-1000s-of-log-entries-per-hours-about-ca-certs/360576)
**Author:** [@fstlouis](https://discuss.elastic.co/u/fstlouis)\ **Replies:** 4\ **Last updated:** [December 5, 2024, 9:38pm UTC](https://discuss.elastic.co/t/elastic-agent-in-fleet-producing-1000s-of-log-entries-per-hours-about-ca-certs/360576 "2024-12-05T21:38:15Z")
I'm starting an on premise Elastic 8.12 deployment. I'm using Fleet managed agents with my own certificates (Entrust). I currentely have about 10 agents enrolled. I've noticed that I'm getting a very high volume of th… --- ## [Variable Width Histogram not working with top hits subaggregation](https://discuss.elastic.co/t/variable-width-histogram-not-working-with-top-hits-subaggregation/370865)
**Author:** [@vijay267](https://discuss.elastic.co/u/vijay267)\ **Replies:** 2\ **Last updated:** [December 5, 2024, 8:43pm UTC](https://discuss.elastic.co/t/variable-width-histogram-not-working-with-top-hits-subaggregation/370865 "2024-12-05T20:43:57Z")
I'm trying to use variable width histogram to group results with similar relevance scores and then apply a secondary sort on them via a top hits aggregation. This is so that folks can sort on relevance and then also sort… --- ## [Dynamic Index Names via Filebeat](https://discuss.elastic.co/t/dynamic-index-names-via-filebeat/371559)
**Author:** [@dfir](https://discuss.elastic.co/u/dfir)\ **Replies:** 1\ **Last updated:** [December 5, 2024, 7:57pm UTC](https://discuss.elastic.co/t/dynamic-index-names-via-filebeat/371559 "2024-12-05T19:57:18Z")
I am tying to use filebeat to ingest IIS logs. I can get this to work with basic configs in the iis.yml. HOWEVER: there is a new need to be able to have filebeat create dynamic index names for IIS logs based on a fol… --- ## [Purpose of "Data Retention" in index template](https://discuss.elastic.co/t/purpose-of-data-retention-in-index-template/371552)
**Author:** [@Stephen\_IAnson](https://discuss.elastic.co/u/Stephen_IAnson)\ **Replies:** 2\ **Last updated:** [December 5, 2024, 5:15pm UTC](https://discuss.elastic.co/t/purpose-of-data-retention-in-index-template/371552 "2024-12-05T17:15:02Z")
Hi, Can someone explain to me what the purpose of the data retention field is in an index template? My understanding is that the ILM is responsible for moving data between phases, rollover and optionally deletion. What… --- ## [ELK Replication- Doc count mismatch between the leader & follower](https://discuss.elastic.co/t/elk-replication-doc-count-mismatch-between-the-leader-follower/371546)
**Author:** [@sudhakartata](https://discuss.elastic.co/u/sudhakartata)\ **Replies:** 0\ **Last updated:** [December 5, 2024, 3:40pm UTC](https://discuss.elastic.co/t/elk-replication-doc-count-mismatch-between-the-leader-follower/371546 "2024-12-05T15:40:17Z")
The doc count in the follower is less than the count from leader. Is that normal? Or is that an issue that needs to be looked into? Was wondering it's a bug. Any help is appreciated --- ## [Receive syslog logs and forward copy to another server](https://discuss.elastic.co/t/receive-syslog-logs-and-forward-copy-to-another-server/371480)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 2\ **Last updated:** [December 5, 2024, 12:54pm UTC](https://discuss.elastic.co/t/receive-syslog-logs-and-forward-copy-to-another-server/371480 "2024-12-05T12:54:43Z")
Hello, I need to receive them via syslog through logstash, process them and send them to the elasticsearch cluster, but I also need the original logs to go a copy to another server to another SIEM that I have. I would … --- ## [How to apply new ILM policy to existing datastream index](https://discuss.elastic.co/t/how-to-apply-new-ilm-policy-to-existing-datastream-index/371531)
**Author:** [@racitup](https://discuss.elastic.co/u/racitup)\ **Replies:** 1\ **Last updated:** [December 5, 2024, 11:55am UTC](https://discuss.elastic.co/t/how-to-apply-new-ilm-policy-to-existing-datastream-index/371531 "2024-12-05T11:55:37Z")
I found several posts about this topic but none gave the actual answer, so thought I would post: I had an existing index on a datastream (filebeat) which had the default 30d ILM policy assigned, but was growing large. I… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=125) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=127)