# Latest

**URL:** https://discuss.elastic.co/latest.md?page=172

[Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md)

**Page:** 173

---

## [I want to deploy elasticsearch cluster with 3 master node and 3 data node and 1 kibana node using ECK Operator on my kubernetes cluster](https://discuss.elastic.co/t/i-want-to-deploy-elasticsearch-cluster-with-3-master-node-and-3-data-node-and-1-kibana-node-using-eck-operator-on-my-kubernetes-cluster/372061)

<div class="topic-metadata">

**Author:** [@lourdhu](https://discuss.elastic.co/u/lourdhu)\
**Replies:** 2\
**Last updated:** [December 19, 2024, 1:47pm UTC](https://discuss.elastic.co/t/i-want-to-deploy-elasticsearch-cluster-with-3-master-node-and-3-data-node-and-1-kibana-node-using-eck-operator-on-my-kubernetes-cluster/372061 "2024-12-19T13:47:27Z")

</div>

I am following the below Elasticsearch home page documents for creating a Elasticsearch cluster with 3 master node and 3 data node and 1 Kibana node along with ingress using ECK Operator on my Kubernetes cluster. NOTE:…

---

## [Can I get the datepicker parameters to use in a KQL query in Kibana](https://discuss.elastic.co/t/can-i-get-the-datepicker-parameters-to-use-in-a-kql-query-in-kibana/372224)

<div class="topic-metadata">

**Author:** [@bjornbear](https://discuss.elastic.co/u/bjornbear)\
**Replies:** 0\
**Last updated:** [December 19, 2024, 1:41pm UTC](https://discuss.elastic.co/t/can-i-get-the-datepicker-parameters-to-use-in-a-kql-query-in-kibana/372224 "2024-12-19T13:41:43Z")

</div>

Is it possible yet to use the Datepicker parameters in a KQL or filter since I have several time fields that I would want to link to the selected time range.

---

## [API key definition](https://discuss.elastic.co/t/api-key-definition/372212)

<div class="topic-metadata">

**Author:** [@Joey\_Visbeen](https://discuss.elastic.co/u/Joey_Visbeen)\
**Replies:** 0\
**Last updated:** [December 19, 2024, 9:46am UTC](https://discuss.elastic.co/t/api-key-definition/372212 "2024-12-19T09:46:04Z")

</div>

I was hoping to start this discussion with somebody from Elastic. Currently, the team I work with have been actively using the REST API's from Kibana and Elasticsearch allot, but have come across the difficulty of settin…

---

## [Elastic 8.17 and \_source](https://discuss.elastic.co/t/elastic-8-17-and-source/372181)

<div class="topic-metadata">

**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Replies:** 2\
**Last updated:** [December 19, 2024, 1:13pm UTC](https://discuss.elastic.co/t/elastic-8-17-and-source/372181 "2024-12-19T13:13:20Z")

</div>

I was reading through 8.17.0 release note. first thing it says \_source is no longer available without subscription. is this means if you create index it won't have \_source and this means not able to update, update\_by\_q…

---

## [Re-attach ELK Elasticsearch PVC master in ECK Elasticsearch](https://discuss.elastic.co/t/re-attach-elk-elasticsearch-pvc-master-in-eck-elasticsearch/372222)

<div class="topic-metadata">

**Author:** [@rakeshbrnwl](https://discuss.elastic.co/u/rakeshbrnwl)\
**Replies:** 0\
**Last updated:** [December 19, 2024, 12:32pm UTC](https://discuss.elastic.co/t/re-attach-elk-elasticsearch-pvc-master-in-eck-elasticsearch/372222 "2024-12-19T12:32:58Z")

</div>

I am planning to migrate ELK stake to ECK stake in kubernetes. But I want to use same PVC (master/data) created earlier by Elasticsearch in ELK in Elasticsearch created by ECK Operator. In my helm release of eck-elasti…

---

## [Elasticsearch Configuration for 220 Million Records (400 GB Data)](https://discuss.elastic.co/t/elasticsearch-configuration-for-220-million-records-400-gb-data/372148)

<div class="topic-metadata">

**Author:** [@dhanunjayavarma](https://discuss.elastic.co/u/dhanunjayavarma)\
**Replies:** 3\
**Last updated:** [December 19, 2024, 12:18pm UTC](https://discuss.elastic.co/t/elasticsearch-configuration-for-220-million-records-400-gb-data/372148 "2024-12-19T12:18:07Z")

</div>

Hi, I am planning to develop a full-text search engine using Elasticsearch 8.x to handle approximately 220 million records, totaling around 400 GB of data. Each document consists of a mix of full-text fields and filter …

---

## [Parsing text and object](https://discuss.elastic.co/t/parsing-text-and-object/372219)

<div class="topic-metadata">

**Author:** [@vahagg1](https://discuss.elastic.co/u/vahagg1)\
**Replies:** 3\
**Last updated:** [December 19, 2024, 11:11am UTC](https://discuss.elastic.co/t/parsing-text-and-object/372219 "2024-12-19T11:11:05Z")

</div>

The problem which i face is like this: I have a single field like message which i want to parse it with json parser but sometimes the value which come to it is text and sometimes is json, I try to use regex like "^{.}\[\\…

---

## [Help with EQL Rule to Detect Unauthorized State Transitions for Traffic Lights](https://discuss.elastic.co/t/help-with-eql-rule-to-detect-unauthorized-state-transitions-for-traffic-lights/371956)

<div class="topic-metadata">

**Author:** [@Kuly2Fraise](https://discuss.elastic.co/u/Kuly2Fraise)\
**Replies:** 6\
**Last updated:** [December 19, 2024, 9:35am UTC](https://discuss.elastic.co/t/help-with-eql-rule-to-detect-unauthorized-state-transitions-for-traffic-lights/371956 "2024-12-19T09:35:54Z")

</div>

Hi Elastic Community, I’m currently working on a project where I need to detect unauthorized state transitions for traffic lights in a real-time event stream using Elastic EQL. I’ve already set up the state machine and …

---

## [Migrating from Elastic NEST v7 to .NET Elastic Client v8: Bulk indexing issue](https://discuss.elastic.co/t/migrating-from-elastic-nest-v7-to-net-elastic-client-v8-bulk-indexing-issue/371929)

<div class="topic-metadata">

**Author:** [@trubs](https://discuss.elastic.co/u/trubs)\
**Replies:** 4\
**Last updated:** [December 19, 2024, 8:59am UTC](https://discuss.elastic.co/t/migrating-from-elastic-nest-v7-to-net-elastic-client-v8-bulk-indexing-issue/371929 "2024-12-19T08:59:21Z")

</div>

I'm new to web development and working with Elastic, and I'm currently migrating a project from Elastic NEST v7 to the .NET Elastic Client v8. I've encountered an issue with bulk indexing in one of the projects. What wo…

---

## [If we can call API with designated cookie to retrieve the log before outputing to downstream in Filebeat](https://discuss.elastic.co/t/if-we-can-call-api-with-designated-cookie-to-retrieve-the-log-before-outputing-to-downstream-in-filebeat/372208)

<div class="topic-metadata">

**Author:** [@Jason9988](https://discuss.elastic.co/u/Jason9988)\
**Replies:** 0\
**Last updated:** [December 19, 2024, 8:15am UTC](https://discuss.elastic.co/t/if-we-can-call-api-with-designated-cookie-to-retrieve-the-log-before-outputing-to-downstream-in-filebeat/372208 "2024-12-19T08:15:22Z")

</div>

if we can call API with designated cookie to retrieve the log before outputting to downstream in Filebeat ? Scenarios: My server have generated the first batch 1000 records log with cookie information I need to extrac…

---

## [Reindex from remote. From v8.5.1 to v7.17.1 error](https://discuss.elastic.co/t/reindex-from-remote-from-v8-5-1-to-v7-17-1-error/372153)

<div class="topic-metadata">

**Author:** [@Khanh\_Tran13579](https://discuss.elastic.co/u/Khanh_Tran13579)\
**Replies:** 5\
**Last updated:** [December 19, 2024, 8:27am UTC](https://discuss.elastic.co/t/reindex-from-remote-from-v8-5-1-to-v7-17-1-error/372153 "2024-12-19T08:27:43Z")

</div>

I am trying to reindex from the remote server. The source version is 8.5.1 and the dest version is 7.17.1. I got the error below. Please help me explain this error and how to solve it. { "error": { "root\_cause": \[…

---

## [Dec 19th, 2024: \[EN\] Elastic observability and search in .NET Aspire](https://discuss.elastic.co/t/dec-19th-2024-en-elastic-observability-and-search-in-net-aspire/369417)

<div class="topic-metadata">

**Author:** [@Rossall](https://discuss.elastic.co/u/Rossall)\
**Replies:** 0\
**Last updated:** [December 19, 2024, 8:00am UTC](https://discuss.elastic.co/t/dec-19th-2024-en-elastic-observability-and-search-in-net-aspire/369417 "2024-12-19T08:00:22Z")

</div>

Intro .NET Aspire is an opinionated, cloud ready stack for building observable, production ready, distributed applications. In this post we'll look at working with Elasticsearch in .NET Aspire, but firstly - on the obse…

---

## [How to migrate index A to index B in Elasticsearch without stopping writes to index A?](https://discuss.elastic.co/t/how-to-migrate-index-a-to-index-b-in-elasticsearch-without-stopping-writes-to-index-a/372198)

<div class="topic-metadata">

**Author:** [@WangXiang727](https://discuss.elastic.co/u/WangXiang727)\
**Replies:** 4\
**Last updated:** [December 19, 2024, 7:38am UTC](https://discuss.elastic.co/t/how-to-migrate-index-a-to-index-b-in-elasticsearch-without-stopping-writes-to-index-a/372198 "2024-12-19T07:38:07Z")

</div>

I'm using Elasticsearch 7.10.0 and need to migrate data from index A to a new index B without interrupting write operations to index A. I want to achieve zero downtime during this migration and to migrate both the existi…

---

## [Issue with Logstash, ILM and Data stream](https://discuss.elastic.co/t/issue-with-logstash-ilm-and-data-stream/364897)

<div class="topic-metadata">

**Author:** [@fio23](https://discuss.elastic.co/u/fio23)\
**Replies:** 5\
**Last updated:** [December 19, 2024, 7:28am UTC](https://discuss.elastic.co/t/issue-with-logstash-ilm-and-data-stream/364897 "2024-12-19T07:28:34Z")

</div>

Hello all, I'm currently trying to create a data stream with a corresponding ILM policy enabled, and then have Logstash to forward logs to that data stream. See below steps I follow: Create Component Template PUT \_co…

---

## [Parsing confluence audit json into Elastic with logstash](https://discuss.elastic.co/t/parsing-confluence-audit-json-into-elastic-with-logstash/371798)

<div class="topic-metadata">

**Author:** [@Logistic\_dilated](https://discuss.elastic.co/u/Logistic_dilated)\
**Replies:** 2\
**Last updated:** [December 19, 2024, 3:02am UTC](https://discuss.elastic.co/t/parsing-confluence-audit-json-into-elastic-with-logstash/371798 "2024-12-19T03:02:26Z")

</div>

Hello, I'm testing scraping our company confluence (self hosted) audit api so permission changes etc. are in ELK (also self hosted). I've got an http\_poller pipeline that is successfully polling the api, however the ev…

---

## [Filebeat input stream missing log lines when file update in very infrequent](https://discuss.elastic.co/t/filebeat-input-stream-missing-log-lines-when-file-update-in-very-infrequent/372178)

<div class="topic-metadata">

**Author:** [@Shaishav](https://discuss.elastic.co/u/Shaishav)\
**Replies:** 0\
**Last updated:** [December 18, 2024, 5:34pm UTC](https://discuss.elastic.co/t/filebeat-input-stream-missing-log-lines-when-file-update-in-very-infrequent/372178 "2024-12-18T17:34:04Z")

</div>

Set up: Kubernetes pod saves the log file on host machine at location /var/log/app\_logs Filebeat 8.16.2 is configured as Daemonset to read from logfile and enrich the data Filebeat.yml filebeat.inputs: - type: filest…

---

## [API logs not coming after setting up Elastic Agent , Beats logs are coming](https://discuss.elastic.co/t/api-logs-not-coming-after-setting-up-elastic-agent-beats-logs-are-coming/372081)

<div class="topic-metadata">

**Author:** [@ErGeek](https://discuss.elastic.co/u/ErGeek)\
**Replies:** 14\
**Last updated:** [December 18, 2024, 6:18pm UTC](https://discuss.elastic.co/t/api-logs-not-coming-after-setting-up-elastic-agent-beats-logs-are-coming/372081 "2024-12-18T18:18:41Z")

</div>

Hi All, We have setup an Elastic Agent to send logs from an Application API to Kafka. We are able to establish the connection using the Elastic Agent. But instead of the API logs, we are getting Beat logs (Filebeat and…

---

## [How to use Alert Supression object in TOML format for threshold rules?](https://discuss.elastic.co/t/how-to-use-alert-supression-object-in-toml-format-for-threshold-rules/372175)

<div class="topic-metadata">

**Author:** [@obsidian](https://discuss.elastic.co/u/obsidian)\
**Replies:** 0\
**Last updated:** [December 18, 2024, 4:37pm UTC](https://discuss.elastic.co/t/how-to-use-alert-supression-object-in-toml-format-for-threshold-rules/372175 "2024-12-18T16:37:04Z")

</div>

I am trying to use the alert supression schema for a threshold rule but only the threshold field and value and cardinality field and value are showing up in elastic and not the alert supression, how do I add that into th…

---

## [Getting invalid SSL configuration Error while setting up password for default user](https://discuss.elastic.co/t/getting-invalid-ssl-configuration-error-while-setting-up-password-for-default-user/372159)

<div class="topic-metadata">

**Author:** [@amit\_tandel](https://discuss.elastic.co/u/amit_tandel)\
**Replies:** 1\
**Last updated:** [December 18, 2024, 2:55pm UTC](https://discuss.elastic.co/t/getting-invalid-ssl-configuration-error-while-setting-up-password-for-default-user/372159 "2024-12-18T14:55:49Z")

</div>

Hello I have installed Elasticsearch v7.17.25 as we are going to use Elastic Search for indexing/search for our Alfresco system. We enabled the Elasticsearch security features and when we try to setup password for defa…

---

## [Switch between the "document" and "summary" columns in Kibana Discover](https://discuss.elastic.co/t/switch-between-the-document-and-summary-columns-in-kibana-discover/372063)

<div class="topic-metadata">

**Author:** [@debuggedme](https://discuss.elastic.co/u/debuggedme)\
**Replies:** 6\
**Last updated:** [December 18, 2024, 3:10pm UTC](https://discuss.elastic.co/t/switch-between-the-document-and-summary-columns-in-kibana-discover/372063 "2024-12-18T15:10:46Z")

</div>

How to switch between the "document" and "summary" columns in Kibana Discover? In version 8.15, I saw "document," but in 8.16, I see "summary," and it is empty. The default column set to \_source doesn't seem to take …

---

## [Enhancing Ingestion Rate of Elastic Agent Reading from Azure Event Hub](https://discuss.elastic.co/t/enhancing-ingestion-rate-of-elastic-agent-reading-from-azure-event-hub/372162)

<div class="topic-metadata">

**Author:** [@RajuParipelly](https://discuss.elastic.co/u/RajuParipelly)\
**Replies:** 3\
**Last updated:** [December 18, 2024, 2:52pm UTC](https://discuss.elastic.co/t/enhancing-ingestion-rate-of-elastic-agent-reading-from-azure-event-hub/372162 "2024-12-18T14:52:07Z")

</div>

I have successfully deployed an Elastic Agent that interfaces with an Azure Event Hub. The current configuration processes approximately 5 to 10 million logs per hour, with individual log sizes ranging between 5 to 20 KB…

---

## [Snapshot restore data stream problems](https://discuss.elastic.co/t/snapshot-restore-data-stream-problems/372170)

<div class="topic-metadata">

**Author:** [@Dave\_Houser](https://discuss.elastic.co/u/Dave_Houser)\
**Replies:** 0\
**Last updated:** [December 18, 2024, 2:48pm UTC](https://discuss.elastic.co/t/snapshot-restore-data-stream-problems/372170 "2024-12-18T14:48:02Z")

</div>

Running 8.13 I am working on restoring a data stream from a snapshot. I am doing this in a surgical way as I need to be able to write to indexes in snapshots while the restore of all other indexes happens in the backgro…

---

## [Metricbeat issue: failed to parse: Limit of total fields \[10000\] has been exceeded while adding new fields](https://discuss.elastic.co/t/metricbeat-issue-failed-to-parse-limit-of-total-fields-10000-has-been-exceeded-while-adding-new-fields/369405)

<div class="topic-metadata">

**Author:** [@alexns](https://discuss.elastic.co/u/alexns)\
**Replies:** 5\
**Last updated:** [December 18, 2024, 2:06pm UTC](https://discuss.elastic.co/t/metricbeat-issue-failed-to-parse-limit-of-total-fields-10000-has-been-exceeded-while-adding-new-fields/369405 "2024-12-18T14:06:33Z")

</div>

We use the AWS module to push SQS and RDS metrics to Metricbeat. Everything is setup with ECK on Kubernetes and Helm. Recently, probably after updating to 8.15.2, these metrics weren't being pushed anymore. The metricb…

---

## [Logging fields with type long using logback-ecs-encoder](https://discuss.elastic.co/t/logging-fields-with-type-long-using-logback-ecs-encoder/372031)

<div class="topic-metadata">

**Author:** [@softarn](https://discuss.elastic.co/u/softarn)\
**Replies:** 1\
**Last updated:** [December 18, 2024, 1:20pm UTC](https://discuss.elastic.co/t/logging-fields-with-type-long-using-logback-ecs-encoder/372031 "2024-12-18T13:20:39Z")

</div>

I'm adding request/response logging to my Spring Boot application and I want to add, for example, the field http.request.body.bytes which is of type long. Currently we are adding the values we want to in MDC, logging an…

---

## [Set system time as timestamp](https://discuss.elastic.co/t/set-system-time-as-timestamp/372079)

<div class="topic-metadata">

**Author:** [@Taras\_Mikityuk](https://discuss.elastic.co/u/Taras_Mikityuk)\
**Replies:** 3\
**Last updated:** [December 18, 2024, 12:39pm UTC](https://discuss.elastic.co/t/set-system-time-as-timestamp/372079 "2024-12-18T12:39:55Z")

</div>

I'm collecting logs using elastic agent. The problem is, in @timestamp field I have time when event actually happened, but I also need time when log arrived in logstash. I was thinking to use ruby(ruby { code =\> …

---

## [Traditional Group By With Aggregations & Pagination](https://discuss.elastic.co/t/traditional-group-by-with-aggregations-pagination/372151)

<div class="topic-metadata">

**Author:** [@jespin](https://discuss.elastic.co/u/jespin)\
**Replies:** 2\
**Last updated:** [December 18, 2024, 11:59am UTC](https://discuss.elastic.co/t/traditional-group-by-with-aggregations-pagination/372151 "2024-12-18T11:59:28Z")

</div>

Hi all, We are looking to rewrite our current Elastic search implementation as it's not fit for purpose. We have a number of hotels which we index in as a single document with all of the packages available for that hot…

---

## [Logstash SNMP Interface Monitoring Configuration with Bandwidth Utilization Calculation](https://discuss.elastic.co/t/logstash-snmp-interface-monitoring-configuration-with-bandwidth-utilization-calculation/372068)

<div class="topic-metadata">

**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Replies:** 0\
**Last updated:** [December 17, 2024, 12:34pm UTC](https://discuss.elastic.co/t/logstash-snmp-interface-monitoring-configuration-with-bandwidth-utilization-calculation/372068 "2024-12-17T12:34:45Z")

</div>

Hi everyone, I'd like to share my Logstash configuration for monitoring network interfaces via SNMP with bandwidth utilization calculation. This configuration collects interface statistics from network devices (tested w…

---

## [Semantic Search in Java](https://discuss.elastic.co/t/semantic-search-in-java/372141)

<div class="topic-metadata">

**Author:** [@erkosh](https://discuss.elastic.co/u/erkosh)\
**Replies:** 1\
**Last updated:** [December 18, 2024, 9:09am UTC](https://discuss.elastic.co/t/semantic-search-in-java/372141 "2024-12-18T09:09:15Z")

</div>

I want to add a semantic search feature to my project but cannot find a good method. I tried using ELSER, but the libraries are deprecated. Is there any way to add semantic search in Java Spring?

---

## [Delete .monitoring index as it's unassigned elk 8.7](https://discuss.elastic.co/t/delete-monitoring-index-as-its-unassigned-elk-8-7/372144)

<div class="topic-metadata">

**Author:** [@amoghbarve](https://discuss.elastic.co/u/amoghbarve)\
**Replies:** 0\
**Last updated:** [December 18, 2024, 8:12am UTC](https://discuss.elastic.co/t/delete-monitoring-index-as-its-unassigned-elk-8-7/372144 "2024-12-18T08:12:16Z")

</div>

Hi All... Need one guidance I am using Elasticsearch 8.7 and my cluster status is yellow Upon checking I find that the . monitoring index is in unassigned status . Can it be deleted or dropped ? Is there any impact ? …

---

## [Snapshot Repository is 8x larger than total clustered data!?!?!](https://discuss.elastic.co/t/snapshot-repository-is-8x-larger-than-total-clustered-data/371625)

<div class="topic-metadata">

**Author:** [@ivanlawrence](https://discuss.elastic.co/u/ivanlawrence)\
**Replies:** 1\
**Last updated:** [December 18, 2024, 8:08am UTC](https://discuss.elastic.co/t/snapshot-repository-is-8x-larger-than-total-clustered-data/371625 "2024-12-18T08:08:16Z")

</div>

My cluster has a total usage reported by kibana (kibana/app/monitoring#/overview) as 1TB (102 indices with one replica.) Snapshots go to backblaze (as the s3 default client) which is reporting 8TB stored. I was snappin…

[Previous page](https://discuss.elastic.co/latest.md?page=171)

[Next page](https://discuss.elastic.co/latest.md?page=173)
