# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=265 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 266 --- ## [Set \_id during pipeline in bulk ingestion](https://discuss.elastic.co/t/set-id-during-pipeline-in-bulk-ingestion/357158)
**Author:** [@wnmills3](https://discuss.elastic.co/u/wnmills3)\ **Replies:** 5\ **Last updated:** [April 11, 2024, 12:04pm UTC](https://discuss.elastic.co/t/set-id-during-pipeline-in-bulk-ingestion/357158 "2024-04-11T12:04:10Z")
How can I set the \_id field to the value of the id property defined in my mappings? In my mappings I have an id field defined as follows: "id": { "type": "text", "fields": { "type": "keyword", "ign… --- ## [Lifecycle Policies per monitoring vm](https://discuss.elastic.co/t/lifecycle-policies-per-monitoring-vm/357194)
**Author:** [@Mr.Tomas](https://discuss.elastic.co/u/Mr.Tomas)\ **Replies:** 0\ **Last updated:** [April 11, 2024, 9:45am UTC](https://discuss.elastic.co/t/lifecycle-policies-per-monitoring-vm/357194 "2024-04-11T09:45:20Z")
Hello community! My first try with Elastic, still learning and reading the documentation. But decided to ask a question in parallel to my studies. I got a task to collect events from ~100 windows servers. I’ve install… --- ## [Upgrade from 8.3 to 8.13 in a dockerized environment](https://discuss.elastic.co/t/upgrade-from-8-3-to-8-13-in-a-dockerized-environment/357156)
**Author:** [@Patrik\_H](https://discuss.elastic.co/u/Patrik_H)\ **Replies:** 2\ **Last updated:** [April 11, 2024, 6:39am UTC](https://discuss.elastic.co/t/upgrade-from-8-3-to-8-13-in-a-dockerized-environment/357156 "2024-04-11T06:39:58Z")
Hi there, Is there any restricted path between minor version upgrades to go from 8.3 to 8.13 per minor versions or can i upgrade my stack from 8.3 to 8.13 in one step? I did not find any documentation which mentions thi… --- ## [Elasticsearch snapshot/restore to s3](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/357132)
**Author:** [@kannetisiva](https://discuss.elastic.co/u/kannetisiva)\ **Replies:** 1\ **Last updated:** [April 11, 2024, 5:03am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/357132 "2024-04-11T05:03:21Z")
HTTP/1.1 500 Internal Server Error X-elastic-product: Elasticsearch content-type: application/json;charset=utf-8 content-length: 654 {"error":{"root\_cause":\[{"type":"repository\_exception","reason":"\[s3\_repo\] Could no… --- ## [How to return bucket with 0 count in terms aggregation](https://discuss.elastic.co/t/how-to-return-bucket-with-0-count-in-terms-aggregation/357180)
**Author:** [@m-amano](https://discuss.elastic.co/u/m-amano)\ **Replies:** 0\ **Last updated:** [April 11, 2024, 1:05am UTC](https://discuss.elastic.co/t/how-to-return-bucket-with-0-count-in-terms-aggregation/357180 "2024-04-11T01:05:52Z")
I want to get return bucket even though the count is 0. How to get daily histogram buckets with 0 count?? My aggregation query is below. I want to every histogram buckets from last Sunday to Saturday. { "size": 0, … --- ## [Not getting any results for a field that requires exact match](https://discuss.elastic.co/t/not-getting-any-results-for-a-field-that-requires-exact-match/357102)
**Author:** [@Sujinthan](https://discuss.elastic.co/u/Sujinthan)\ **Replies:** 6\ **Last updated:** [April 10, 2024, 7:37pm UTC](https://discuss.elastic.co/t/not-getting-any-results-for-a-field-that-requires-exact-match/357102 "2024-04-10T19:37:08Z")
I'm building a search feature that allows users to filter by certain field. One of the field is DOI. Filtering by DOI should only return results with exact match. I'm trying to index the following object: public class … --- ## [Problem to Create index pattern](https://discuss.elastic.co/t/problem-to-create-index-pattern/357162)
**Author:** [@erickml0](https://discuss.elastic.co/u/erickml0)\ **Replies:** 2\ **Last updated:** [April 10, 2024, 7:36pm UTC](https://discuss.elastic.co/t/problem-to-create-index-pattern/357162 "2024-04-10T19:36:34Z")
Hi, community. I'm trying to create the index pattern (i don't have one), it contains 44,238 documents but the company didn't create the index at the beginning. I thought that was the amount of documents but hours have… --- ## [Resource estimation for calculating Max TPS](https://discuss.elastic.co/t/resource-estimation-for-calculating-max-tps/356182)
**Author:** [@tshahab](https://discuss.elastic.co/u/tshahab)\ **Replies:** 2\ **Last updated:** [April 10, 2024, 6:30pm UTC](https://discuss.elastic.co/t/resource-estimation-for-calculating-max-tps/356182 "2024-04-10T18:30:20Z")
Dear Elastic Community, Need help regarding calculating system resource for EFK stack to calculate Max TPS. Current Setup: Environment: Kubernetes Cluster Elasticsearch: Version: 7.17.3 6 Pods/Containers Each Pod h… --- ## [Logstash with Docker ( Unable to connect to database )](https://discuss.elastic.co/t/logstash-with-docker-unable-to-connect-to-database/357072)
**Author:** [@Kaique292](https://discuss.elastic.co/u/Kaique292)\ **Replies:** 12\ **Last updated:** [April 10, 2024, 5:04pm UTC](https://discuss.elastic.co/t/logstash-with-docker-unable-to-connect-to-database/357072 "2024-04-10T17:04:39Z")
I've already changed the driver, I've already checked the connection string for a possible error, but apparently everything is fine. I am trying to remotely connect to a SQL Server database through JDBC via Logstash runn… --- ## [Parsing array of json objects](https://discuss.elastic.co/t/parsing-array-of-json-objects/357138)
**Author:** [@sonukowlwar](https://discuss.elastic.co/u/sonukowlwar)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 4:42pm UTC](https://discuss.elastic.co/t/parsing-array-of-json-objects/357138 "2024-04-10T16:42:40Z")
Hello , I am new to logstash and would like to parse the below json file and add three seperate fields header1 and header2 and header3. Any suggestion on how to do it { "name": "abc", "headers": \[ { "header1": "val… --- ## [Ruby filter to modify nested json field failing](https://discuss.elastic.co/t/ruby-filter-to-modify-nested-json-field-failing/357128)
**Author:** [@elastico12](https://discuss.elastic.co/u/elastico12)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 4:37pm UTC](https://discuss.elastic.co/t/ruby-filter-to-modify-nested-json-field-failing/357128 "2024-04-10T16:37:49Z")
Hi, I have never used ruby before, or logstash. i have been trying to modify nested JSON field using ruby. JSON looks like below. { "a" : { "b" : "c=d; e=f; g=h" } My end goal is to remove e=f from the nested key b. … --- ## [Logstash error "out of range for an integer"](https://discuss.elastic.co/t/logstash-error-out-of-range-for-an-integer/357152)
**Author:** [@Karthick\_D](https://discuss.elastic.co/u/Karthick_D)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 4:03pm UTC](https://discuss.elastic.co/t/logstash-error-out-of-range-for-an-integer/357152 "2024-04-10T16:03:48Z")
Getting this error on logatsh-plain.log \[2024-04-10T13:22:24,797\]\[WARN \]\[logstash.outputs.elasticsearch\]\[geoip\]\[2a0f384086e1f5f8eba88bc1849f4a5149188de0fdacb6e00d827adfb445e001\] Could not index event to Elasticsearch. {… --- ## [Filebeat connection to elasticsearch Error](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-error/356553)
**Author:** [@Vlad2](https://discuss.elastic.co/u/Vlad2)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 4:20pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-error/356553 "2024-04-10T16:20:52Z")
--- ## [Http output in logstash with additional body requirements](https://discuss.elastic.co/t/http-output-in-logstash-with-additional-body-requirements/357098)
**Author:** [@Mike\_Yates](https://discuss.elastic.co/u/Mike_Yates)\ **Replies:** 3\ **Last updated:** [April 10, 2024, 3:41pm UTC](https://discuss.elastic.co/t/http-output-in-logstash-with-additional-body-requirements/357098 "2024-04-10T15:41:06Z")
I have a logstash config file below but the api I'm using requires a body that looks like this, how can i pass that information to the http output configuration. \[ { "class":"stuff", "details":"more stuff" } \] inp… --- ## [Elastic not searching in http.response.body.content field](https://discuss.elastic.co/t/elastic-not-searching-in-http-response-body-content-field/357130)
**Author:** [@CoreCPU](https://discuss.elastic.co/u/CoreCPU)\ **Replies:** 3\ **Last updated:** [April 10, 2024, 3:11pm UTC](https://discuss.elastic.co/t/elastic-not-searching-in-http-response-body-content-field/357130 "2024-04-10T15:11:36Z")
Hello, I have an index which contains the field "http.response.body.content". I updated the ignore\_above values since this field was ignored. However, it would seem that elasticsearch does not search the content of th… --- ## [How to invoke processors with BulkIngester with Java APIs](https://discuss.elastic.co/t/how-to-invoke-processors-with-bulkingester-with-java-apis/356878)
**Author:** [@wnmills3](https://discuss.elastic.co/u/wnmills3)\ **Replies:** 10\ **Last updated:** [April 10, 2024, 2:57pm UTC](https://discuss.elastic.co/t/how-to-invoke-processors-with-bulkingester-with-java-apis/356878 "2024-04-10T14:57:59Z")
I'm trying to use the Java API's to invoke a processor while doing bulk indexing. I've set up the processor on my index to use the intfloat\_\_multilingual-e5-base (see below) to map the value of the passage field to a pas… --- ## [Filebeat Not Shipping Container logs from Kubernetes](https://discuss.elastic.co/t/filebeat-not-shipping-container-logs-from-kubernetes/356891)
**Author:** [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\ **Replies:** 10\ **Last updated:** [April 10, 2024, 2:39pm UTC](https://discuss.elastic.co/t/filebeat-not-shipping-container-logs-from-kubernetes/356891 "2024-04-10T14:39:18Z")
I tried the config settings for autodiscover described here: I confirmed that the pod whose log I want to capture does produce output when I run the command kubectl logs \[podName\] I also confirmed the existence of the… --- ## [MetricBeat Pod, Container and Volume Metricset for Kubernetes Module not working](https://discuss.elastic.co/t/metricbeat-pod-container-and-volume-metricset-for-kubernetes-module-not-working/356903)
**Author:** [@KarwendelMann](https://discuss.elastic.co/u/KarwendelMann)\ **Replies:** 6\ **Last updated:** [April 10, 2024, 2:37pm UTC](https://discuss.elastic.co/t/metricbeat-pod-container-and-volume-metricset-for-kubernetes-module-not-working/356903 "2024-04-10T14:37:20Z")
Hey there! I've been working on setting up Metricbeat within a local Kubernetes cluster to collect metrics from a microservices system, specifically from the onlineboutique application. My Kubernetes cluster is running … --- ## [Create a new event from some fields](https://discuss.elastic.co/t/create-a-new-event-from-some-fields/357082)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 2\ **Last updated:** [April 10, 2024, 2:10pm UTC](https://discuss.elastic.co/t/create-a-new-event-from-some-fields/357082 "2024-04-10T14:10:10Z")
Hi, I have a log record which looks like { @timestamp: some\_timestamp, metadata\_field: metadata\_value, other\_field: other\_value, message: some\_message } Is it possible to make new event from this log record w… --- ## [Is there a way to read from only the index set as write index](https://discuss.elastic.co/t/is-there-a-way-to-read-from-only-the-index-set-as-write-index/357135)
**Author:** [@Maitri](https://discuss.elastic.co/u/Maitri)\ **Replies:** 3\ **Last updated:** [April 10, 2024, 1:59pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-read-from-only-the-index-set-as-write-index/357135 "2024-04-10T13:59:33Z")
Hi, I have created two index with the same alias, Index1 and Index2. Current live data goes into Index1. The reason for creating index2 is I am developing a feature and when it is released I want to remove Index1 and sw… --- ## [Regarding Bulk Indexing Requests](https://discuss.elastic.co/t/regarding-bulk-indexing-requests/357034)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 17\ **Last updated:** [April 10, 2024, 1:55pm UTC](https://discuss.elastic.co/t/regarding-bulk-indexing-requests/357034 "2024-04-10T13:55:42Z")
Hi, Is there a way I can view the size (number of documents) of a bulk index request once it hits the elasticsearch queue? I want to estimate how efficiently logstash is batching requests that it pushes to elasticsearch… --- ## [Logstash - Kafka input handshake fails](https://discuss.elastic.co/t/logstash-kafka-input-handshake-fails/357154)
**Author:** [@thibaut\_a](https://discuss.elastic.co/u/thibaut_a)\ **Replies:** 0\ **Last updated:** [April 10, 2024, 1:55pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-handshake-fails/357154 "2024-04-10T13:55:01Z")
Hi, I try to configure a Kafka input in Logstash, but the handshake fails and I don't have a lot of logs to find a solution. \[2024-04-10T13:40:11,511\]\[ERROR\]\[logstash.inputs.kafka \]\[main\]\[kafka\] Unable to poll Kafka… --- ## [Filter mutate add\_field to create a nested field](https://discuss.elastic.co/t/filter-mutate-add-field-to-create-a-nested-field/357091)
**Author:** [@Armalyca](https://discuss.elastic.co/u/Armalyca)\ **Replies:** 2\ **Last updated:** [April 10, 2024, 12:43pm UTC](https://discuss.elastic.co/t/filter-mutate-add-field-to-create-a-nested-field/357091 "2024-04-10T12:43:50Z")
Hello, I am new to logstash and I have a question about creating nested field with the add\_field filter I use logstash 7.17. I want to create a nested field from a string, but it doesn't work. The other way around (cr… --- ## [Usage CPU during search query](https://discuss.elastic.co/t/usage-cpu-during-search-query/357059)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 12:41pm UTC](https://discuss.elastic.co/t/usage-cpu-during-search-query/357059 "2024-04-10T12:41:05Z")
Hi I'm wondering why during search query, only one are overloading CPU, while we have index divided on 3 shards each on nodes. --- ## [Vector embedding huge size increase](https://discuss.elastic.co/t/vector-embedding-huge-size-increase/356670)
**Author:** [@Chenko](https://discuss.elastic.co/u/Chenko)\ **Replies:** 2\ **Last updated:** [April 10, 2024, 12:40pm UTC](https://discuss.elastic.co/t/vector-embedding-huge-size-increase/356670 "2024-04-10T12:40:57Z")
Hello! I am currently doing a few tests on Vector Search and Vector Embeddings and found that with a subset of 1.000 documents I have an index with the size of 15mb(+-). When I create a new Index for the vector embeddin… --- ## [Stopping Elasticsearch using 'systemctl stop elasticsearch.service' operation consistently fails](https://discuss.elastic.co/t/stopping-elasticsearch-using-systemctl-stop-elasticsearch-service-operation-consistently-fails/357126)
**Author:** [@Mitanshu\_Bhoot](https://discuss.elastic.co/u/Mitanshu_Bhoot)\ **Replies:** 0\ **Last updated:** [April 10, 2024, 10:38am UTC](https://discuss.elastic.co/t/stopping-elasticsearch-using-systemctl-stop-elasticsearch-service-operation-consistently-fails/357126 "2024-04-10T10:38:00Z")
Elasticsearch's stop operation fails when attempting to gracefully stop using the systemctl command. The node contains a substantial amount of data, around 10TBs with 200 shards. ES version is 8.9. Is there any solution … --- ## [On k8s, elasticsearch events return degrated status, but all is green on cluster](https://discuss.elastic.co/t/on-k8s-elasticsearch-events-return-degrated-status-but-all-is-green-on-cluster/356112)
**Author:** [@jeromepp](https://discuss.elastic.co/u/jeromepp)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 10:38am UTC](https://discuss.elastic.co/t/on-k8s-elasticsearch-events-return-degrated-status-but-all-is-green-on-cluster/356112 "2024-04-10T10:38:37Z")
Hello, We use ECK with latest Helm version and Elastisearch 8.12. But this seems to be not relevant to version, there were a similar case without any response : Get events for Elasticsearch cluster health degraded , but… --- ## [ES snapshot getting failed when ES pod get restarted](https://discuss.elastic.co/t/es-snapshot-getting-failed-when-es-pod-get-restarted/357125)
**Author:** [@Nilesh\_Tilani](https://discuss.elastic.co/u/Nilesh_Tilani)\ **Replies:** 0\ **Last updated:** [April 10, 2024, 10:35am UTC](https://discuss.elastic.co/t/es-snapshot-getting-failed-when-es-pod-get-restarted/357125 "2024-04-10T10:35:00Z")
Hello Team, I have enable the snapshot using below keystore feature elasticsearch-keystore add-file gcs.client.default.credentials\_file ./backup.json but when ES pod get restarted keystore config remove above config … --- ## [Setting Rollover Alias for Multiple Indices Under a Single Template](https://discuss.elastic.co/t/setting-rollover-alias-for-multiple-indices-under-a-single-template/355235)
**Author:** [@saisimo01](https://discuss.elastic.co/u/saisimo01)\ **Replies:** 3\ **Last updated:** [April 10, 2024, 9:59am UTC](https://discuss.elastic.co/t/setting-rollover-alias-for-multiple-indices-under-a-single-template/355235 "2024-04-10T09:59:21Z")
Hello Elastic Community, I am currently managing multiple Elastic indices formatted like es-project1-app1, es-project1-app2,... es-project2-app1, es-project2-app2, etc., across various projects. At the moment, it seems … --- ## [Elasticsearch bulk indexing the documents using Java API Client](https://discuss.elastic.co/t/elasticsearch-bulk-indexing-the-documents-using-java-api-client/356995)
**Author:** [@harsha\_kannan](https://discuss.elastic.co/u/harsha_kannan)\ **Replies:** 1\ **Last updated:** [April 10, 2024, 8:51am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-indexing-the-documents-using-java-api-client/356995 "2024-04-10T08:51:02Z")
0 I have inserted some documents into my index using ElasticsearchRepository's saveAll() method and Bulk Indexing method. When I tried fetching records using elasticSearchOperations and I see the fields name has changed… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=264) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=266)