# Elastic Stack

**URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1180

[Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md)

**Page:** 1181

---

## [Query Prefix Numbers and Dates](https://discuss.elastic.co/t/query-prefix-numbers-and-dates/288044)

<div class="topic-metadata">

**Author:** [@elipongr](https://discuss.elastic.co/u/elipongr)\
**Replies:** 1\
**Last updated:** [November 1, 2021, 4:27am UTC](https://discuss.elastic.co/t/query-prefix-numbers-and-dates/288044 "2021-11-01T04:27:09Z")

</div>

So hey i am working with customer where we use flowable which has query models where we can write ES Querys. In our case we need to search for all elemets in array which have the numberValue idk lets say 222. However i…

---

## [Minimize elastic search write operations in storage account](https://discuss.elastic.co/t/minimize-elastic-search-write-operations-in-storage-account/288024)

<div class="topic-metadata">

**Author:** [@pindropviolence](https://discuss.elastic.co/u/pindropviolence)\
**Replies:** 1\
**Last updated:** [November 1, 2021, 4:26am UTC](https://discuss.elastic.co/t/minimize-elastic-search-write-operations-in-storage-account/288024 "2021-11-01T04:26:23Z")

</div>

Hi , I have set up Elastic Stack with beats and i have been able to monitor the logs without any issues. AZ logs getting streamed via event hubs , beats from multiple subscriptions. Have set up curator jobs to delete …

---

## [Getting this exception : Caused by: org.elasticsearch.repositories.RepositoryException: \[index\_id\] Could not determine repository generation from root blobs when trying a non-root user in dockerfile for Elasticsearch image I am using Elasticsearch 7.10.0](https://discuss.elastic.co/t/getting-this-exception-caused-by-org-elasticsearch-repositories-repositoryexception-index-id-could-not-determine-repository-generation-from-root-blobs-when-trying-a-non-root-user-in-dockerfile-for-elasticsearch-image-i-am-using-elasticsearch-7-10-0/288042)

<div class="topic-metadata">

**Author:** [@Sagarika\_Mahalik](https://discuss.elastic.co/u/Sagarika_Mahalik)\
**Replies:** 0\
**Last updated:** [October 29, 2021, 10:30pm UTC](https://discuss.elastic.co/t/getting-this-exception-caused-by-org-elasticsearch-repositories-repositoryexception-index-id-could-not-determine-repository-generation-from-root-blobs-when-trying-a-non-root-user-in-dockerfile-for-elasticsearch-image-i-am-using-elasticsearch-7-10-0/288042 "2021-10-29T22:30:08Z")

</div>

Getting this exception : Caused by: org.Elasticsearch.repositories.RepositoryException: \[index\_id\] Could not determine repository generation from root blobs when trying a non-root user in dockerfile for Elasticsearch ima…

---

## [Troubles with active directory user authentication](https://discuss.elastic.co/t/troubles-with-active-directory-user-authentication/287755)

<div class="topic-metadata">

**Author:** [@auto\_reply\_007](https://discuss.elastic.co/u/auto_reply_007)\
**Replies:** 10\
**Last updated:** [November 1, 2021, 4:21am UTC](https://discuss.elastic.co/t/troubles-with-active-directory-user-authentication/287755 "2021-11-01T04:21:20Z")

</div>

I have trial license Elasticsearch node, and i want to integrate it to use Active directory authentication Here's snippet from my Elasticsearch.yml file xpack: security: enabled: true transport: ssl: …

---

## [Upgrade the cluster from 7.7 to 7.15 version guidance](https://discuss.elastic.co/t/upgrade-the-cluster-from-7-7-to-7-15-version-guidance/287950)

<div class="topic-metadata">

**Author:** [@tahseen\_fatima](https://discuss.elastic.co/u/tahseen_fatima)\
**Replies:** 3\
**Last updated:** [November 1, 2021, 4:19am UTC](https://discuss.elastic.co/t/upgrade-the-cluster-from-7-7-to-7-15-version-guidance/287950 "2021-11-01T04:19:09Z")

</div>

Hi Experts, I have an ELK setup of 7.7 version with 3 nodes of Elasticsearch, 2 kibana and 2 logstash on a windows server 64 bit. A. Below is the existing cluster information:- Configuration of Elasticsearch servers: …

---

## [Handling Large XML Files](https://discuss.elastic.co/t/handling-large-xml-files/288068)

<div class="topic-metadata">

**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Replies:** 1\
**Last updated:** [October 31, 2021, 11:54pm UTC](https://discuss.elastic.co/t/handling-large-xml-files/288068 "2021-10-31T23:54:03Z")

</div>

I have a large XML file that I'm trying to parse. But when I use max\_lines I somehow lose the @timestamp field. Is there a way to use both? input { file { path =\> \[ "C:/temp/TEST/\*.xml" \] start\_position =\> "be…

---

## [Need help sending emails separately with only one watcher running (using forearch)](https://discuss.elastic.co/t/need-help-sending-emails-separately-with-only-one-watcher-running-using-forearch/288026)

<div class="topic-metadata">

**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\
**Replies:** 0\
**Last updated:** [October 29, 2021, 5:13pm UTC](https://discuss.elastic.co/t/need-help-sending-emails-separately-with-only-one-watcher-running-using-forearch/288026 "2021-10-29T17:13:35Z")

</div>

Hello guys need help in the structure of my watcher, I'm trying to create an alert where a notification is sent by email, but the notifications are going in message form (in the same initial email) and when it reaches 10…

---

## [Logstash Pipeline Config API](https://discuss.elastic.co/t/logstash-pipeline-config-api/288087)

<div class="topic-metadata">

**Author:** [@cryonix\_pod](https://discuss.elastic.co/u/cryonix_pod)\
**Replies:** 2\
**Last updated:** [October 31, 2021, 6:00pm UTC](https://discuss.elastic.co/t/logstash-pipeline-config-api/288087 "2021-10-31T18:00:50Z")

</div>

Due to the different flow of traffic I need to change the value of pipeline.workers attribute dynamically. But currently I am using sed and other bash tool to edit the pipeline.yaml file and update the value. Is there a…

---

## [ILM rollover not working on max\_size?](https://discuss.elastic.co/t/ilm-rollover-not-working-on-max-size/287999)

<div class="topic-metadata">

**Author:** [@TimWardFS](https://discuss.elastic.co/u/TimWardFS)\
**Replies:** 3\
**Last updated:** [October 31, 2021, 3:14pm UTC](https://discuss.elastic.co/t/ilm-rollover-not-working-on-max-size/287999 "2021-10-31T15:14:31Z")

</div>

I have an ILM policy with both max\_age and max\_size specified. Rollover happens as expected about ten minutes after max\_age, but does not seem to happen on max\_size. With max\_size set to 10MB I've seen shard sizes up to …

---

## [How can I parse nested JSON strings to JSON objects?](https://discuss.elastic.co/t/how-can-i-parse-nested-json-strings-to-json-objects/287944)

<div class="topic-metadata">

**Author:** [@dudwell](https://discuss.elastic.co/u/dudwell)\
**Replies:** 6\
**Last updated:** [October 31, 2021, 2:50pm UTC](https://discuss.elastic.co/t/how-can-i-parse-nested-json-strings-to-json-objects/287944 "2021-10-31T14:50:37Z")

</div>

Could some possibly provide some guidance on how to parse out an array of JSON strings into individual JSON objects with Logstash json parser??? My logstash filter look as follows... filter { grok { add\_tag =\> \[…

---

## [Metricbeat error for pulling oracle database metrics with an oracle client](https://discuss.elastic.co/t/metricbeat-error-for-pulling-oracle-database-metrics-with-an-oracle-client/288094)

<div class="topic-metadata">

**Author:** [@nsharath](https://discuss.elastic.co/u/nsharath)\
**Replies:** 0\
**Last updated:** [October 31, 2021, 1:46pm UTC](https://discuss.elastic.co/t/metricbeat-error-for-pulling-oracle-database-metrics-with-an-oracle-client/288094 "2021-10-31T13:46:31Z")

</div>

We are using the following software stack:- Server : RHEL6 (It has oracle client installed. I tried with both full client and thin client) beat : metricbeat-7.9.3-x86\_64 database : Oracle 19 (not installed locally. RH…

---

## [Bulk IndexRequest Fails](https://discuss.elastic.co/t/bulk-indexrequest-fails/288074)

<div class="topic-metadata">

**Author:** [@madhurir](https://discuss.elastic.co/u/madhurir)\
**Replies:** 3\
**Last updated:** [October 31, 2021, 7:29am UTC](https://discuss.elastic.co/t/bulk-indexrequest-fails/288074 "2021-10-31T07:29:40Z")

</div>

I am using version 7.15 and trying to create bulk index request in my dev environment. Though the single indexing works, bulk indexing doesn't. I don't see any exception but I see the following in my IntelliJ logs \[ …

---

## [Looking for a well supported jdbc output plugin](https://discuss.elastic.co/t/looking-for-a-well-supported-jdbc-output-plugin/288071)

<div class="topic-metadata">

**Author:** [@David\_Levine](https://discuss.elastic.co/u/David_Levine)\
**Replies:** 3\
**Last updated:** [October 31, 2021, 12:29am UTC](https://discuss.elastic.co/t/looking-for-a-well-supported-jdbc-output-plugin/288071 "2021-10-31T00:29:34Z")

</div>

I'm relatively new to logstash and am hoping some of you might be able to point me in the right direction. I'm looking for a reasonably well supported jdbc output plugin that works against the latest version of logstas…

---

## [Disable dynamic mapping over all nodes](https://discuss.elastic.co/t/disable-dynamic-mapping-over-all-nodes/288062)

<div class="topic-metadata">

**Author:** [@lstoneir](https://discuss.elastic.co/u/lstoneir)\
**Replies:** 1\
**Last updated:** [October 30, 2021, 8:54pm UTC](https://discuss.elastic.co/t/disable-dynamic-mapping-over-all-nodes/288062 "2021-10-30T20:54:49Z")

</div>

I want to disable dynamic mappings for all indices over all nodes of cluster. how can i do this?

---

## [Add date and seconds to odd(?) ISO 8601-ish timestamp?](https://discuss.elastic.co/t/add-date-and-seconds-to-odd-iso-8601-ish-timestamp/288031)

<div class="topic-metadata">

**Author:** [@tjswe](https://discuss.elastic.co/u/tjswe)\
**Replies:** 2\
**Last updated:** [October 30, 2021, 6:47pm UTC](https://discuss.elastic.co/t/add-date-and-seconds-to-odd-iso-8601-ish-timestamp/288031 "2021-10-30T18:47:20Z")

</div>

Im logging AIS-data (ship positions) into Elasticsearch. The "timestamp" field from the receiver is 99% of the time a regular ISO8601 timestamp like "2021-10-29T17:55:30Z" which logstash takes in without issues. But som…

---

## [Logstash - Grok dynamic parsing including nested fields](https://discuss.elastic.co/t/logstash-grok-dynamic-parsing-including-nested-fields/287919)

<div class="topic-metadata">

**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\
**Replies:** 2\
**Last updated:** [October 30, 2021, 11:21am UTC](https://discuss.elastic.co/t/logstash-grok-dynamic-parsing-including-nested-fields/287919 "2021-10-30T11:21:11Z")

</div>

Hello, We have the following pattern which includes dynamic fields nested inside a tag(tag3 in the example below) that needs to be parsed & grok seems to be the go-to-filter to achieve this. However, given fields inside…

---

## [How to match spans near the beginning of a field](https://discuss.elastic.co/t/how-to-match-spans-near-the-beginning-of-a-field/288055)

<div class="topic-metadata">

**Author:** [@Kev86](https://discuss.elastic.co/u/Kev86)\
**Replies:** 0\
**Last updated:** [October 30, 2021, 6:55am UTC](https://discuss.elastic.co/t/how-to-match-spans-near-the-beginning-of-a-field/288055 "2021-10-30T06:55:47Z")

</div>

I have this query and want a span first in SearchField. So that ES shows the string that match closest to the beginning of the field first: { "query": { "bool": { "should": \[ { …

---

## [How to Improve Snapshot to S3 rate?](https://discuss.elastic.co/t/how-to-improve-snapshot-to-s3-rate/288048)

<div class="topic-metadata">

**Author:** [@iShiBin](https://discuss.elastic.co/u/iShiBin)\
**Replies:** 0\
**Last updated:** [October 30, 2021, 3:24am UTC](https://discuss.elastic.co/t/how-to-improve-snapshot-to-s3-rate/288048 "2021-10-30T03:24:11Z")

</div>

The archive rate is only around 60mb/s on a large cluster (72 warm nodes). How to improve the rate to around 100mb/s? What I have tried: Enable the S3-accelerate mode. However, ES doesn't support it per Enabling Tran…

---

## [Snapshot to S3 partially failed with error Unable to execute HTTP request: \<bucket\_name\>.s3.us-west-2.amazonaws.com](https://discuss.elastic.co/t/snapshot-to-s3-partially-failed-with-error-unable-to-execute-http-request-bucket-name-s3-us-west-2-amazonaws-com/287657)

<div class="topic-metadata">

**Author:** [@iShiBin](https://discuss.elastic.co/u/iShiBin)\
**Replies:** 7\
**Last updated:** [October 30, 2021, 3:14am UTC](https://discuss.elastic.co/t/snapshot-to-s3-partially-failed-with-error-unable-to-execute-http-request-bucket-name-s3-us-west-2-amazonaws-com/287657 "2021-10-30T03:14:32Z")

</div>

20 out of 90 shards failed to complete the snapshot with the following archive output. { "snapshot": { "snapshot": "exabeam-2021.08.10", "uuid": "fRjaWFFfQDK7bHbJRR0yHw", "version\_id": 608019…

---

## [Parse XML by Document not Element](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028)

<div class="topic-metadata">

**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Replies:** 5\
**Last updated:** [October 29, 2021, 10:32pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028 "2021-10-29T22:32:21Z")

</div>

As a test, I am trying to ingest the following simple XML file (the actual production file is huge). \<note\> \<to\>Tove\</to\> \<from\>Jani\</from\> \<heading\>Reminder\</heading\> \<body\>Don't forget me this weekend!\</body\> \</note\> …

---

## [Fleet-server installation error](https://discuss.elastic.co/t/fleet-server-installation-error/288033)

<div class="topic-metadata">

**Author:** [@Duane\_Wylie](https://discuss.elastic.co/u/Duane_Wylie)\
**Replies:** 0\
**Last updated:** [October 29, 2021, 7:37pm UTC](https://discuss.elastic.co/t/fleet-server-installation-error/288033 "2021-10-29T19:37:17Z")

</div>

I'm trying to stand up an on-prem Elasticstack deployment (Ubuntu 20.04 and Elastic 7.15). I have Elasticsearch and Kibana setup and working. I'm in the process of setting up the Fleet-server. I'm getting this message…

---

## [Rules folder location](https://discuss.elastic.co/t/rules-folder-location/288010)

<div class="topic-metadata">

**Author:** [@Dalador](https://discuss.elastic.co/u/Dalador)\
**Replies:** 1\
**Last updated:** [October 30, 2021, 12:00am UTC](https://discuss.elastic.co/t/rules-folder-location/288010 "2021-10-30T00:00:41Z")

</div>

Hello I am using Docker ELK and I keep Elasticsearch data permanently by using the following on .yml volumes: /path/to/storage:/usr/share/Elasticsearch/data But how can i save my EQL, KQL rules in security applicati…

---

## [How to save logs, charts, and reports after Kibana is uninstalled](https://discuss.elastic.co/t/how-to-save-logs-charts-and-reports-after-kibana-is-uninstalled/287880)

<div class="topic-metadata">

**Author:** [@li\_li](https://discuss.elastic.co/u/li_li)\
**Replies:** 1\
**Last updated:** [October 29, 2021, 11:46pm UTC](https://discuss.elastic.co/t/how-to-save-logs-charts-and-reports-after-kibana-is-uninstalled/287880 "2021-10-29T23:46:31Z")

</div>

Hello, I have a problem.Kibana Docker uninstall, how to save chart data and report data, the next installation docker can be directly into the display?

---

## [Not able to connect kibana to remote instance](https://discuss.elastic.co/t/not-able-to-connect-kibana-to-remote-instance/287859)

<div class="topic-metadata">

**Author:** [@Tingda\_Du](https://discuss.elastic.co/u/Tingda_Du)\
**Replies:** 8\
**Last updated:** [October 29, 2021, 11:36pm UTC](https://discuss.elastic.co/t/not-able-to-connect-kibana-to-remote-instance/287859 "2021-10-29T23:36:56Z")

</div>

Got this error not sure why. kibana\_1 | {"type":"log","@timestamp":"2021-10-27T22:59:48Z","tags":\["info","plugins-system"\],"pid":8,"message":"Setting up \[2\] plugins: \[inspector,data\]"} kibana\_1 | {"type":"log","@times…

---

## [Timefilter refresh issues on Chromium and edge browsers](https://discuss.elastic.co/t/timefilter-refresh-issues-on-chromium-and-edge-browsers/287933)

<div class="topic-metadata">

**Author:** [@susmeeta](https://discuss.elastic.co/u/susmeeta)\
**Replies:** 1\
**Last updated:** [October 29, 2021, 11:18pm UTC](https://discuss.elastic.co/t/timefilter-refresh-issues-on-chromium-and-edge-browsers/287933 "2021-10-29T23:18:21Z")

</div>

We use Canvas dashboards on Kibana 7.9.1 and use time filters to retrieve data for different time period. We have been using Chrome browsers, but recently found that time filters and browsers are not editable on Chrome v…

---

## [TSVB is not working after ELK up gradation from 7.10 to 7.11.2 with the epoch time field](https://discuss.elastic.co/t/tsvb-is-not-working-after-elk-up-gradation-from-7-10-to-7-11-2-with-the-epoch-time-field/287928)

<div class="topic-metadata">

**Author:** [@anjilinga](https://discuss.elastic.co/u/anjilinga)\
**Replies:** 1\
**Last updated:** [October 29, 2021, 10:28pm UTC](https://discuss.elastic.co/t/tsvb-is-not-working-after-elk-up-gradation-from-7-10-to-7-11-2-with-the-epoch-time-field/287928 "2021-10-29T22:28:48Z")

</div>

Hi, ELK is upgraded from 7.10 to 7.11.2, TSVB is working as expected. It is simple tsvb configuration. the aggrgation is count and group by everthing. The time stamp of the indexpattern is date but it is in epoch time…

---

## [Increase "ignore\_above" value for message.keyword](https://discuss.elastic.co/t/increase-ignore-above-value-for-message-keyword/287962)

<div class="topic-metadata">

**Author:** [@enryoku](https://discuss.elastic.co/u/enryoku)\
**Replies:** 1\
**Last updated:** [October 29, 2021, 10:04pm UTC](https://discuss.elastic.co/t/increase-ignore-above-value-for-message-keyword/287962 "2021-10-29T22:04:21Z")

</div>

Hello, I am trying to increase the default value for "ignore\_above" from 256 to 1024 for the message.keyword field by adding a component index template in Elasticsearch and apply it to the base index pattern. My index …

---

## [Metricbeat & Mongo 5](https://discuss.elastic.co/t/metricbeat-mongo-5/288037)

<div class="topic-metadata">

**Author:** [@Ziemowit](https://discuss.elastic.co/u/Ziemowit)\
**Replies:** 0\
**Last updated:** [October 29, 2021, 9:18pm UTC](https://discuss.elastic.co/t/metricbeat-mongo-5/288037 "2021-10-29T21:18:08Z")

</div>

Hey I have a pretty simple config of Metric Beat metricbeat.modules: - module: mongodb enabled: true hosts: \[ "mongodb://root:secret@mongo" \] I run it in docker (also mongo itself runs in docker). And it work…

---

## [New Kibana reposting job goes in Pending state instead of Processing](https://discuss.elastic.co/t/new-kibana-reposting-job-goes-in-pending-state-instead-of-processing/288036)

<div class="topic-metadata">

**Author:** [@asim\_np](https://discuss.elastic.co/u/asim_np)\
**Replies:** 0\
**Last updated:** [October 29, 2021, 9:14pm UTC](https://discuss.elastic.co/t/new-kibana-reposting-job-goes-in-pending-state-instead-of-processing/288036 "2021-10-29T21:14:32Z")

</div>

Using ELK Stack 7.12.1, After deleting pending jobs from Kibana reposting using the below querry, POST /.reporting\*/\_delete\_by\_query { "query": { "bool": { "must\_not": { "exists":…

---

## [Emulate region map max(field) in Maps](https://discuss.elastic.co/t/emulate-region-map-max-field-in-maps/287863)

<div class="topic-metadata">

**Author:** [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\
**Replies:** 2\
**Last updated:** [October 29, 2021, 7:38pm UTC](https://discuss.elastic.co/t/emulate-region-map-max-field-in-maps/287863 "2021-10-29T19:38:32Z")

</div>

Hello, I want to emulate the region map on a min field value, not the document count, but I can not get it with the New Maps layout because is grouping by document count. How can I get this old functionality? I just wa…

[Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1179)

[Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1181)