# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1025 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1026 --- ## [Getting no error but data not saved in ES](https://discuss.elastic.co/t/getting-no-error-but-data-not-saved-in-es/298905)
**Author:** [@AkAmit](https://discuss.elastic.co/u/AkAmit)\ **Replies:** 0\ **Last updated:** [March 6, 2022, 7:24am UTC](https://discuss.elastic.co/t/getting-no-error-but-data-not-saved-in-es/298905 "2022-03-06T07:24:46Z")
Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. Sending Logstash logs to /var/log/log… --- ## [Logstash 8.1, Protobuf error - add\_file](https://discuss.elastic.co/t/logstash-8-1-protobuf-error-add-file/298894)
**Author:** [@pdoten](https://discuss.elastic.co/u/pdoten)\ **Replies:** 2\ **Last updated:** [March 6, 2022, 4:06am UTC](https://discuss.elastic.co/t/logstash-8-1-protobuf-error-add-file/298894 "2022-03-06T04:06:57Z")
Long time user here, love Elasticstack but cant get over this problem. Trying to create input for protobuf. Input stanza below. LS stops after throwing java exception, cant get pass this -tried to get this to work in 7.… --- ## [Field Available in Discovery, but Not in Visualization](https://discuss.elastic.co/t/field-available-in-discovery-but-not-in-visualization/298903)
**Author:** [@cyberphor](https://discuss.elastic.co/u/cyberphor)\ **Replies:** 0\ **Last updated:** [March 6, 2022, 3:56am UTC](https://discuss.elastic.co/t/field-available-in-discovery-but-not-in-visualization/298903 "2022-03-06T03:56:39Z")
Hello, I'm running Elasticsearch, Kibana, and Winlogbeat (all of which are version 8.0.1) on a Windows 10 desktop. When I query for event.code:4104 using the "Discovery" tab, one of the available fields is powershell.f… --- ## [Rejecting mapping update to \[xxx\] as the final mapping would have more than 1 type: \[\_doc, xxx\]](https://discuss.elastic.co/t/rejecting-mapping-update-to-xxx-as-the-final-mapping-would-have-more-than-1-type-doc-xxx/298839)
**Author:** [@Ramesh\_balasubramani](https://discuss.elastic.co/u/Ramesh_balasubramani)\ **Replies:** 11\ **Last updated:** [March 5, 2022, 6:57pm UTC](https://discuss.elastic.co/t/rejecting-mapping-update-to-xxx-as-the-final-mapping-would-have-more-than-1-type-doc-xxx/298839 "2022-03-05T18:57:31Z")
I am using Elasticsearch 7.10.2 version. I need to create one index, Index name = device \_type name = device But I am not able to create getting error. POST device/device/169062 { "device\_name": "DESK… --- ## [Get documents starting at a high from](https://discuss.elastic.co/t/get-documents-starting-at-a-high-from/298714)
**Author:** [@NominaSumpta](https://discuss.elastic.co/u/NominaSumpta)\ **Replies:** 9\ **Last updated:** [March 5, 2022, 6:21pm UTC](https://discuss.elastic.co/t/get-documents-starting-at-a-high-from/298714 "2022-03-05T18:21:26Z")
Hi, I have an with ~ 1.500.000 (million) documents or so. I only want to get 1.000 results from it, but I want to start counting backwards. So, I want to retrieve documents 1.499.000 (- 1.000) through 1.500.000. I've se… --- ## [Multi\_Match Vs Query\_String\_Query?](https://discuss.elastic.co/t/multi-match-vs-query-string-query/298892)
**Author:** [@Nishikant\_Tayade](https://discuss.elastic.co/u/Nishikant_Tayade)\ **Replies:** 0\ **Last updated:** [March 5, 2022, 1:46pm UTC](https://discuss.elastic.co/t/multi-match-vs-query-string-query/298892 "2022-03-05T13:46:49Z")
I am using ES 7.10.2 I have fields with keyword and text mappings. When user specifies the search string, I wanted to parse the pieces of search string, for each specific field, in the specific way that field mapping i… --- ## [How to check if host.ip in particular CIDR](https://discuss.elastic.co/t/how-to-check-if-host-ip-in-particular-cidr/298648)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 1\ **Last updated:** [March 4, 2022, 2:27pm UTC](https://discuss.elastic.co/t/how-to-check-if-host-ip-in-particular-cidr/298648 "2022-03-04T14:27:46Z")
Hello, I want to add an ingest pipeline script that will add particular fields if host.ip falls into the CIDR notation range. I was trying some stuff but still, that does not work The script does not return errors but… --- ## [Elastic search upgrade](https://discuss.elastic.co/t/elastic-search-upgrade/298827)
**Author:** [@aman123](https://discuss.elastic.co/u/aman123)\ **Replies:** 4\ **Last updated:** [March 5, 2022, 8:27am UTC](https://discuss.elastic.co/t/elastic-search-upgrade/298827 "2022-03-05T08:27:14Z")
Hi All, I'm upgrading the Elasticsearch cluster, currently I have 10 node clusters running on individual instances , All nodes are having older version, if I upgrade one by one all nodes starting from data node to maste… --- ## [Runtime\_mapping for aggregation context?](https://discuss.elastic.co/t/runtime-mapping-for-aggregation-context/298886)
**Author:** [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\ **Replies:** 0\ **Last updated:** [March 5, 2022, 5:29am UTC](https://discuss.elastic.co/t/runtime-mapping-for-aggregation-context/298886 "2022-03-05T05:29:12Z")
Hello ! I have a requirement which I would need calculate the total amount spiked compared to previous of latest month. I was able to achieve this by using watcher . However, this would be dependent on watcher so I … --- ## [White Papers on Using Elastic for Unstructured Data Queries on ZFS Filesystem](https://discuss.elastic.co/t/white-papers-on-using-elastic-for-unstructured-data-queries-on-zfs-filesystem/298877)
**Author:** [@minorsatellite](https://discuss.elastic.co/u/minorsatellite)\ **Replies:** 4\ **Last updated:** [March 5, 2022, 4:36am UTC](https://discuss.elastic.co/t/white-papers-on-using-elastic-for-unstructured-data-queries-on-zfs-filesystem/298877 "2022-03-05T04:36:32Z")
Im new to Elastic. Im wondering if anyone has successfully used Elastic alongside a ZFS Dataset in order to perform filesystem searches, and if so, what might that look like, and would it a user-friendly solution for non… --- ## [Reattaching Cold Data](https://discuss.elastic.co/t/reattaching-cold-data/298785)
**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\ **Replies:** 7\ **Last updated:** [March 5, 2022, 4:19am UTC](https://discuss.elastic.co/t/reattaching-cold-data/298785 "2022-03-05T04:19:29Z")
If you were to have cold data on removable hard disks (for long-term offline storage), what is the best way to re-attach one of these disks to the cold data node without causing trouble? For example, I was wondering if t… --- ## [\[logstash.outputs.elasticsearch\]\[main\] Failed to install template](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-main-failed-to-install-template/298835)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 1\ **Last updated:** [March 4, 2022, 10:36pm UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-main-failed-to-install-template/298835 "2022-03-04T22:36:05Z")
Hi everyone, After enabling SSL communication on my Elasticsearch server (v7.14.0), i noticed an error in my logstash logs: \[ERROR\]\[logstash.outputs.elasticsearch\]\[main\] Failed to install template {:message=\>"Got respo… --- ## [Logstash Date parsing error](https://discuss.elastic.co/t/logstash-date-parsing-error/298780)
**Author:** [@userR](https://discuss.elastic.co/u/userR)\ **Replies:** 11\ **Last updated:** [March 4, 2022, 10:11pm UTC](https://discuss.elastic.co/t/logstash-date-parsing-error/298780 "2022-03-04T22:11:27Z")
Hi! I am trying to parse the following date format: 2022-03-03 10:45:02,520 My current configuration for logstash to parse is: grok { match =\> \[ "message", "%{TIMESTAMP\_ISO8601:logdate}....." \] } d… --- ## [Docker - Elasticsearch 8 - unable to create temporary keystore](https://discuss.elastic.co/t/docker-elasticsearch-8-unable-to-create-temporary-keystore/298878)
**Author:** [@Realsnack](https://discuss.elastic.co/u/Realsnack)\ **Replies:** 0\ **Last updated:** [March 4, 2022, 9:50pm UTC](https://discuss.elastic.co/t/docker-elasticsearch-8-unable-to-create-temporary-keystore/298878 "2022-03-04T21:50:00Z")
I am upgrading to Elasticsearch 8 on my home docker cluster and I am getting the error: ERROR: unable to create temporary keystore at \[/usr/share/elasticsearch/config/elasticsearch.keystore.tmp\], write permissions requi… --- ## [ES6.8 vs monitoring retention](https://discuss.elastic.co/t/es6-8-vs-monitoring-retention/298283)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 7:02pm UTC](https://discuss.elastic.co/t/es6-8-vs-monitoring-retention/298283 "2022-03-04T19:02:40Z")
Reading Logging indices retention it seems that ES6.8 might no adhere to xpack.monitoring.history.duration default value of 168h ~7days. And without any default ilm policy I created my own: { "delete\_after\_7days" : {… --- ## [Multi-field mapping/search "highlighting" difference between 7.10.2 and 7.16.3?](https://discuss.elastic.co/t/multi-field-mapping-search-highlighting-difference-between-7-10-2-and-7-16-3/298739)
**Author:** [@mrodent](https://discuss.elastic.co/u/mrodent)\ **Replies:** 1\ **Last updated:** [March 4, 2022, 6:28pm UTC](https://discuss.elastic.co/t/multi-field-mapping-search-highlighting-difference-between-7-10-2-and-7-16-3/298739 "2022-03-04T18:28:15Z")
The following mapping and query DSL dict work OK when the version of ES is 7.10.2, but they never work where the version of ES is 7.16.3: mappings = \\ { "mappings": { "properties": … --- ## [Filebeat checkpoint module not getting data from syslog](https://discuss.elastic.co/t/filebeat-checkpoint-module-not-getting-data-from-syslog/298864)
**Author:** [@mfloris](https://discuss.elastic.co/u/mfloris)\ **Replies:** 1\ **Last updated:** [March 4, 2022, 5:36pm UTC](https://discuss.elastic.co/t/filebeat-checkpoint-module-not-getting-data-from-syslog/298864 "2022-03-04T17:36:36Z")
I'm trying send CheckPoint Firewall logs to Elasticsearch 8.0. I have machine A 192.168.1.123 running Rsyslog receiving logs on port 514 that logs to a file and machine B 192.168.1.234 running Elasticsearch and Kibana. … --- ## [Howto modify num\_of\_shards with docker-compose](https://discuss.elastic.co/t/howto-modify-num-of-shards-with-docker-compose/298868)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 0\ **Last updated:** [March 4, 2022, 5:23pm UTC](https://discuss.elastic.co/t/howto-modify-num-of-shards-with-docker-compose/298868 "2022-03-04T17:23:03Z")
Hey there, is there a way to modify the number\_of\_shards via docker-compose or to provide an index template? --- ## [Design practice to prevent oversharding](https://discuss.elastic.co/t/design-practice-to-prevent-oversharding/298759)
**Author:** [@cyril.r](https://discuss.elastic.co/u/cyril.r)\ **Replies:** 8\ **Last updated:** [March 4, 2022, 4:57pm UTC](https://discuss.elastic.co/t/design-practice-to-prevent-oversharding/298759 "2022-03-04T16:57:03Z")
Hi there, I'm strugling with an application design and the impact that it has on the cluster. Let me explain the use case: My customers can create databases inside my application to store and retrieve data. Every tim… --- ## [6.8 to 7.x - This index has date fields with deprecated formats](https://discuss.elastic.co/t/6-8-to-7-x-this-index-has-date-fields-with-deprecated-formats/298865)
**Author:** [@MatthiasLenhardt](https://discuss.elastic.co/u/MatthiasLenhardt)\ **Replies:** 0\ **Last updated:** [March 4, 2022, 4:42pm UTC](https://discuss.elastic.co/t/6-8-to-7-x-this-index-has-date-fields-with-deprecated-formats/298865 "2022-03-04T16:42:03Z")
Hi, we are planing to upgrade Elasticsearch from 6.8 to 7.x. Kibana " 7.0 Upgrade Assistant" tells us for some indices "Date field format uses patterns which may change meaning in 7.0", e.g.: This index has date field… --- ## [Reading structured streaming data from Elasticsearch into Spark using Python](https://discuss.elastic.co/t/reading-structured-streaming-data-from-elasticsearch-into-spark-using-python/298815)
**Author:** [@saurab](https://discuss.elastic.co/u/saurab)\ **Replies:** 2\ **Last updated:** [March 4, 2022, 4:34pm UTC](https://discuss.elastic.co/t/reading-structured-streaming-data-from-elasticsearch-into-spark-using-python/298815 "2022-03-04T16:34:13Z")
I believe Elasticsearch supports Spark Structured Streaming since version 6. I have seen examples of WRITING structured streaming data from spark to Elasticsearch in append mode. But I have not seen a single example of r… --- ## [Which aggregation should be used for transform operations on multifield?](https://discuss.elastic.co/t/which-aggregation-should-be-used-for-transform-operations-on-multifield/298409)
**Author:** [@Scylanth](https://discuss.elastic.co/u/Scylanth)\ **Replies:** 2\ **Last updated:** [March 4, 2022, 4:33pm UTC](https://discuss.elastic.co/t/which-aggregation-should-be-used-for-transform-operations-on-multifield/298409 "2022-03-04T16:33:24Z")
Hello, I am using 7.16.3 Elasticsearch and I am trying to perform a transform aggregation on an index to create a summary index (basically have the same index but with an aggregated timestamp by day to be smaller). Amo… --- ## [Aggregation on Multisearch responses](https://discuss.elastic.co/t/aggregation-on-multisearch-responses/298862)
**Author:** [@hbot](https://discuss.elastic.co/u/hbot)\ **Replies:** 0\ **Last updated:** [March 4, 2022, 4:29pm UTC](https://discuss.elastic.co/t/aggregation-on-multisearch-responses/298862 "2022-03-04T16:29:02Z")
Hi, Is it possible to run an aggregation on multiple response sets returned by the multisearch API? As an example, say I execute 10 different queries through the multisearch API, and each query returns 10 documents with… --- ## [Error connecting to package registry - No Internet connection](https://discuss.elastic.co/t/error-connecting-to-package-registry-no-internet-connection/298843)
**Author:** [@Bavaria](https://discuss.elastic.co/u/Bavaria)\ **Replies:** 2\ **Last updated:** [March 4, 2022, 4:05pm UTC](https://discuss.elastic.co/t/error-connecting-to-package-registry-no-internet-connection/298843 "2022-03-04T16:05:03Z")
Hey, I searched very long, is there a chance to prevent those warning for not able to connecting to package registry or if its clicked in Kibana I am getting errors in my cmd. The Server does not have internet access, s… --- ## [Event aggregation with logstash error due to mixed task\_id](https://discuss.elastic.co/t/event-aggregation-with-logstash-error-due-to-mixed-task-id/298705)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 4:02pm UTC](https://discuss.elastic.co/t/event-aggregation-with-logstash-error-due-to-mixed-task-id/298705 "2022-03-04T16:02:53Z")
Hi all I have a case like this: my email log has a task\_id i called it event.id due to how the log was configed now all the log are seperated and only joined by the event.id field. Now i've tried to use the aggregate … --- ## [Logstash 8.0 is adding extra field](https://discuss.elastic.co/t/logstash-8-0-is-adding-extra-field/297594)
**Author:** [@kannan\_raj](https://discuss.elastic.co/u/kannan_raj)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 3:50pm UTC](https://discuss.elastic.co/t/logstash-8-0-is-adding-extra-field/297594 "2022-03-04T15:50:19Z")
logstash consuming JSON messages from Kafka and indexing them into Elasticsearch. we found that logstash version 8.0 is adding the additional field event with the value of the actual message. could you please let me kno… --- ## [Metricbeat : Error fetching data for metricset elasticsearch](https://discuss.elastic.co/t/metricbeat-error-fetching-data-for-metricset-elasticsearch/298792)
**Author:** [@rp346](https://discuss.elastic.co/u/rp346)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 3:49pm UTC](https://discuss.elastic.co/t/metricbeat-error-fetching-data-for-metricset-elasticsearch/298792 "2022-03-04T15:49:08Z")
I have setup metricbeat in kubernetes with following deployment manifest DaemonSet--- apiVersion: v1 kind: ServiceAccount metadata: name: metricbeat namespace: kube-logging labels: app: metricbeat --- apiVe… --- ## [Read from spark big index and memory usage](https://discuss.elastic.co/t/read-from-spark-big-index-and-memory-usage/298707)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 5\ **Last updated:** [March 4, 2022, 3:37pm UTC](https://discuss.elastic.co/t/read-from-spark-big-index-and-memory-usage/298707 "2022-03-04T15:37:44Z")
We use Spark to read a full index of 80Go then write data as a parquet file. I see memory consumption is going up until 24G per Spark worker. Hence my question: during the scroll search (1 hour), is the data stored in … --- ## [Incorrect indice creation date](https://discuss.elastic.co/t/incorrect-indice-creation-date/298539)
**Author:** [@Helicube](https://discuss.elastic.co/u/Helicube)\ **Replies:** 6\ **Last updated:** [March 4, 2022, 3:15pm UTC](https://discuss.elastic.co/t/incorrect-indice-creation-date/298539 "2022-03-04T15:15:42Z")
Hello, I have an issue with the Index Rotation. My Graylog server automatically create a new Indice every day (Index rotation). But sometimes, the creation date entered in Elasticsearch is not correct. The date entered … --- ## [Set custom Transform schedule time in kibana](https://discuss.elastic.co/t/set-custom-transform-schedule-time-in-kibana/298794)
**Author:** [@dash5000](https://discuss.elastic.co/u/dash5000)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 3:01pm UTC](https://discuss.elastic.co/t/set-custom-transform-schedule-time-in-kibana/298794 "2022-03-04T15:01:12Z")
Hello Everyone, I just started using transform in kibana UI to aggregate data and also to have a longer retention period for visualization. The data collections work fine, my transform basically collectors two fields w… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1024) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1026)