# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1082 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1083 --- ## [Elastic Search ILM implementation - Upsert issue](https://discuss.elastic.co/t/elastic-search-ilm-implementation-upsert-issue/295192)
**Author:** [@manikantavarma20](https://discuss.elastic.co/u/manikantavarma20)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 2:23pm UTC](https://discuss.elastic.co/t/elastic-search-ilm-implementation-upsert-issue/295192 "2022-01-24T14:23:31Z")
We have 2 indices - "event" and "event-001" "event-001" is created by using ILM Both the indices have the same alias "event\_" It is used to fetch and update the data in the above 2 indices Query: GET event\_/alias R… --- ## [Filebeat Elastic Kibana - Json based Log Processing](https://discuss.elastic.co/t/filebeat-elastic-kibana-json-based-log-processing/295224)
**Author:** [@RashidKhan](https://discuss.elastic.co/u/RashidKhan)\ **Replies:** 4\ **Last updated:** [January 24, 2022, 1:25pm UTC](https://discuss.elastic.co/t/filebeat-elastic-kibana-json-based-log-processing/295224 "2022-01-24T13:25:51Z")
Hi Team, I have a log file that has mixed structure, some are multiple line json, some are single line json and some are plain text. My goal is to Parse Json and send to elastic, and ignore the plain text. I read thro… --- ## [Auditbeat 7.16.2 crashes servers due to memory issues](https://discuss.elastic.co/t/auditbeat-7-16-2-crashes-servers-due-to-memory-issues/295047)
**Author:** [@JensVanDeynse](https://discuss.elastic.co/u/JensVanDeynse)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 2:06pm UTC](https://discuss.elastic.co/t/auditbeat-7-16-2-crashes-servers-due-to-memory-issues/295047 "2022-01-24T14:06:34Z")
Hello ELK version: 7.16.2 Auditbeat version: 7.16.2 I'm trying to set up an Elastic SIEM cluster which is as of now running the way it should but we are still experiencing a memory issue with auditbeat. It seems that… --- ## [Install filter-plugin failed](https://discuss.elastic.co/t/install-filter-plugin-failed/295237)
**Author:** [@dominbdg](https://discuss.elastic.co/u/dominbdg)\ **Replies:** 0\ **Last updated:** [January 24, 2022, 2:04pm UTC](https://discuss.elastic.co/t/install-filter-plugin-failed/295237 "2022-01-24T14:04:16Z")
Hello, Today I upgraded logstash to the latest version and needs to install logstash-filter-elapsed plugin. I'm receiving below error: ./logstash-plugin install logstash-filter-elapsed Using bundled JDK: /usr/share/lo… --- ## [Control phase transition timings in ILM](https://discuss.elastic.co/t/control-phase-transition-timings-in-ilm/294904)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 1:48pm UTC](https://discuss.elastic.co/t/control-phase-transition-timings-in-ilm/294904 "2022-01-24T13:48:45Z")
I'm trying to implement an ilm policy where the index will rollover every month or if the size reached 50GB. And since by default, ilm based the age in index creation date, and creating the index mid december means, we … --- ## [Replace Log4j from 2.x to 2.17.0 or later](https://discuss.elastic.co/t/replace-log4j-from-2-x-to-2-17-0-or-later/292897)
**Author:** [@marisubu](https://discuss.elastic.co/u/marisubu)\ **Replies:** 4\ **Last updated:** [January 24, 2022, 1:16pm UTC](https://discuss.elastic.co/t/replace-log4j-from-2-x-to-2-17-0-or-later/292897 "2022-01-24T13:16:53Z")
Our oldest products includes Elastic stack versions 5.2, 6.2.2 and 7.10.2. We're in situation that we can not perform elastic versions uplift to the latest/fixed versions. Can we replace Log4j 2.x versions with 2.17.x o… --- ## [Filebeat: Kafka Input doesn't push the topic offset](https://discuss.elastic.co/t/filebeat-kafka-input-doesnt-push-the-topic-offset/292436)
**Author:** [@Giero](https://discuss.elastic.co/u/Giero)\ **Replies:** 6\ **Last updated:** [January 24, 2022, 12:54pm UTC](https://discuss.elastic.co/t/filebeat-kafka-input-doesnt-push-the-topic-offset/292436 "2022-01-24T12:54:07Z")
I'm using Filebeat 7.16.0 with Kafka Input and Elasticsearch Output. Previously I had java application to take care of this flow. I'm using the same consumer group as previous java application and messages appear in El… --- ## [Multiple Index](https://discuss.elastic.co/t/multiple-index/294012)
**Author:** [@graimato](https://discuss.elastic.co/u/graimato)\ **Replies:** 2\ **Last updated:** [January 24, 2022, 12:43pm UTC](https://discuss.elastic.co/t/multiple-index/294012 "2022-01-24T12:43:13Z")
Dear all I have a problem with filebeat configuration. I enable two different module and I need to write log in two different index , myindex-module1-data myindex-module2-data Is it possible? I read about filebeat.p… --- ## [CA generated without key using instances.yml](https://discuss.elastic.co/t/ca-generated-without-key-using-instances-yml/295221)
**Author:** [@Mamol27](https://discuss.elastic.co/u/Mamol27)\ **Replies:** 0\ **Last updated:** [January 24, 2022, 11:39am UTC](https://discuss.elastic.co/t/ca-generated-without-key-using-instances-yml/295221 "2022-01-24T11:39:34Z")
Hi! I tried to generate certificates like this How to Secure Elasticsearch Clusters - Completely and Effectively, but in folder ca I have got only ca.crt without key. It could be a problem if I need to add some certifica… --- ## [Update ELK 7.14 at 7.16](https://discuss.elastic.co/t/update-elk-7-14-at-7-16/295195)
**Author:** [@marc0069urca](https://discuss.elastic.co/u/marc0069urca)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 10:16am UTC](https://discuss.elastic.co/t/update-elk-7-14-at-7-16/295195 "2022-01-24T10:16:34Z")
Hello, I wanted to know step by step in the order of the Elasticsearch machine to Kibana the implementation of a complete upgrade of an ELK , can you tell me the procedure to upgrade from our tool 7.14 to the current ve… --- ## [Export Elasticsearch Index to local](https://discuss.elastic.co/t/export-elasticsearch-index-to-local/295193)
**Author:** [@AsakaRin](https://discuss.elastic.co/u/AsakaRin)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 9:54am UTC](https://discuss.elastic.co/t/export-elasticsearch-index-to-local/295193 "2022-01-24T09:54:36Z")
I want to make a clone database Elasticsearch from server to my local. I already have permission to access API and create snapshots but I can't access the server direct and download snapshots to my local. Is there any wa… --- ## [Custom Logs Ingest Pipeline](https://discuss.elastic.co/t/custom-logs-ingest-pipeline/294409)
**Author:** [@jsteenkamp](https://discuss.elastic.co/u/jsteenkamp)\ **Replies:** 8\ **Last updated:** [January 24, 2022, 9:34am UTC](https://discuss.elastic.co/t/custom-logs-ingest-pipeline/294409 "2022-01-24T09:34:47Z")
@ruflin This is more or less a duplicate of Fleet Custom Logs Ingest Pipeline, but I still find it hard to figure out which steps I have to follow. I would really appreciate better documentation or some useful direction… --- ## [LogStash Special Character Split Error](https://discuss.elastic.co/t/logstash-special-character-split-error/295137)
**Author:** [@eagles40cnuh](https://discuss.elastic.co/u/eagles40cnuh)\ **Replies:** 6\ **Last updated:** [January 24, 2022, 9:10am UTC](https://discuss.elastic.co/t/logstash-special-character-split-error/295137 "2022-01-24T09:10:40Z")
Hi! I have some problem with Logstash Filter Filebeat collect -\> "aaa.log" file and then message field -\> "ID∮NAME∮TELNO" And Logstash.conf is input { beats { port =\> xxxx } filter { mutate { split =\> { "message"… --- ## [Right order to stop ELK](https://discuss.elastic.co/t/right-order-to-stop-elk/295199)
**Author:** [@Ram\_N](https://discuss.elastic.co/u/Ram_N)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 9:01am UTC](https://discuss.elastic.co/t/right-order-to-stop-elk/295199 "2022-01-24T09:01:19Z")
Hi What is the recommended to stop Elasticsearch, logstash, kibana? Which should be stopped first? At times if Elasticsearch is stopped first, logstash is trying to establish connection and logstash is not getting stopp… --- ## [ElasticSearch won't search specific field](https://discuss.elastic.co/t/elasticsearch-wont-search-specific-field/295201)
**Author:** [@bert\_coenen](https://discuss.elastic.co/u/bert_coenen)\ **Replies:** 0\ **Last updated:** [January 24, 2022, 8:59am UTC](https://discuss.elastic.co/t/elasticsearch-wont-search-specific-field/295201 "2022-01-24T08:59:30Z")
I have a problem searching a specific field inside my index. little background: On my project we need to search inside a terminology Server like FHIR but then our own. So we have an object that contains a Code (123564/… --- ## [Number of shards and deployment sizing](https://discuss.elastic.co/t/number-of-shards-and-deployment-sizing/294618)
**Author:** [@yelloyonder](https://discuss.elastic.co/u/yelloyonder)\ **Replies:** 4\ **Last updated:** [January 24, 2022, 8:57am UTC](https://discuss.elastic.co/t/number-of-shards-and-deployment-sizing/294618 "2022-01-24T08:57:26Z")
We have created a small deployment on Elastic Cloud which we are testing with non-live data. Reading this blog post How many shards should I have in my Elasticsearch cluster? | Elastic Blog it suggests approximately 20 … --- ## [Custom time frame for charts](https://discuss.elastic.co/t/custom-time-frame-for-charts/294194)
**Author:** [@DanielCC](https://discuss.elastic.co/u/DanielCC)\ **Replies:** 3\ **Last updated:** [January 24, 2022, 8:42am UTC](https://discuss.elastic.co/t/custom-time-frame-for-charts/294194 "2022-01-24T08:42:58Z")
Hello there, Which version of Kibana allows me to have this option for my panels? --- ## [Getting 502 bad gateway error while creating new index](https://discuss.elastic.co/t/getting-502-bad-gateway-error-while-creating-new-index/295183)
**Author:** [@Nikhil\_Sonkar](https://discuss.elastic.co/u/Nikhil_Sonkar)\ **Replies:** 3\ **Last updated:** [January 24, 2022, 8:23am UTC](https://discuss.elastic.co/t/getting-502-bad-gateway-error-while-creating-new-index/295183 "2022-01-24T08:23:54Z")
502 Bad Gateway 502 Bad Gateway --- ## [How to zoom in the Y-Axis in a line chart using kibana 7.16?](https://discuss.elastic.co/t/how-to-zoom-in-the-y-axis-in-a-line-chart-using-kibana-7-16/295162)
**Author:** [@GustavoEH](https://discuss.elastic.co/u/GustavoEH)\ **Replies:** 2\ **Last updated:** [January 24, 2022, 7:53am UTC](https://discuss.elastic.co/t/how-to-zoom-in-the-y-axis-in-a-line-chart-using-kibana-7-16/295162 "2022-01-24T07:53:47Z")
Hello everyone, I'm new in Elasticsearch/Kibana and this is my first post, maybe the question is simple but I already expend hours trying to solve this issue, also looking for something similar in many forums but nothin… --- ## [How is Kibana accessing Large amounts of data from Elasticsearch in few seconds?](https://discuss.elastic.co/t/how-is-kibana-accessing-large-amounts-of-data-from-elasticsearch-in-few-seconds/295039)
**Author:** [@Harish\_fragger](https://discuss.elastic.co/u/Harish_fragger)\ **Replies:** 3\ **Last updated:** [January 24, 2022, 7:20am UTC](https://discuss.elastic.co/t/how-is-kibana-accessing-large-amounts-of-data-from-elasticsearch-in-few-seconds/295039 "2022-01-24T07:20:52Z")
I am trying to understand how is kibana fetching 129,306,319 hits from Elasticsearch and displaying it in Discover -\> Inspect -\> Responses? I can see from the request query that they are retrieving only 500 records at a… --- ## [Users password expiration or blocked accounts](https://discuss.elastic.co/t/users-password-expiration-or-blocked-accounts/295086)
**Author:** [@d.silwon](https://discuss.elastic.co/u/d.silwon)\ **Replies:** 6\ **Last updated:** [January 24, 2022, 6:35am UTC](https://discuss.elastic.co/t/users-password-expiration-or-blocked-accounts/295086 "2022-01-24T06:35:23Z")
Dears, In our Elastic cluster we have few local users which login to Kibana for browsing indexes with applications logs. Twice a week two users reports me problems with login to Kibana. After password reset these users… --- ## [How to migrate all dashboard, visualization, map, template, kibana settings data?](https://discuss.elastic.co/t/how-to-migrate-all-dashboard-visualization-map-template-kibana-settings-data/295173)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 7\ **Last updated:** [January 24, 2022, 6:31am UTC](https://discuss.elastic.co/t/how-to-migrate-all-dashboard-visualization-map-template-kibana-settings-data/295173 "2022-01-24T06:31:28Z")
I want to completely migrate the data of the 7.12 cluster to the 7.16 cluster, including all the data, which can be understood as mirroring. I suspect that the data on kibana is all on the kibana index. If it is copied … --- ## [Why do i see multiple nodes in the data folder](https://discuss.elastic.co/t/why-do-i-see-multiple-nodes-in-the-data-folder/295040)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 4\ **Last updated:** [January 24, 2022, 5:49am UTC](https://discuss.elastic.co/t/why-do-i-see-multiple-nodes-in-the-data-folder/295040 "2022-01-24T05:49:49Z")
Hello , I have upgraded Elasticsearch last month twice to fix log4j vulnerability . In EMEA i see only one folder and used disk space is 400GB. But in NAFTA i see 3 folders under node and overall it is consuming 750GB w… --- ## [Fingerprint option in logstash filter not working properly?](https://discuss.elastic.co/t/fingerprint-option-in-logstash-filter-not-working-properly/295115)
**Author:** [@Anjali\_Kushwaha](https://discuss.elastic.co/u/Anjali_Kushwaha)\ **Replies:** 9\ **Last updated:** [January 24, 2022, 5:39am UTC](https://discuss.elastic.co/t/fingerprint-option-in-logstash-filter-not-working-properly/295115 "2022-01-24T05:39:37Z")
I have used below code but its only inserting one record to ES . filter { fingerprint { target =\> "document\_id" method =\> "SHA256" key =\> "9ced3827c6a1c9dafac6da9abac41386ba1038ac95b3a865a0951bc2e948c58c" … --- ## [Change index name for threatintel module, good idea?](https://discuss.elastic.co/t/change-index-name-for-threatintel-module-good-idea/295158)
**Author:** [@xentoo](https://discuss.elastic.co/u/xentoo)\ **Replies:** 1\ **Last updated:** [January 24, 2022, 3:12am UTC](https://discuss.elastic.co/t/change-index-name-for-threatintel-module-good-idea/295158 "2022-01-24T03:12:09Z")
Hello, I am trying to change the name of the index for threatintel module from "filebeat" to "threatintel". It does not make much sense to me to bury these documents amongst millions of logs. Or does it? Am I missing s… --- ## [Elasticsearch top\_hits aggregation result and doc\_count are different](https://discuss.elastic.co/t/elasticsearch-top-hits-aggregation-result-and-doc-count-are-different/293400)
**Author:** [@hyun](https://discuss.elastic.co/u/hyun)\ **Replies:** 2\ **Last updated:** [January 23, 2022, 11:33pm UTC](https://discuss.elastic.co/t/elasticsearch-top-hits-aggregation-result-and-doc-count-are-different/293400 "2022-01-23T23:33:56Z")
Query GET /someindex/\_search { "size": 0, "query": { "ids": { "types": \[\], "values": \["08a2","08a3","03a2","03a3","84a1"\] } }, "aggregations": { "498": { "terms":… --- ## [Unmapped field in document not appearing in filter drop down](https://discuss.elastic.co/t/unmapped-field-in-document-not-appearing-in-filter-drop-down/295128)
**Author:** [@Tech\_solution](https://discuss.elastic.co/u/Tech_solution)\ **Replies:** 4\ **Last updated:** [January 23, 2022, 11:08pm UTC](https://discuss.elastic.co/t/unmapped-field-in-document-not-appearing-in-filter-drop-down/295128 "2022-01-23T23:08:58Z")
Hi, I am new to Kibana dashboard . The issue I came across is that I want to create a dashboard for an unmapped field while that field is not mapped in index ( it doesn't appear in drop down for filters or in the fields… --- ## [Recreate index from GET mappings output](https://discuss.elastic.co/t/recreate-index-from-get-mappings-output/295159)
**Author:** [@markrity](https://discuss.elastic.co/u/markrity)\ **Replies:** 0\ **Last updated:** [January 23, 2022, 8:30pm UTC](https://discuss.elastic.co/t/recreate-index-from-get-mappings-output/295159 "2022-01-23T20:30:26Z")
I'm trying to find a way to recreate all index mappings from when I use GET /\_all/\_mapping or iterate over each index. For creating development cluster purposes. I have tried to use the json I get from get all mappings… --- ## [Xpath Functions in Logstash](https://discuss.elastic.co/t/xpath-functions-in-logstash/295119)
**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\ **Replies:** 4\ **Last updated:** [January 23, 2022, 6:43pm UTC](https://discuss.elastic.co/t/xpath-functions-in-logstash/295119 "2022-01-23T18:43:31Z")
I am needing to use xpath functions like string-join() and concat() to extract what I need from my XML file, however whenever I use these functions I get the following warning in my logstash-plain.log: \[2022-01-21T20:4… --- ## [Getting every 2nd element from \_source](https://discuss.elastic.co/t/getting-every-2nd-element-from-source/295123)
**Author:** [@EnJay](https://discuss.elastic.co/u/EnJay)\ **Replies:** 6\ **Last updated:** [January 23, 2022, 6:19pm UTC](https://discuss.elastic.co/t/getting-every-2nd-element-from-source/295123 "2022-01-23T18:19:25Z")
Hello dear community. I have been looking for information for a very long time. I need to make mapping from search for every 2nd element. For example, I have "size": 1000, while I want to get every second element as a … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1081) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1083)