# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1396 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1397 --- ## [How can i filter errors in filebeat](https://discuss.elastic.co/t/how-can-i-filter-errors-in-filebeat/273893)
**Author:** [@Martin\_perez](https://discuss.elastic.co/u/Martin_perez)\ **Replies:** 4\ **Last updated:** [May 27, 2021, 8:20am UTC](https://discuss.elastic.co/t/how-can-i-filter-errors-in-filebeat/273893 "2021-05-27T08:20:14Z")
Hi, I have an elk stack mounted, which receives data from two filebeats. I want to configure my filebeat configuration files so that it only collects files whose error is anything other than 500, and I have tried to do … --- ## [Logtash |7.12.1 | Multiple pipelines](https://discuss.elastic.co/t/logtash-7-12-1-multiple-pipelines/273607)
**Author:** [@Tarun\_N](https://discuss.elastic.co/u/Tarun_N)\ **Replies:** 5\ **Last updated:** [May 27, 2021, 8:12am UTC](https://discuss.elastic.co/t/logtash-7-12-1-multiple-pipelines/273607 "2021-05-27T08:12:34Z")
Hello All I have to configure multiple pipelines for which I carried out the changes in pipelines.yml with simple test configuration, after removing all the code to debug to make it work. Below is the sample configurati… --- ## [Filebeat cannot merge partial events](https://discuss.elastic.co/t/filebeat-cannot-merge-partial-events/274119)
**Author:** [@delta9](https://discuss.elastic.co/u/delta9)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 7:55am UTC](https://discuss.elastic.co/t/filebeat-cannot-merge-partial-events/274119 "2021-05-27T07:55:39Z")
Hi all. We use filebeat to collect events from pods in k8s cluster. To do this we run it as daemonset and mounts /var/lib/docker/containers, /var/log/pods and /var/log/containers from cluster nodes into filebeat pods. … --- ## [FileBeat Startup error - No paths were defined for input accessing](https://discuss.elastic.co/t/filebeat-startup-error-no-paths-were-defined-for-input-accessing/274014)
**Author:** [@Parveen\_Sharma](https://discuss.elastic.co/u/Parveen_Sharma)\ **Replies:** 4\ **Last updated:** [May 27, 2021, 7:52am UTC](https://discuss.elastic.co/t/filebeat-startup-error-no-paths-were-defined-for-input-accessing/274014 "2021-05-27T07:52:17Z")
I am using filebeat 7.8.1 Following is the filebeat configuration file - type: kubernetes in\_cluster: true tags: - "kubernetes" templates: - condi… --- ## [Index is not Rollover](https://discuss.elastic.co/t/index-is-not-rollover/273636)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [May 27, 2021, 7:42am UTC](https://discuss.elastic.co/t/index-is-not-rollover/273636 "2021-05-27T07:42:56Z")
I have 8 node cluster 3 master 3 hot node and 2 warm node.I have an index called log-wlb-system in hot node it will rotate every index after 1 day or when its size reach to 1gb and in warm node the retention is 2 days … --- ## [Is it possible to embed a canvas in a document?](https://discuss.elastic.co/t/is-it-possible-to-embed-a-canvas-in-a-document/274132)
**Author:** [@Ali\_Nazemian](https://discuss.elastic.co/u/Ali_Nazemian)\ **Replies:** 3\ **Last updated:** [May 27, 2021, 7:26am UTC](https://discuss.elastic.co/t/is-it-possible-to-embed-a-canvas-in-a-document/274132 "2021-05-27T07:26:06Z")
I am looking for a way that a Canvas object can be embedded in a document? I am trying to create a custom template for it to generate some high-level reports in our organisation. I was wondering if anybody has used Canva… --- ## [Reindex API is extremely slow](https://discuss.elastic.co/t/reindex-api-is-extremely-slow/273510)
**Author:** [@abhinav2](https://discuss.elastic.co/u/abhinav2)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 6:57am UTC](https://discuss.elastic.co/t/reindex-api-is-extremely-slow/273510 "2021-05-27T06:57:26Z")
I'm getting a write speed of 680 KB/s per node for the reindex API. Cluster is a 16 data node cluster with 3 master nodes. Each data node has 1.3 TB gp3 Disk with 3000 IOPS and 125 MBps throughput. Reindex API is using… --- ## [Terms query success on ID, fails on everything else](https://discuss.elastic.co/t/terms-query-success-on-id-fails-on-everything-else/274075)
**Author:** [@pyrovoice](https://discuss.elastic.co/u/pyrovoice)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 6:47am UTC](https://discuss.elastic.co/t/terms-query-success-on-id-fails-on-everything-else/274075 "2021-05-27T06:47:50Z")
Doing the following search: curl --location --request POST 'localhost:9200/author/\_search?pretty' \\ --header 'Content-Type: application/json' \\ --data-raw '{ "query": { "terms": { "\_id": \[ "myName" \] … --- ## [Help in a search query](https://discuss.elastic.co/t/help-in-a-search-query/274112)
**Author:** [@appydam](https://discuss.elastic.co/u/appydam)\ **Replies:** 2\ **Last updated:** [May 27, 2021, 6:24am UTC](https://discuss.elastic.co/t/help-in-a-search-query/274112 "2021-05-27T06:24:59Z")
I have {id, title, short\_description and description} in my document. I have to do search query on description text and want to reach title. Like if a person enters something in the description then matching titles appe… --- ## [ElasticSearch Windows 2019 service fails to start](https://discuss.elastic.co/t/elasticsearch-windows-2019-service-fails-to-start/274137)
**Author:** [@ddemland](https://discuss.elastic.co/u/ddemland)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 6:13am UTC](https://discuss.elastic.co/t/elasticsearch-windows-2019-service-fails-to-start/274137 "2021-05-27T06:13:56Z")
I have installed ElasticSerach on Windows 2019 server. When I start the service manually it starts, but seems to be a little slow. When I reboot the system the Event log shows the following error: The Elasticsearch serv… --- ## [Same Condition Logs Parse on Logstash](https://discuss.elastic.co/t/same-condition-logs-parse-on-logstash/274139)
**Author:** [@Nazakat](https://discuss.elastic.co/u/Nazakat)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 6:13am UTC](https://discuss.elastic.co/t/same-condition-logs-parse-on-logstash/274139 "2021-05-27T06:13:30Z")
how to parse logs in same condition? --- ## [Help/Advice needed setting up geo-ip filters in an on-prem Logstash to SIEM in Elastic Cloud instance](https://discuss.elastic.co/t/help-advice-needed-setting-up-geo-ip-filters-in-an-on-prem-logstash-to-siem-in-elastic-cloud-instance/273593)
**Author:** [@ronmer](https://discuss.elastic.co/u/ronmer)\ **Replies:** 11\ **Last updated:** [May 27, 2021, 5:26am UTC](https://discuss.elastic.co/t/help-advice-needed-setting-up-geo-ip-filters-in-an-on-prem-logstash-to-siem-in-elastic-cloud-instance/273593 "2021-05-27T05:26:54Z")
I need help confirming the exact steps required to get geo-ip information to map to the SIEM network map in Kabana My environment: Windows and Linux hosts running either Auditbeat, Packetbeat or Winlogbeat, Packetbeat … --- ## [Elastic-APM Invalid index name \[\_license\]](https://discuss.elastic.co/t/elastic-apm-invalid-index-name-license/274131)
**Author:** [@pszm](https://discuss.elastic.co/u/pszm)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 5:01am UTC](https://discuss.elastic.co/t/elastic-apm-invalid-index-name-license/274131 "2021-05-27T05:01:24Z")
We are trying to get elastic-apm install (for now on our development systems). According to Homebrew, we have the latest elasticsearch-oss (7.10.2), kibana-oss (7.10.2) and today installed apm-server-oss (which is at ve… --- ## [Elasticsearch query type alert in kibana alerts](https://discuss.elastic.co/t/elasticsearch-query-type-alert-in-kibana-alerts/273921)
**Author:** [@Priyanka\_chauhan](https://discuss.elastic.co/u/Priyanka_chauhan)\ **Replies:** 5\ **Last updated:** [May 27, 2021, 4:46am UTC](https://discuss.elastic.co/t/elasticsearch-query-type-alert-in-kibana-alerts/273921 "2021-05-27T04:46:14Z")
I am creating elasticsearch query type alerts in Kibana Alerts and Action section. I have put a condition to send an alert when blacklist IP met with threshold condition in my query. Alerts are generated in index with p… --- ## [Transfer data/fields from one index to another](https://discuss.elastic.co/t/transfer-data-fields-from-one-index-to-another/273961)
**Author:** [@Nisha2297](https://discuss.elastic.co/u/Nisha2297)\ **Replies:** 4\ **Last updated:** [May 27, 2021, 4:07am UTC](https://discuss.elastic.co/t/transfer-data-fields-from-one-index-to-another/273961 "2021-05-27T04:07:16Z")
Can anybody help me moving the data from one index to another in Kibana. I have an index "abc" with lot number of documents and say 100 fields and I want to extract 10 fields out of it and with some filter applied to re… --- ## [Metricbeat Elasticsearch Module not working](https://discuss.elastic.co/t/metricbeat-elasticsearch-module-not-working/274020)
**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 3:54am UTC](https://discuss.elastic.co/t/metricbeat-elasticsearch-module-not-working/274020 "2021-05-27T03:54:44Z")
Issue - Trying to set up metricbeat to collect ES metric but upon testing the elasticsearch-xpack module, its not able to connect except for "node\_stats" metric and generating the error as shown below. Can someone poin… --- ## [Give A User Permissions To On-board New Users](https://discuss.elastic.co/t/give-a-user-permissions-to-on-board-new-users/274059)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 3:08am UTC](https://discuss.elastic.co/t/give-a-user-permissions-to-on-board-new-users/274059 "2021-05-27T03:08:06Z")
I want to create a user who can create and manage new user onboarding. What I did is, created a user and assign a role with manage\_security permission for the cluster. Problem: the user who is assigned with manage\_secu… --- ## [“No results match your search criteria”](https://discuss.elastic.co/t/no-results-match-your-search-criteria/274043)
**Author:** [@sosogh](https://discuss.elastic.co/u/sosogh)\ **Replies:** 2\ **Last updated:** [May 27, 2021, 2:05am UTC](https://discuss.elastic.co/t/no-results-match-your-search-criteria/274043 "2021-05-27T02:05:51Z")
The timestamp format in the original log is "2021-05-26 18:00:00". I use fluentd to "modify" it to "2021-05-26T18:00:00.000Z" , supply it as the value of key "fulltime" , and then send it to ES. When creating index … --- ## [After enable xpack i cant change passwords](https://discuss.elastic.co/t/after-enable-xpack-i-cant-change-passwords/274092)
**Author:** [@Dimitri\_Goldshtein](https://discuss.elastic.co/u/Dimitri_Goldshtein)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 1:22am UTC](https://discuss.elastic.co/t/after-enable-xpack-i-cant-change-passwords/274092 "2021-05-27T01:22:07Z")
After enable xpack i cant change passwords, commen to discovery.seed\_hosts and cluster.initial\_master\_nodes did not help! I get this error when trying to setup password: /usr/share/elasticsearch/bin# sudo ./elast… --- ## [Filebeat (with ILM) -\> Logstash -\> Elasticsearch not working](https://discuss.elastic.co/t/filebeat-with-ilm-logstash-elasticsearch-not-working/273895)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 5\ **Last updated:** [May 27, 2021, 1:18am UTC](https://discuss.elastic.co/t/filebeat-with-ilm-logstash-elasticsearch-not-working/273895 "2021-05-27T01:18:32Z")
Hi, I think I am missing some simple settings here. I have enabled the ILM in beats: setup.ilm.enabled: true setup.ilm.rollover\_alias: "log-dev-filebeat" setup.ilm.pattern: "{now/d}-000001" setup.ilm.policy\_name: "fil… --- ## [Merge lines in filebeat](https://discuss.elastic.co/t/merge-lines-in-filebeat/273981)
**Author:** [@syrine\_chelly](https://discuss.elastic.co/u/syrine_chelly)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 1:14am UTC](https://discuss.elastic.co/t/merge-lines-in-filebeat/273981 "2021-05-27T01:14:00Z")
this is an extract from my file log i want to merge the lines for example under "Server 'ILPPRAGLM', Line 14" in one event every time it finds Server it merges the lines how can i do it without merging the other lin… --- ## [Does Kibana Alerting API also configure Elastic Stack Monitoring Alerts?](https://discuss.elastic.co/t/does-kibana-alerting-api-also-configure-elastic-stack-monitoring-alerts/273776)
**Author:** [@Sumit\_Gupta2](https://discuss.elastic.co/u/Sumit_Gupta2)\ **Replies:** 4\ **Last updated:** [May 26, 2021, 11:55pm UTC](https://discuss.elastic.co/t/does-kibana-alerting-api-also-configure-elastic-stack-monitoring-alerts/273776 "2021-05-26T23:55:07Z")
I have been using Kibana Alert API to create stack monitoring alerts for elastic search cluster, however, it doesn't work: POST /api/alerts/alert { "params":{ "aggType":"avg", "termSize":6, "threshol… --- ## [Logstash stdout output Module](https://discuss.elastic.co/t/logstash-stdout-output-module/274127)
**Author:** [@vishakh](https://discuss.elastic.co/u/vishakh)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 11:45pm UTC](https://discuss.elastic.co/t/logstash-stdout-output-module/274127 "2021-05-26T23:45:54Z")
ELK Stack-7.10.0 Filebeat-7.10.0 I've configured the filebeat output to logstash. From the Kibana-UI, I've configured the logstash pipeline as below. Objective: Print the log from filebeat on the stdout input { b… --- ## [Actions & alerts - suppress + using to alert on security detections](https://discuss.elastic.co/t/actions-alerts-suppress-using-to-alert-on-security-detections/274021)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 8:02am UTC](https://discuss.elastic.co/t/actions-alerts-suppress-using-to-alert-on-security-detections/274021 "2021-05-26T08:02:46Z")
Hi, Looking at the guide for Alerts & Actions there is mention of suppressing alerts - However i just cannot seem to find this option. Also i am trying to find a better way to manage the detection alerts, on prem i u… --- ## [How to decide storage size of an index in Elasticsearch](https://discuss.elastic.co/t/how-to-decide-storage-size-of-an-index-in-elasticsearch/274076)
**Author:** [@rknd](https://discuss.elastic.co/u/rknd)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 11:10pm UTC](https://discuss.elastic.co/t/how-to-decide-storage-size-of-an-index-in-elasticsearch/274076 "2021-05-26T23:10:06Z")
I want to ask, How should i decide storage size for an index ? Is there any storage limit for shards or am i able to store 20 gb per shard and will it perform same ? Thanks for answering and explaining --- ## [Elasticsearch gc overhead](https://discuss.elastic.co/t/elasticsearch-gc-overhead/274089)
**Author:** [@muralikri32](https://discuss.elastic.co/u/muralikri32)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 11:09pm UTC](https://discuss.elastic.co/t/elasticsearch-gc-overhead/274089 "2021-05-26T23:09:08Z")
Hi team, i am using ES 6.6.X version in K8 environment and running as 3 node cluster. each node has 64GB ram and i configured heap memory as 32GB. it is quite running fine for 2 months and recently i could see lot of gc… --- ## [Unable to post data via ElasticLowLevelClient](https://discuss.elastic.co/t/unable-to-post-data-via-elasticlowlevelclient/274078)
**Author:** [@NeilRedfern](https://discuss.elastic.co/u/NeilRedfern)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 2:42pm UTC](https://discuss.elastic.co/t/unable-to-post-data-via-elasticlowlevelclient/274078 "2021-05-26T14:42:54Z")
Hello, I'm attempting to add data into our Elastic cloud instance following the examples within the post Exporting Azure Log data to the ELK stack with Azure Functions and Event Hubs However despite my best efforts I r… --- ## [Legacy index templates](https://discuss.elastic.co/t/legacy-index-templates/273763)
**Author:** [@6NMgfDwZ3](https://discuss.elastic.co/u/6NMgfDwZ3)\ **Replies:** 0\ **Last updated:** [May 23, 2021, 3:48pm UTC](https://discuss.elastic.co/t/legacy-index-templates/273763 "2021-05-23T15:48:21Z")
Hello, Environment: Elasticsearch 7.12.1 (Basic License) Kibana 7.12.1 (Basic License) Filebeat 7.12.1 (Basic License) Ubuntu 21.04 I have an index template named myapp the index pattern myapp-\*. This index template… --- ## [Make all current indexes use another Index Lifecycle Management policy that has a hot-delete phase?](https://discuss.elastic.co/t/make-all-current-indexes-use-another-index-lifecycle-management-policy-that-has-a-hot-delete-phase/273444)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 5\ **Last updated:** [May 26, 2021, 10:57pm UTC](https://discuss.elastic.co/t/make-all-current-indexes-use-another-index-lifecycle-management-policy-that-has-a-hot-delete-phase/273444 "2021-05-26T22:57:36Z")
Hello Piggybacking on my previous topic, Implement Hot-Warm-Cold-Frozen-Delete phases with only one node but multiple storage destinations? I created a Index Lifecycle Management policy that has a hot phase, rollsover t… --- ## [Strange reindex behaviour](https://discuss.elastic.co/t/strange-reindex-behaviour/274108)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 6:38pm UTC](https://discuss.elastic.co/t/strange-reindex-behaviour/274108 "2021-05-26T18:38:04Z")
I'm doing some reindex of daily index to monthly ones and sometimes the reindex process enters in a loop and starts again after it already reindexed the documents. For example, considering the index indexName-2021.05.01 … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1395) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1397)