# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1876 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1877 --- ## ["cluster\_concurrent\_rebalance" : "40" was ignored](https://discuss.elastic.co/t/cluster-concurrent-rebalance-40-was-ignored/243556)
**Author:** [@mannoj87](https://discuss.elastic.co/u/mannoj87)\ **Replies:** 18\ **Last updated:** [August 11, 2020, 6:17am UTC](https://discuss.elastic.co/t/cluster-concurrent-rebalance-40-was-ignored/243556 "2020-08-11T06:17:56Z")
"cluster\_concurrent\_rebalance" : "40" -\> We had this as the setting, when I changed cluster.routing.allocation.awareness.attributes=rack\_id After this I see relocating\_shards = 1000, and it was relocating almost all th… --- ## [What can cause heap size to fluctuate wildly (hacksaw graphs) on a new 7.8.0 cluster?](https://discuss.elastic.co/t/what-can-cause-heap-size-to-fluctuate-wildly-hacksaw-graphs-on-a-new-7-8-0-cluster/244120)
**Author:** [@Jacket](https://discuss.elastic.co/u/Jacket)\ **Replies:** 5\ **Last updated:** [August 11, 2020, 6:04am UTC](https://discuss.elastic.co/t/what-can-cause-heap-size-to-fluctuate-wildly-hacksaw-graphs-on-a-new-7-8-0-cluster/244120 "2020-08-11T06:04:52Z")
Hi there, I've recently migrated a medium sized cluster (15TB, 25B documents) from 6.8 to 7.8.0 and noticed the heap constantly fluctuates between \<1GB and the maximum set size (22GB). However the cluster is working per… --- ## [Can we backup index pattern](https://discuss.elastic.co/t/can-we-backup-index-pattern/244484)
**Author:** [@syafeera](https://discuss.elastic.co/u/syafeera)\ **Replies:** 2\ **Last updated:** [August 11, 2020, 5:57am UTC](https://discuss.elastic.co/t/can-we-backup-index-pattern/244484 "2020-08-11T05:57:13Z")
Hi, may i know can we backup index pattern? --- ## [Create a table of links to dashboards in Kibana](https://discuss.elastic.co/t/create-a-table-of-links-to-dashboards-in-kibana/244486)
**Author:** [@pranayv66](https://discuss.elastic.co/u/pranayv66)\ **Replies:** 5\ **Last updated:** [August 11, 2020, 5:56am UTC](https://discuss.elastic.co/t/create-a-table-of-links-to-dashboards-in-kibana/244486 "2020-08-11T05:56:15Z")
Hi Community, I've a couple of dashboards created in Kibana. I want to create a "table of contents" on the home page dashboard, that will navigate to a new dashboard on its click. I'm using Kibana version 6.3. Can anyon… --- ## [What is the tools/features in Logstash which are related to Integrations part](https://discuss.elastic.co/t/what-is-the-tools-features-in-logstash-which-are-related-to-integrations-part/244487)
**Author:** [@Ismaeel\_Enjreny](https://discuss.elastic.co/u/Ismaeel_Enjreny)\ **Replies:** 0\ **Last updated:** [August 11, 2020, 5:28am UTC](https://discuss.elastic.co/t/what-is-the-tools-features-in-logstash-which-are-related-to-integrations-part/244487 "2020-08-11T05:28:46Z")
Hello dears, What is the tools/features in Logstash which are related to Integrations part in enterprise license Based on the following link: https://www.elastic.co/subscriptions --- ## [Exclude indices from Elastic Search role](https://discuss.elastic.co/t/exclude-indices-from-elastic-search-role/244159)
**Author:** [@amjadbrsc](https://discuss.elastic.co/u/amjadbrsc)\ **Replies:** 3\ **Last updated:** [August 11, 2020, 3:05am UTC](https://discuss.elastic.co/t/exclude-indices-from-elastic-search-role/244159 "2020-08-11T03:05:30Z")
Hi, I have 20 indices. These indices has different naming convention. I am trying to define a ES role where a user can have access to 19 indices except one. I can define a role and mention all 19 indices in the role one… --- ## [Failed to authenticate user \[elastic\]](https://discuss.elastic.co/t/failed-to-authenticate-user-elastic/244285)
**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\ **Replies:** 3\ **Last updated:** [August 11, 2020, 2:18am UTC](https://discuss.elastic.co/t/failed-to-authenticate-user-elastic/244285 "2020-08-11T02:18:28Z")
Running elasticsearch 7.5.2 & noticed that elastic service is continuously generating following info messages indicating elastic authentication failure in logs all the time. Everything works fine on the cluster but the i… --- ## [Beats Multiline Pattern Error](https://discuss.elastic.co/t/beats-multiline-pattern-error/244458)
**Author:** [@ben\_men](https://discuss.elastic.co/u/ben_men)\ **Replies:** 4\ **Last updated:** [August 11, 2020, 1:59am UTC](https://discuss.elastic.co/t/beats-multiline-pattern-error/244458 "2020-08-11T01:59:05Z")
I am trying to ingest multiline log event with filebeat and using Multiline Pattern but it is not working out as expected. Here is one of the multiline event I have: 03 Aug 2020 02:39:53,456 DEBUG \[sometext\] \[sometext\]… --- ## [Why I can't use Elasticsearch for as primary database? not the same way](https://discuss.elastic.co/t/why-i-cant-use-elasticsearch-for-as-primary-database-not-the-same-way/244476)
**Author:** [@chawki](https://discuss.elastic.co/u/chawki)\ **Replies:** 0\ **Last updated:** [August 11, 2020, 1:41am UTC](https://discuss.elastic.co/t/why-i-cant-use-elasticsearch-for-as-primary-database-not-the-same-way/244476 "2020-08-11T01:41:10Z")
This is a common question but in different use case. My application is a buy and sells platform does not launch yet. It uses MongoDB on the server(local) as the primary database to store Ads and I have integrated elasti… --- ## [Failed to create the stack. AWS Lambda Stack event CREATE\_FAILED, ResourceStatusReason: The specified log group does not exist](https://discuss.elastic.co/t/failed-to-create-the-stack-aws-lambda-stack-event-create-failed-resourcestatusreason-the-specified-log-group-does-not-exist/244115)
**Author:** [@JamaesTiger](https://discuss.elastic.co/u/JamaesTiger)\ **Replies:** 2\ **Last updated:** [August 11, 2020, 12:36am UTC](https://discuss.elastic.co/t/failed-to-create-the-stack-aws-lambda-stack-event-create-failed-resourcestatusreason-the-specified-log-group-does-not-exist/244115 "2020-08-11T00:36:20Z")
I have tested how to use functionbeat for AWS cloudwatch via lambda function. But when i deploy funtionbeat, stack creating is failed after the following error generating. # ./functionbeat -v -e -d "\*" deploy cloudwatc… --- ## [Low Watermak warning](https://discuss.elastic.co/t/low-watermak-warning/244334)
**Author:** [@ecarmona](https://discuss.elastic.co/u/ecarmona)\ **Replies:** 8\ **Last updated:** [August 10, 2020, 11:17pm UTC](https://discuss.elastic.co/t/low-watermak-warning/244334 "2020-08-10T23:17:41Z")
Dear All, I hope this message finds you all well. I am working in a mechanism that get the Cluster health every now and then to know if elastic is up and running (Green or Yellow) and then trigger an alert in the case … --- ## [Set processor for a specific Index](https://discuss.elastic.co/t/set-processor-for-a-specific-index/243960)
**Author:** [@Sajal\_Chakraborty](https://discuss.elastic.co/u/Sajal_Chakraborty)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 10:51pm UTC](https://discuss.elastic.co/t/set-processor-for-a-specific-index/243960 "2020-08-10T22:51:12Z")
I am trying to use the below code to create a Set Processor for Index default-\*, but somehow it's not working PUT /\_ingest/pipeline/addaht { "description" : "sets queue wise AHT constants", "processors": \[ { "set":… --- ## [Mapping issue](https://discuss.elastic.co/t/mapping-issue/244421)
**Author:** [@gbasic](https://discuss.elastic.co/u/gbasic)\ **Replies:** 1\ **Last updated:** [August 10, 2020, 10:04pm UTC](https://discuss.elastic.co/t/mapping-issue/244421 "2020-08-10T22:04:14Z")
Hi I am a beginner in Elasticsearch so pardon my silly questions. I am trying to feed a json file through Logstash to Elasticsearch so I can later create visualizations in Kibana. However after I create the index and set… --- ## [Enable Logstash to Talk with Outside Server?](https://discuss.elastic.co/t/enable-logstash-to-talk-with-outside-server/244321)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 10:01pm UTC](https://discuss.elastic.co/t/enable-logstash-to-talk-with-outside-server/244321 "2020-08-10T22:01:42Z")
Hi Logstash, While processing data records, I need my Logstash instance (ver 7.7.1) to send a copy of the entire data record to an outside server. The server does some processing, then sends back a one-string response.… --- ## [Elasticsearch and AWS authenticating using IP instead of IAM user](https://discuss.elastic.co/t/elasticsearch-and-aws-authenticating-using-ip-instead-of-iam-user/244408)
**Author:** [@clevallyAstute](https://discuss.elastic.co/u/clevallyAstute)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 9:56pm UTC](https://discuss.elastic.co/t/elasticsearch-and-aws-authenticating-using-ip-instead-of-iam-user/244408 "2020-08-10T21:56:24Z")
I have an AWS Elasticsearch environment and I believe I’m having trouble authenticating to the Elasticsearch server via IP. We’d like not to use any IAM user policy for this piece of software if possible, and only use an… --- ## [Ingest rate](https://discuss.elastic.co/t/ingest-rate/244465)
**Author:** [@dietrich](https://discuss.elastic.co/u/dietrich)\ **Replies:** 0\ **Last updated:** [August 10, 2020, 9:19pm UTC](https://discuss.elastic.co/t/ingest-rate/244465 "2020-08-10T21:19:01Z")
How do I calculate the number of documents ingested per day? Thread 157245 suggested it's possible to get the number of index requests per second, but the thread did not give any API call(s) for getting this number, and… --- ## [Cross Cluster with 9300 Port](https://discuss.elastic.co/t/cross-cluster-with-9300-port/243818)
**Author:** [@3c3m](https://discuss.elastic.co/u/3c3m)\ **Replies:** 3\ **Last updated:** [August 10, 2020, 9:17pm UTC](https://discuss.elastic.co/t/cross-cluster-with-9300-port/243818 "2020-08-10T21:17:25Z")
Hi, In a scenario, I have 3 different clusters. Do each cluster added to Remote Clusters have to see each other on port 9300? Thanks, Ecem --- ## [Blank message field](https://discuss.elastic.co/t/blank-message-field/244441)
**Author:** [@matheus.santoro](https://discuss.elastic.co/u/matheus.santoro)\ **Replies:** 5\ **Last updated:** [August 10, 2020, 9:17pm UTC](https://discuss.elastic.co/t/blank-message-field/244441 "2020-08-10T21:17:23Z")
Trying to create a simple, non-filtered pipeline, but Kibana shows the message field as blank Pipeline conf is input { tcp { port =\> 9600 codec =\> json mode =\> server } } output { elasticsearch { … --- ## [Filebeat version 6.4.0 unable to send logs to Kafka directly . Filebeat not starting](https://discuss.elastic.co/t/filebeat-version-6-4-0-unable-to-send-logs-to-kafka-directly-filebeat-not-starting/244350)
**Author:** [@Dibyendu\_Dutta](https://discuss.elastic.co/u/Dibyendu_Dutta)\ **Replies:** 8\ **Last updated:** [August 10, 2020, 9:09pm UTC](https://discuss.elastic.co/t/filebeat-version-6-4-0-unable-to-send-logs-to-kafka-directly-filebeat-not-starting/244350 "2020-08-10T21:09:30Z")
I want to send filebeat (logs) data to kafka directly. I have commented logstash/elasticsearch output and set the kafka output like below. But after that my filebeat instance is not starting . Getting error message Fail… --- ## [Replace field value when equals "-"](https://discuss.elastic.co/t/replace-field-value-when-equals/244453)
**Author:** [@gbasic](https://discuss.elastic.co/u/gbasic)\ **Replies:** 1\ **Last updated:** [August 10, 2020, 9:00pm UTC](https://discuss.elastic.co/t/replace-field-value-when-equals/244453 "2020-08-10T21:00:33Z")
Hi I would need to replace a field value (WindDirection,Temp,CityName,WindSpeed,Humidity) to some other value when it equals "-" This is my data: { "@timestamp" =\> 2020-08-10T18:54:08.055Z, "Europe" =\> { … --- ## [Elasticsearch scan of all IDs slows down dramatically (exponentially?)](https://discuss.elastic.co/t/elasticsearch-scan-of-all-ids-slows-down-dramatically-exponentially/242076)
**Author:** [@Timothy\_Wall](https://discuss.elastic.co/u/Timothy_Wall)\ **Replies:** 3\ **Last updated:** [August 10, 2020, 7:59pm UTC](https://discuss.elastic.co/t/elasticsearch-scan-of-all-ids-slows-down-dramatically-exponentially/242076 "2020-08-10T19:59:32Z")
I'm trying to dump all IDs from an index. The dump starts with an ETA of under ten minutes, but then progressively slows down, ultimately taking over an hour to finish. Configuration: single node, two shards, no replic… --- ## [Auditbeat Not Sending -k tags](https://discuss.elastic.co/t/auditbeat-not-sending-k-tags/244439)
**Author:** [@MikeHarness](https://discuss.elastic.co/u/MikeHarness)\ **Replies:** 1\ **Last updated:** [August 10, 2020, 7:22pm UTC](https://discuss.elastic.co/t/auditbeat-not-sending-k-tags/244439 "2020-08-10T19:22:39Z")
I've just set up auditbeat on a RHEL8 box to log various commands. In the 'audit.rules.d' directory I've added a new file with entries such as: -a always,exit -F path=/usr/bin/wget -F perm=x -k external\_call Running a… --- ## [Using missing\_bucket with a composite aggregation with a date\_histogram source doesn't seem to create missing buckets](https://discuss.elastic.co/t/using-missing-bucket-with-a-composite-aggregation-with-a-date-histogram-source-doesnt-seem-to-create-missing-buckets/243635)
**Author:** [@acommend](https://discuss.elastic.co/u/acommend)\ **Replies:** 1\ **Last updated:** [August 10, 2020, 6:21pm UTC](https://discuss.elastic.co/t/using-missing-bucket-with-a-composite-aggregation-with-a-date-histogram-source-doesnt-seem-to-create-missing-buckets/243635 "2020-08-10T18:21:24Z")
Using missing\_bucket with a composite aggregation with a date\_histogram source doesn't seem to create missing buckets. In the example below, 2 documents are created with a date field that are 45 minutes apart. Using a … --- ## [Decode\_json\_fields with message field](https://discuss.elastic.co/t/decode-json-fields-with-message-field/243943)
**Author:** [@cnb3](https://discuss.elastic.co/u/cnb3)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 6:13pm UTC](https://discuss.elastic.co/t/decode-json-fields-with-message-field/243943 "2020-08-10T18:13:46Z")
I am running into a problem of trying to process decode\_json\_fields on the message field out of a cloudwatch event. I have this field populated with a json string generated in a lambda function using JSON.stringify(obj) … --- ## [Number\_format\_exception: Elasticsearch Query Template - check if value of int field present inside array\[int\] field](https://discuss.elastic.co/t/number-format-exception-elasticsearch-query-template-check-if-value-of-int-field-present-inside-array-int-field/244220)
**Author:** [@tinhn](https://discuss.elastic.co/u/tinhn)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 6:07pm UTC](https://discuss.elastic.co/t/number-format-exception-elasticsearch-query-template-check-if-value-of-int-field-present-inside-array-int-field/244220 "2020-08-10T18:07:44Z")
Hello, I'm trying to write a logstash filter that reads from an elasticsearch index using a template. I want to check if an integer matches a value stored in an array of ints. // incoming record { "id": 2 } // query… --- ## [How to merge two different data source into single index](https://discuss.elastic.co/t/how-to-merge-two-different-data-source-into-single-index/244442)
**Author:** [@Hassaan\_mustafa](https://discuss.elastic.co/u/Hassaan_mustafa)\ **Replies:** 0\ **Last updated:** [August 10, 2020, 4:39pm UTC](https://discuss.elastic.co/t/how-to-merge-two-different-data-source-into-single-index/244442 "2020-08-10T16:39:44Z")
I want to Update certain field of index data coming for metricbeat but the fields that need to updated its data is in different data source its in radius how to achieve this --- ## [How do I avoid split brain in dual zone on 7.5?](https://discuss.elastic.co/t/how-do-i-avoid-split-brain-in-dual-zone-on-7-5/244297)
**Author:** [@Mill](https://discuss.elastic.co/u/Mill)\ **Replies:** 6\ **Last updated:** [August 10, 2020, 4:01pm UTC](https://discuss.elastic.co/t/how-do-i-avoid-split-brain-in-dual-zone-on-7-5/244297 "2020-08-10T16:01:51Z")
I have five servers in zone A and zone B each. So I can configure 3 master nodes in zone A and 2 master nodes in zone B a cluster. Or I can use a cluster of 3 master nodes in zone A and zone B, and set up a tribe node… --- ## [Filebeat queue.mem vs Filebeat Netflow module queue\_size](https://discuss.elastic.co/t/filebeat-queue-mem-vs-filebeat-netflow-module-queue-size/244403)
**Author:** [@tlazarus](https://discuss.elastic.co/u/tlazarus)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 3:32pm UTC](https://discuss.elastic.co/t/filebeat-queue-mem-vs-filebeat-netflow-module-queue-size/244403 "2020-08-10T15:32:28Z")
When using the Filebeat Netflow Module, there are 2 queue options - one in the primary yaml file (queue.mem) and another in the module yaml (queue\_size). What is the difference between the 2 options? I would assume that … --- ## [AIX "beat like" daemon, written in Python3](https://discuss.elastic.co/t/aix-beat-like-daemon-written-in-python3/244070)
**Author:** [@bhm-ibm](https://discuss.elastic.co/u/bhm-ibm)\ **Replies:** 3\ **Last updated:** [August 10, 2020, 3:37pm UTC](https://discuss.elastic.co/t/aix-beat-like-daemon-written-in-python3/244070 "2020-08-10T15:37:32Z")
Hello All, I've been ask to write an ELK Metricbeat/Filebeat daemon for AIX , like it is currently existing for Linux and Windows. It is called "Metraixbeat" and I wanted to share it with others ! I created also dozen … --- ## [Getting 401 Unauthorized with Elasticsearch module](https://discuss.elastic.co/t/getting-401-unauthorized-with-elasticsearch-module/243737)
**Author:** [@scriptdb](https://discuss.elastic.co/u/scriptdb)\ **Replies:** 2\ **Last updated:** [August 10, 2020, 3:31pm UTC](https://discuss.elastic.co/t/getting-401-unauthorized-with-elasticsearch-module/243737 "2020-08-10T15:31:25Z")
Hi, I'm using metricbeat 7.6.2 and trying to get elasticsearch metrics (with module). - module: elasticsearch metricsets: - node period: 10s enabled: true hosts: \["http://internal:8081"\] basepath: "/es/" … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1875) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1877)