# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1413 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1414 --- ## [Ruby pipeline permission denied sending to logstash](https://discuss.elastic.co/t/ruby-pipeline-permission-denied-sending-to-logstash/272584)
**Author:** [@Stan3005](https://discuss.elastic.co/u/Stan3005)\ **Replies:** 3\ **Last updated:** [May 17, 2021, 9:49am UTC](https://discuss.elastic.co/t/ruby-pipeline-permission-denied-sending-to-logstash/272584 "2021-05-17T09:49:27Z")
Hi, So I've set up an ELK stack. First time setting one up but installed elastic, kibana and logstash. When I go to localhost:5601 it takes me to the elastic dashboard. The problem I'm having is when I'm trying to sen… --- ## [\[ES v7.9.1\] What is the largest useful size setting for terms aggregations](https://discuss.elastic.co/t/es-v7-9-1-what-is-the-largest-useful-size-setting-for-terms-aggregations/273153)
**Author:** [@str8y](https://discuss.elastic.co/u/str8y)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 9:28am UTC](https://discuss.elastic.co/t/es-v7-9-1-what-is-the-largest-useful-size-setting-for-terms-aggregations/273153 "2021-05-17T09:28:38Z")
In order to get all the existing data we set the size parameter for each terms aggregation to 2147483647 (Integer.MAX) Depending on the number of documents, these queries run forever and consume massive memory. Actually… --- ## [How to connect sets of data in Kibana?](https://discuss.elastic.co/t/how-to-connect-sets-of-data-in-kibana/273150)
**Author:** [@Timo\_Warken](https://discuss.elastic.co/u/Timo_Warken)\ **Replies:** 2\ **Last updated:** [May 17, 2021, 9:26am UTC](https://discuss.elastic.co/t/how-to-connect-sets-of-data-in-kibana/273150 "2021-05-17T09:26:58Z")
Hi there, I am not too deep in the matter, so please have forbearance. Here is my question: Is ist possible in Kibana to have two different datasets connected on one Point? For example if I have a set of data related to … --- ## [Output - support of keydb](https://discuss.elastic.co/t/output-support-of-keydb/273145)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 8:57am UTC](https://discuss.elastic.co/t/output-support-of-keydb/273145 "2021-05-17T08:57:55Z")
Hello, do you plan to support KEYDB (fork of Redis) as a output of Beats? --- ## [Connecting documents from different kibana indexes](https://discuss.elastic.co/t/connecting-documents-from-different-kibana-indexes/271983)
**Author:** [@LL\_FED](https://discuss.elastic.co/u/LL_FED)\ **Replies:** 8\ **Last updated:** [May 17, 2021, 7:36am UTC](https://discuss.elastic.co/t/connecting-documents-from-different-kibana-indexes/271983 "2021-05-17T07:36:16Z")
Hello Elastic-Community, I am wondering if there is a way to connect two datasets from two different indexes with each other to create a visualization. For example: From Index\_1 I receive general informations of a dev… --- ## [Filebeat dropping events](https://discuss.elastic.co/t/filebeat-dropping-events/271653)
**Author:** [@smanoranjan005](https://discuss.elastic.co/u/smanoranjan005)\ **Replies:** 6\ **Last updated:** [May 17, 2021, 7:50am UTC](https://discuss.elastic.co/t/filebeat-dropping-events/271653 "2021-05-17T07:50:38Z")
Hi, I am using filebeat as daemonset in Kubernetes to transfer application logs to Logstash to Elasticsearch. But the issue here is there is some data drop during this transfer to ES. There is only one filter in filebe… --- ## [GeoPoint vs GeoLocation and Search and Sorting](https://discuss.elastic.co/t/geopoint-vs-geolocation-and-search-and-sorting/273018)
**Author:** [@Kirtash](https://discuss.elastic.co/u/Kirtash)\ **Replies:** 4\ **Last updated:** [May 17, 2021, 7:41am UTC](https://discuss.elastic.co/t/geopoint-vs-geolocation-and-search-and-sorting/273018 "2021-05-17T07:41:28Z")
Good morning, Now I'm beginning to work with geoLocations and I'm using the library nest to interact to ElasticSearch, but my first question is: Whats is more correct? GeoLocation type or geoPoint: Location "locatio… --- ## [Restarting elasticsearch (via systemctl): Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for details](https://discuss.elastic.co/t/restarting-elasticsearch-via-systemctl-job-for-elasticsearch-service-failed-because-the-control-process-exited-with-error-code-see-systemctl-status-elasticsearch-service-and-journalctl-xe-for-details/273137)
**Author:** [@confidential](https://discuss.elastic.co/u/confidential)\ **Replies:** 1\ **Last updated:** [May 17, 2021, 7:39am UTC](https://discuss.elastic.co/t/restarting-elasticsearch-via-systemctl-job-for-elasticsearch-service-failed-because-the-control-process-exited-with-error-code-see-systemctl-status-elasticsearch-service-and-journalctl-xe-for-details/273137 "2021-05-17T07:39:38Z")
Restarting elasticsearch (via systemctl): Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for details. ## JVM c… --- ## [How to avoid outputting filebeat logs to /var/log/messages](https://discuss.elastic.co/t/how-to-avoid-outputting-filebeat-logs-to-var-log-messages/272642)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 2\ **Last updated:** [May 17, 2021, 7:31am UTC](https://discuss.elastic.co/t/how-to-avoid-outputting-filebeat-logs-to-var-log-messages/272642 "2021-05-17T07:31:54Z")
I do not want to output filebeat logs to /var/log/messages. So, I configured to\_syslog and Environment, referring to the following article. However, they do not work. Can you please tell me how to do it correctly? H… --- ## [Need Help with Logstash Multiline Codec](https://discuss.elastic.co/t/need-help-with-logstash-multiline-codec/273134)
**Author:** [@Pemitha\_Randiya](https://discuss.elastic.co/u/Pemitha_Randiya)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 7:01am UTC](https://discuss.elastic.co/t/need-help-with-logstash-multiline-codec/273134 "2021-05-17T07:01:39Z")
Here is my sample log data : 2021/05/17 12:17:41.866 | 382DC9789F284D3C9D7CC269F1851094|0000SUMMARY:\[INFO \]: 2021/05/17 12:17:41.866 |--------------------------------------------------------------------- 2021/05/1… --- ## [Does the index API block responses until the indexing process is through?](https://discuss.elastic.co/t/does-the-index-api-block-responses-until-the-indexing-process-is-through/273115)
**Author:** [@Courtan](https://discuss.elastic.co/u/Courtan)\ **Replies:** 9\ **Last updated:** [May 17, 2021, 6:54am UTC](https://discuss.elastic.co/t/does-the-index-api-block-responses-until-the-indexing-process-is-through/273115 "2021-05-17T06:54:14Z")
I am wondering wether the index API waits to return a response until the whole indexing process is complete. More specifically: When I send a document to the API for indexing, will ES only check wether the request, json… --- ## [How to use an array as a param for feature logging (Elasticsearch learning to rank plugin)](https://discuss.elastic.co/t/how-to-use-an-array-as-a-param-for-feature-logging-elasticsearch-learning-to-rank-plugin/273112)
**Author:** [@ndkmath1](https://discuss.elastic.co/u/ndkmath1)\ **Replies:** 4\ **Last updated:** [May 17, 2021, 6:41am UTC](https://discuss.elastic.co/t/how-to-use-an-array-as-a-param-for-feature-logging-elasticsearch-learning-to-rank-plugin/273112 "2021-05-17T06:41:02Z")
I want to create a feature in my featureset and its param is an array. I known how to use a single value for feature logging but I don't know how to use an array. I tried something like below but it didn't help. Do you h… --- ## [Impact of setting max\_inner\_result\_window at index level](https://discuss.elastic.co/t/impact-of-setting-max-inner-result-window-at-index-level/273128)
**Author:** [@surabhi11](https://discuss.elastic.co/u/surabhi11)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 6:20am UTC](https://discuss.elastic.co/t/impact-of-setting-max-inner-result-window-at-index-level/273128 "2021-05-17T06:20:02Z")
Hi, We have a user index and it has recipients (nested object). Since recipients are saved in User domain as nested object, so in order to pull the recipients data with pagination we are using inner\_hits query in respe… --- ## [CPU loading: 100% under load after 10-12 hrs of usage](https://discuss.elastic.co/t/cpu-loading-100-under-load-after-10-12-hrs-of-usage/271041)
**Author:** [@Dmitry\_Uglatch](https://discuss.elastic.co/u/Dmitry_Uglatch)\ **Replies:** 8\ **Last updated:** [May 17, 2021, 6:06am UTC](https://discuss.elastic.co/t/cpu-loading-100-under-load-after-10-12-hrs-of-usage/271041 "2021-05-17T06:06:45Z")
Hello, we're switching from ES 5.5 to 7.12. After applying it works perfect for about 10-12 hrs: and then we get CPU loading: 100% Even simple queries: GET /products/\_search?size=0 works for 5-6 seconds. And it return… --- ## [Daily reingestion of data](https://discuss.elastic.co/t/daily-reingestion-of-data/272972)
**Author:** [@SamKidman](https://discuss.elastic.co/u/SamKidman)\ **Replies:** 2\ **Last updated:** [May 17, 2021, 4:42am UTC](https://discuss.elastic.co/t/daily-reingestion-of-data/272972 "2021-05-17T04:42:21Z")
We've had an elasticsearch setup for a while now that has worked by reingesting all data from a relational database into a new index in denormalised form. We now want to perform immediate updates to the data in elastics… --- ## [Watcher X-Pack Monitoring: Nodes Changed](https://discuss.elastic.co/t/watcher-x-pack-monitoring-nodes-changed/272473)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 14\ **Last updated:** [May 17, 2021, 4:24am UTC](https://discuss.elastic.co/t/watcher-x-pack-monitoring-nodes-changed/272473 "2021-05-17T04:24:06Z")
Hi I have a cluster with 4nodes of data and 1node for kibana. about 5 days ago one of the built-in watcher (Nodes Changed) run most of the times. { "watch\_id": "kVfChQJ8TWWjr5Mrbw\_nEg\_elasticsearch\_nodes", "node": … --- ## [Suggest score up by hit position](https://discuss.elastic.co/t/suggest-score-up-by-hit-position/273118)
**Author:** [@Yuya\_Takao](https://discuss.elastic.co/u/Yuya_Takao)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 1:21am UTC](https://discuss.elastic.co/t/suggest-score-up-by-hit-position/273118 "2021-05-17T01:21:52Z")
Hi there, I want to suggest score up by hit position. Is there any idea? I know boost option, but I have no idea to score up by hit position. for example: \[search word 'a'\] largest score 'august' second largest sc… --- ## [Does indices()-\>refresh() wait for the completion of Bulk API?](https://discuss.elastic.co/t/does-indices-refresh-wait-for-the-completion-of-bulk-api/272914)
**Author:** [@Catalin](https://discuss.elastic.co/u/Catalin)\ **Replies:** 1\ **Last updated:** [May 17, 2021, 12:38am UTC](https://discuss.elastic.co/t/does-indices-refresh-wait-for-the-completion-of-bulk-api/272914 "2021-05-17T00:38:12Z")
In order to populate an index I have disabled its refresh\_interval and I'm using the Bulk API for inserting. The problem is I'm doing this in chunks of data and I need that after each chunk all the information is availa… --- ## [Reindex order](https://discuss.elastic.co/t/reindex-order/273114)
**Author:** [@alexs1](https://discuss.elastic.co/u/alexs1)\ **Replies:** 3\ **Last updated:** [May 17, 2021, 12:36am UTC](https://discuss.elastic.co/t/reindex-order/273114 "2021-05-17T00:36:34Z")
Hello, Does the reindex save the original order of the documents? Does the answer the same when using slice? Thank you! --- ## [What happens to snapshots when a new node joins the cluster?](https://discuss.elastic.co/t/what-happens-to-snapshots-when-a-new-node-joins-the-cluster/272907)
**Author:** [@nadler](https://discuss.elastic.co/u/nadler)\ **Replies:** 1\ **Last updated:** [May 17, 2021, 12:36am UTC](https://discuss.elastic.co/t/what-happens-to-snapshots-when-a-new-node-joins-the-cluster/272907 "2021-05-17T00:36:19Z")
Hey all, I am implementing a backup and restore process on our clusters with hdfs and i was wandering what happens when a new node joins the cluster or an old one is replaced with a new machine? Because at first, we ne… --- ## [Secure Elasticsearch with LDAP or Active Directory - questions](https://discuss.elastic.co/t/secure-elasticsearch-with-ldap-or-active-directory-questions/272944)
**Author:** [@d.silwon](https://discuss.elastic.co/u/d.silwon)\ **Replies:** 0\ **Last updated:** [May 13, 2021, 6:43pm UTC](https://discuss.elastic.co/t/secure-elasticsearch-with-ldap-or-active-directory-questions/272944 "2021-05-13T18:43:11Z")
Dears, Who can explain me the differences between two these methods of authentication? Active Directory user authentication vs LDAP user authentication On the base of Elastic documentation I understand that I need t… --- ## [Master nodes metrics](https://discuss.elastic.co/t/master-nodes-metrics/272994)
**Author:** [@Timur\_Makarchuk](https://discuss.elastic.co/u/Timur_Makarchuk)\ **Replies:** 1\ **Last updated:** [May 17, 2021, 12:28am UTC](https://discuss.elastic.co/t/master-nodes-metrics/272994 "2021-05-17T00:28:56Z")
What are some key metrics to monitor master nodes? We currently have a cluster with 30 nodes. We have 3 master-eligible nodes which also serve as data-nodes. I suspect our cluster is oversharded (~1500 shards per node)… --- ## [Setup logstash to stop reading a file after some time](https://discuss.elastic.co/t/setup-logstash-to-stop-reading-a-file-after-some-time/273117)
**Author:** [@Amit\_Singh3](https://discuss.elastic.co/u/Amit_Singh3)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 10:33pm UTC](https://discuss.elastic.co/t/setup-logstash-to-stop-reading-a-file-after-some-time/273117 "2021-05-16T22:33:09Z")
We are using logstsash to read a folder to read logs, due to log rotation tons of files are being created in the folder, what if I want logstash to stop reading from a file if there is no new data in a file for 5 minute… --- ## [Ingest-convert.sh output incorrect for PaloAlto (pawn) Ingest Pipeline](https://discuss.elastic.co/t/ingest-convert-sh-output-incorrect-for-paloalto-pawn-ingest-pipeline/273076)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 4:38pm UTC](https://discuss.elastic.co/t/ingest-convert-sh-output-incorrect-for-paloalto-pawn-ingest-pipeline/273076 "2021-05-16T16:38:10Z")
Hi, I am planning to use Logstash instead of Filebeats for loading and parsing palo alto logs to Elastic. I setup the Filebeats as a one-time to generate the Ingest Pipeline Definition using the "pawn" Filebeats module… --- ## [Beats not able to connect with Logstash](https://discuss.elastic.co/t/beats-not-able-to-connect-with-logstash/273093)
**Author:** [@robinhood\_lko](https://discuss.elastic.co/u/robinhood_lko)\ **Replies:** 2\ **Last updated:** [May 16, 2021, 4:29pm UTC](https://discuss.elastic.co/t/beats-not-able-to-connect-with-logstash/273093 "2021-05-16T16:29:09Z")
Hi All, I have deployed ELK Stack using OSS versions Elasticsearch and Kibana. While connecting with Beats using Logstash, I am seeing following error on the system. Please can someone help me on this. Thanks \[2021-05-… --- ## [Kibana not accessible anymore when index was close](https://discuss.elastic.co/t/kibana-not-accessible-anymore-when-index-was-close/273075)
**Author:** [@blitzkrieg](https://discuss.elastic.co/u/blitzkrieg)\ **Replies:** 6\ **Last updated:** [May 16, 2021, 2:50pm UTC](https://discuss.elastic.co/t/kibana-not-accessible-anymore-when-index-was-close/273075 "2021-05-16T14:50:37Z")
Hi Guys, I'm new to elasticsearch and would like to ask your expertise about. We are using ECK 7.9.3 and trying to restore a snapshot however, I was advice that make sure that all indices are closed before restoring sn… --- ## [Index policy or Index template for Elasticsearh](https://discuss.elastic.co/t/index-policy-or-index-template-for-elasticsearh/272931)
**Author:** [@KA3AXCTAH](https://discuss.elastic.co/u/KA3AXCTAH)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 2:00pm UTC](https://discuss.elastic.co/t/index-policy-or-index-template-for-elasticsearh/272931 "2021-05-16T14:00:43Z")
I have elasticsearch cluster for storing logs, and i have indices like this logs-2021.01.01 logs-2021.01.02 logs.2021.01.03 ...etc so indices creates at daily basis, and i have index template for this indices PUT \_ind… --- ## [\[Elasticsearch\] Estimating the duration of a reindex](https://discuss.elastic.co/t/elasticsearch-estimating-the-duration-of-a-reindex/272929)
**Author:** [@daniele.saccon](https://discuss.elastic.co/u/daniele.saccon)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 1:56pm UTC](https://discuss.elastic.co/t/elasticsearch-estimating-the-duration-of-a-reindex/272929 "2021-05-16T13:56:19Z")
Hello everybody, is it possible to estimate the duration of a reindex between one cluster and another? Knowing the total GB and the characteristics of the VMs hosting the Elasticsearchs. Thanks Daniele --- ## [Input file plugin](https://discuss.elastic.co/t/input-file-plugin/273109)
**Author:** [@hsalim](https://discuss.elastic.co/u/hsalim)\ **Replies:** 0\ **Last updated:** [May 16, 2021, 12:14pm UTC](https://discuss.elastic.co/t/input-file-plugin/273109 "2021-05-16T12:14:16Z")
Question 1: Has anyone seen following message with input File plugin? \[2021-05-15T23:26:23,609\]\[DEBUG\]\[logstash.javapipeline \] Shutdown waiting for worker thread {:pipeline\_id=\>"rawdiameter1200", :thread=\>"#\ **Author:** [@Anoop\_kumar\_SRIVASTA](https://discuss.elastic.co/u/Anoop_kumar_SRIVASTA)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 10:42am UTC](https://discuss.elastic.co/t/unable-to-install-output-jdbc-plugin-on-logstash/273062 "2021-05-16T10:42:40Z") Hi I am facing issue while connecting logstash output send to mysql database..below are my configuration setting output { jdbc{ driver\_jar\_path =\> "/app/logstash-CBS/mysql-connector-java-8.0.25.jar" driver\_class =\> … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1412) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1414)