# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1883 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1884 --- ## [Not able to load custom data using csv file in logstash](https://discuss.elastic.co/t/not-able-to-load-custom-data-using-csv-file-in-logstash/243891)
**Author:** [@Divya\_Mathur\_ELK](https://discuss.elastic.co/u/Divya_Mathur_ELK)\ **Replies:** 3\ **Last updated:** [August 6, 2020, 1:02pm UTC](https://discuss.elastic.co/t/not-able-to-load-custom-data-using-csv-file-in-logstash/243891 "2020-08-06T13:02:20Z")
Hi Team- I am really new to elk and looking for the help. i am trying importing data from csv to elasticsearch using logstash but that is getting failed. Below is my CSV file. \`\`\` Sl.No InstanceName InstanceID … --- ## [Elasticsearch nodes continually disconneting/reconnecting. Resulting in very high number of unassigned shards](https://discuss.elastic.co/t/elasticsearch-nodes-continually-disconneting-reconnecting-resulting-in-very-high-number-of-unassigned-shards/243784)
**Author:** [@anda](https://discuss.elastic.co/u/anda)\ **Replies:** 17\ **Last updated:** [August 6, 2020, 12:59pm UTC](https://discuss.elastic.co/t/elasticsearch-nodes-continually-disconneting-reconnecting-resulting-in-very-high-number-of-unassigned-shards/243784 "2020-08-06T12:59:43Z")
I'm running an es cluster (7.4) with 3 master, 3 data, and 1 coord node. After running smoothly for months, nodes began to continually disconnect/reconnect from the cluster. From observing, it seems like 1 particular ma… --- ## [Further optimization to ES queries / performance](https://discuss.elastic.co/t/further-optimization-to-es-queries-performance/244020)
**Author:** [@jyridevelopingsw](https://discuss.elastic.co/u/jyridevelopingsw)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 12:25pm UTC](https://discuss.elastic.co/t/further-optimization-to-es-queries-performance/244020 "2020-08-06T12:25:12Z")
We have been very pleased with ES performance, but wondering is there a way to increase search performance even further? When running performance tests against ES cluster, the cluster is not the bottleneck and has enough… --- ## [How to use value from elasticsearch index as parameter in ES SQL functions, like DATE\_DIFF()](https://discuss.elastic.co/t/how-to-use-value-from-elasticsearch-index-as-parameter-in-es-sql-functions-like-date-diff/242365)
**Author:** [@burakka](https://discuss.elastic.co/u/burakka)\ **Replies:** 1\ **Last updated:** [July 27, 2020, 7:59am UTC](https://discuss.elastic.co/t/how-to-use-value-from-elasticsearch-index-as-parameter-in-es-sql-functions-like-date-diff/242365 "2020-07-27T07:59:26Z")
Hello guys! I want to use Elastic SQL with functions in Kibana Canvas. Therefore I tried to use the function DATE\_DIFF(), with values from an existing Elasticsearch index as parameter in the sql function. I looked at t… --- ## [Logstash 7.8 beats input with SSL throws error](https://discuss.elastic.co/t/logstash-7-8-beats-input-with-ssl-throws-error/243273)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 3\ **Last updated:** [August 6, 2020, 11:55am UTC](https://discuss.elastic.co/t/logstash-7-8-beats-input-with-ssl-throws-error/243273 "2020-08-06T11:55:59Z")
I added the SSL to the beats input and now the pipline throws an error. Cert and key are verified. See below 1) Config, 2) Cert and Key verify and 3) error in pipeline log. input { beats { port =\> 5046 ssl =\>… --- ## [Add custom fields, attributes to certificates](https://discuss.elastic.co/t/add-custom-fields-attributes-to-certificates/244004)
**Author:** [@Kosodrom](https://discuss.elastic.co/u/Kosodrom)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 10:33am UTC](https://discuss.elastic.co/t/add-custom-fields-attributes-to-certificates/244004 "2020-08-06T10:33:00Z")
Hi, is it possible to add custom fields to certificates during the creation using elasticsearch certutil https://www.elastic.co/guide/en/elasticsearch/reference/7.8/certutil.html? My Use case: I want to create new clie… --- ## [Filtering different visualizations in the same dashboard differently](https://discuss.elastic.co/t/filtering-different-visualizations-in-the-same-dashboard-differently/239477)
**Author:** [@Reuven\_Weiser](https://discuss.elastic.co/u/Reuven_Weiser)\ **Replies:** 3\ **Last updated:** [August 6, 2020, 10:20am UTC](https://discuss.elastic.co/t/filtering-different-visualizations-in-the-same-dashboard-differently/239477 "2020-08-06T10:20:40Z")
Assume I have an index of employees, each with a sub-object representing their manager: { "document-id": "XYZ", "firstName": "John", "lastName": "Smith", "office": "midtown", "manager": { "document-id": "ABC", … --- ## [Aggregate on string prefix extract value from suffix](https://discuss.elastic.co/t/aggregate-on-string-prefix-extract-value-from-suffix/243984)
**Author:** [@Yonatan\_Omer](https://discuss.elastic.co/u/Yonatan_Omer)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 9:01am UTC](https://discuss.elastic.co/t/aggregate-on-string-prefix-extract-value-from-suffix/243984 "2020-08-06T09:01:17Z")
In our case we have unknown number of metric types. In order to avoid nested types due to the known problem Locene flattening the objects, and avoid mapping explosion, I'm considering using (prefixed) flat strings and d… --- ## [Kibana discover display bug?](https://discuss.elastic.co/t/kibana-discover-display-bug/243731)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 3\ **Last updated:** [August 5, 2020, 2:33pm UTC](https://discuss.elastic.co/t/kibana-discover-display-bug/243731 "2020-08-05T14:33:20Z")
Hi, On 2 servers in 2 seperate clusters I have installed Kibana 7.7 . The thing is, when I start and go to 'discover, all is ok (except for an error message I have a seperate topic for). After initial start I am greete… --- ## [Metricbeat IIS Module - There is more data to return than will fit in the supplied buffer](https://discuss.elastic.co/t/metricbeat-iis-module-there-is-more-data-to-return-than-will-fit-in-the-supplied-buffer/240442)
**Author:** [@kagoadvs](https://discuss.elastic.co/u/kagoadvs)\ **Replies:** 4\ **Last updated:** [August 6, 2020, 9:52am UTC](https://discuss.elastic.co/t/metricbeat-iis-module-there-is-more-data-to-return-than-will-fit-in-the-supplied-buffer/240442 "2020-08-06T09:52:24Z")
After enabling the IIS Module in Metricbeat 7.8 (Windows Server 2019 Standard), I am continuously getting errors in the Metricbeat log file: |2020-07-08T20:35:20.831-0700|ERROR|\[website\]|application\_pool/reader.go:94|Th… --- ## [Filebeat Sincedb may not cleanup complete](https://discuss.elastic.co/t/filebeat-sincedb-may-not-cleanup-complete/243989)
**Author:** [@HuanFeng](https://discuss.elastic.co/u/HuanFeng)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 9:23am UTC](https://discuss.elastic.co/t/filebeat-sincedb-may-not-cleanup-complete/243989 "2020-08-06T09:23:50Z")
Hi, everyone I'm testing filebeat for our project, and I found that the sincedb is not completly cleanup. the below step is the mock situation for our project, and the 3-4 steps is moking the filebeat was down and conf… --- ## [Pipeline aggregation in Composite Aggregation](https://discuss.elastic.co/t/pipeline-aggregation-in-composite-aggregation/243125)
**Author:** [@shreedhan](https://discuss.elastic.co/u/shreedhan)\ **Replies:** 4\ **Last updated:** [August 6, 2020, 9:12am UTC](https://discuss.elastic.co/t/pipeline-aggregation-in-composite-aggregation/243125 "2020-08-06T09:12:42Z")
I'm trying to get Pipeline Aggregation for Composite Aggregation similar to this post. I'm getting similar error in version 6.8. Looks like it was a bug in version 6.2. I'm wondering if this has been fixed in version 6.8… --- ## [Elastic performance on node 32GB RAM vs 64GB RAM](https://discuss.elastic.co/t/elastic-performance-on-node-32gb-ram-vs-64gb-ram/243986)
**Author:** [@Petr.Simik](https://discuss.elastic.co/u/Petr.Simik)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 9:04am UTC](https://discuss.elastic.co/t/elastic-performance-on-node-32gb-ram-vs-64gb-ram/243986 "2020-08-06T09:04:38Z")
We have cluster in private cloud with node configuration (8vCPU, 64GB RAM, 1TB disk SSD/VSAN) in 13 data nodes + 3 master nodes. Due to better utilization of our cloud resources, our IT team asked me to reconfigure node… --- ## [Is it possible to implement this kind of friend search?](https://discuss.elastic.co/t/is-it-possible-to-implement-this-kind-of-friend-search/243985)
**Author:** [@demonatic](https://discuss.elastic.co/u/demonatic)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 9:01am UTC](https://discuss.elastic.co/t/is-it-possible-to-implement-this-kind-of-friend-search/243985 "2020-08-06T09:01:40Z")
The basic idea is simple, we have some external kv pairs imply friend id and corresponding intimacy value that can pass to elasticsearch. We want to query first and get some match documents with corresponding text releva… --- ## [TransportClient Async calls not invoking all action listeners under heavy load in cross-cluster setup](https://discuss.elastic.co/t/transportclient-async-calls-not-invoking-all-action-listeners-under-heavy-load-in-cross-cluster-setup/243983)
**Author:** [@Roman\_Margolis](https://discuss.elastic.co/u/Roman_Margolis)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 8:59am UTC](https://discuss.elastic.co/t/transportclient-async-calls-not-invoking-all-action-listeners-under-heavy-load-in-cross-cluster-setup/243983 "2020-08-06T08:59:28Z")
Hi, I managed to reproduce a scenario where using Transport Client 6.2.4 to send about 100 heavy search requests (heavy aggregation searches that hit many shards on the targeted cluster) with action listeners (every s… --- ## [How to split a field value into separated fields in elasticsearch](https://discuss.elastic.co/t/how-to-split-a-field-value-into-separated-fields-in-elasticsearch/240938)
**Author:** [@Hung\_M\_Le](https://discuss.elastic.co/u/Hung_M_Le)\ **Replies:** 6\ **Last updated:** [August 6, 2020, 8:51am UTC](https://discuss.elastic.co/t/how-to-split-a-field-value-into-separated-fields-in-elasticsearch/240938 "2020-08-06T08:51:22Z")
Hi I have the following issue that I hope to get some help to resolve background: . I ingest a log file using filebeat . I defined inside elasticsearch grok and kv statements to split incoming data into separated fi… --- ## [Kibana Markdown Visualisation](https://discuss.elastic.co/t/kibana-markdown-visualisation/243901)
**Author:** [@david.preston](https://discuss.elastic.co/u/david.preston)\ **Replies:** 2\ **Last updated:** [August 6, 2020, 8:41am UTC](https://discuss.elastic.co/t/kibana-markdown-visualisation/243901 "2020-08-06T08:41:22Z")
Hi, Is there any way to add a data source to the markdown visualisation used in dashboard? I can see that a canvas markdown element can be configured with a data source - allowing for dynamic behaviour. However, I'm b… --- ## [Logstash output plugin not working without any error](https://discuss.elastic.co/t/logstash-output-plugin-not-working-without-any-error/243532)
**Author:** [@ksarkar](https://discuss.elastic.co/u/ksarkar)\ **Replies:** 2\ **Last updated:** [August 6, 2020, 8:28am UTC](https://discuss.elastic.co/t/logstash-output-plugin-not-working-without-any-error/243532 "2020-08-06T08:28:37Z")
I have confugured Logstash to feed Elastic search indexes. Here we have integrated with postgresql DB and reading data using timestamp field. There are multiple (10/12) pipeline has been configured to read latest data … --- ## [Can't send dashboard to kibana](https://discuss.elastic.co/t/cant-send-dashboard-to-kibana/243887)
**Author:** [@headtea](https://discuss.elastic.co/u/headtea)\ **Replies:** 4\ **Last updated:** [August 6, 2020, 8:21am UTC](https://discuss.elastic.co/t/cant-send-dashboard-to-kibana/243887 "2020-08-06T08:21:51Z")
Because we like the default "failed logins" dashboard that auditbeat provides, in auditbeat.yml, we have setup.dashboards.enabled: true. It worked fine and now that I've enabled HTTPS for kibana I'm facing some issues. … --- ## [Debugging ingest pipelines](https://discuss.elastic.co/t/debugging-ingest-pipelines/243977)
**Author:** [@trigger-unhappy](https://discuss.elastic.co/u/trigger-unhappy)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 8:16am UTC](https://discuss.elastic.co/t/debugging-ingest-pipelines/243977 "2020-08-06T08:16:38Z")
Since I'm using custom apache log format and had issues with apache module extracting the correct source ip I wrote my own ingest pipeline for apache access logs and disabled the module. That part works fine, but now I a… --- ## [Counting the duplicates and non-duplicates of a count aggregation](https://discuss.elastic.co/t/counting-the-duplicates-and-non-duplicates-of-a-count-aggregation/243916)
**Author:** [@benji87](https://discuss.elastic.co/u/benji87)\ **Replies:** 2\ **Last updated:** [August 6, 2020, 8:08am UTC](https://discuss.elastic.co/t/counting-the-duplicates-and-non-duplicates-of-a-count-aggregation/243916 "2020-08-06T08:08:58Z")
Hello all, I'm hoping someone can help me with my issue, as I feel it's a simple problem to solve and it's just me missing something! I'm fairly new to ELK (3 months in), so hopefully this isn't too much of a newbie qu… --- ## [Parsing Nested JSON with Array](https://discuss.elastic.co/t/parsing-nested-json-with-array/243794)
**Author:** [@nikhilsuryan](https://discuss.elastic.co/u/nikhilsuryan)\ **Replies:** 16\ **Last updated:** [August 6, 2020, 8:05am UTC](https://discuss.elastic.co/t/parsing-nested-json-with-array/243794 "2020-08-06T08:05:50Z")
Hi Team, Can anyone help me in parsing the below and store the same in elastic Search. I am new to Logstash. Wanted to store the metric in Elastic search. { "collectionTime": { "timeStamp": "2020-08-04T17:27:16Z", … --- ## [Tuning search to speed up shingle analyzer response time from 150ms to 30 ms](https://discuss.elastic.co/t/tuning-search-to-speed-up-shingle-analyzer-response-time-from-150ms-to-30-ms/243975)
**Author:** [@vivekatwal](https://discuss.elastic.co/u/vivekatwal)\ **Replies:** 0\ **Last updated:** [August 6, 2020, 8:03am UTC](https://discuss.elastic.co/t/tuning-search-to-speed-up-shingle-analyzer-response-time-from-150ms-to-30-ms/243975 "2020-08-06T08:03:41Z")
Hi, I am using shingle as search analyzer RAM: 16GB DDR4 SSD: 256 GB Heap size: 4gb Index size is: 700mb Total documents in index : 4 million A simple match query is made GET index/\_search { "query":{ "matc… --- ## [Related ES 5.3 buffer memory issue](https://discuss.elastic.co/t/related-es-5-3-buffer-memory-issue/243355)
**Author:** [@Sourabh\_Korde](https://discuss.elastic.co/u/Sourabh_Korde)\ **Replies:** 2\ **Last updated:** [August 6, 2020, 7:39am UTC](https://discuss.elastic.co/t/related-es-5-3-buffer-memory-issue/243355 "2020-08-06T07:39:58Z")
I am using ES 5.3 version with high level REST client. When am requesting lengthy description its showing exception like \[ERROR|2020-07-21 11:37:23.971|VACS|37891724|VACS-27229170-20200721:113703-963|hsbc-search-servic… --- ## [Unable to login to Kibana after changing Space settings for specific role](https://discuss.elastic.co/t/unable-to-login-to-kibana-after-changing-space-settings-for-specific-role/239888)
**Author:** [@peternijem](https://discuss.elastic.co/u/peternijem)\ **Replies:** 18\ **Last updated:** [August 6, 2020, 7:25am UTC](https://discuss.elastic.co/t/unable-to-login-to-kibana-after-changing-space-settings-for-specific-role/239888 "2020-08-06T07:25:06Z")
Hi, I have created a user 'X' with role 'Y'. A space 'Z' is assinged to this role. In the settings of the Space for this role, I've decided to hide all features including 'dashboards' from it and keep only dev tools for… --- ## [Elastic Cloud availability zones](https://discuss.elastic.co/t/elastic-cloud-availability-zones/243859)
**Author:** [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Replies:** 3\ **Last updated:** [August 6, 2020, 7:24am UTC](https://discuss.elastic.co/t/elastic-cloud-availability-zones/243859 "2020-08-06T07:24:21Z")
Hello, In the cloud management when choosing , when choosing 2 availability zones, is the data replicated or splitted between data centers, but does it also mean, that both data centers are participating queries effort… --- ## [Csv reports from elasticsearch curl command](https://discuss.elastic.co/t/csv-reports-from-elasticsearch-curl-command/243965)
**Author:** [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\ **Replies:** 1\ **Last updated:** [August 6, 2020, 7:21am UTC](https://discuss.elastic.co/t/csv-reports-from-elasticsearch-curl-command/243965 "2020-08-06T07:21:08Z")
Hi All, I m trying to get curl command reports in csv format rather than json. Is there a way to achieve it. --- ## [Help with process cluster event timeout exception](https://discuss.elastic.co/t/help-with-process-cluster-event-timeout-exception/243959)
**Author:** [@Ayyappa\_P](https://discuss.elastic.co/u/Ayyappa_P)\ **Replies:** 1\ **Last updated:** [August 6, 2020, 7:20am UTC](https://discuss.elastic.co/t/help-with-process-cluster-event-timeout-exception/243959 "2020-08-06T07:20:48Z")
Hi There, I have an index which consists only 8 documents, but while inserting that, I have got the Cluster Event Exception, When I try it after few seconds, then it worked fine. This is a big problem for me. This is t… --- ## [Custom similarity without TF/IDF scoring](https://discuss.elastic.co/t/custom-similarity-without-tf-idf-scoring/243930)
**Author:** [@Alex\_D](https://discuss.elastic.co/u/Alex_D)\ **Replies:** 0\ **Last updated:** [August 5, 2020, 8:24pm UTC](https://discuss.elastic.co/t/custom-similarity-without-tf-idf-scoring/243930 "2020-08-05T20:24:21Z")
Hi all ! It isn't necessary to use TF / IDF scoring relevance for my full-text search task. I am trying to receive score in range 0-100 based only on doc.length variable. My formula: "similarity": { "custom\_similar… --- ## [ILM settings not being applied](https://discuss.elastic.co/t/ilm-settings-not-being-applied/243643)
**Author:** [@wardallen](https://discuss.elastic.co/u/wardallen)\ **Replies:** 4\ **Last updated:** [August 6, 2020, 6:55am UTC](https://discuss.elastic.co/t/ilm-settings-not-being-applied/243643 "2020-08-06T06:55:36Z")
I have a logstash index storing data that will be used in reporting, but only ever for the last 5 minutes. Hence, I want to have a very aggressive deletion policy. Roll the index every 15 minutes Delete the rolled (ol… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1882) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1884)