# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1698 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1699 --- ## [Logstash import CSV data classification](https://discuss.elastic.co/t/logstash-import-csv-data-classification/255362)
**Author:** [@stevezemlicka](https://discuss.elastic.co/u/stevezemlicka)\ **Replies:** 4\ **Last updated:** [November 13, 2020, 7:59pm UTC](https://discuss.elastic.co/t/logstash-import-csv-data-classification/255362 "2020-11-13T19:59:07Z")
First off, I am relatively inexperienced with ELK and am trying to perform a demo of ELK because I'm convinced it is a powerful and useful application for much of our work. I have been presented with a CSV export of dat… --- ## [Logstash not reading CSV file when running as CSV](https://discuss.elastic.co/t/logstash-not-reading-csv-file-when-running-as-csv/255374)
**Author:** [@errupeshmca](https://discuss.elastic.co/u/errupeshmca)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 7:46pm UTC](https://discuss.elastic.co/t/logstash-not-reading-csv-file-when-running-as-csv/255374 "2020-11-13T19:46:16Z")
Hi, I am trying to run ELK through service on windows, Kibana and Elasticsearch is working fine as service. I have configured service through nssm and used below reference to configure logstash Logstash starting p… --- ## [Loosing filter settings when changing the dashboard at v 7.9.2](https://discuss.elastic.co/t/loosing-filter-settings-when-changing-the-dashboard-at-v-7-9-2/254968)
**Author:** [@mayer](https://discuss.elastic.co/u/mayer)\ **Replies:** 4\ **Last updated:** [November 13, 2020, 7:44pm UTC](https://discuss.elastic.co/t/loosing-filter-settings-when-changing-the-dashboard-at-v-7-9-2/254968 "2020-11-13T19:44:54Z")
Dear All, I am using ELK stack version 7.9.2 on Debian buster (amd64) I build 2 dashboards with a visualization where I can navigate between these 2 dashboards. I enter a filter with the "+ add filter" button in one of… --- ## [Parser error when logstash reading updated CSV file](https://discuss.elastic.co/t/parser-error-when-logstash-reading-updated-csv-file/254813)
**Author:** [@errupeshmca](https://discuss.elastic.co/u/errupeshmca)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 7:19pm UTC](https://discuss.elastic.co/t/parser-error-when-logstash-reading-updated-csv-file/254813 "2020-11-13T19:19:58Z")
Hi, Logstash giving parser error whenever CSV file gets updated with new records. However logstash works fine when you delete the old csv file and replace with new updated CSV file, but in that scenerio logstash all rec… --- ## [Encrypt data volume for Elasticsearch service on Azure](https://discuss.elastic.co/t/encrypt-data-volume-for-elasticsearch-service-on-azure/254980)
**Author:** [@anshuman](https://discuss.elastic.co/u/anshuman)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 7:10pm UTC](https://discuss.elastic.co/t/encrypt-data-volume-for-elasticsearch-service-on-azure/254980 "2020-11-13T19:10:33Z")
Do we have an option to encrypt the Elasticsearch data (or volume) which is offered as platform as a service on Azure using our own encryption key? --- ## [Add custom visualization to Lens](https://discuss.elastic.co/t/add-custom-visualization-to-lens/255329)
**Author:** [@pchakour](https://discuss.elastic.co/u/pchakour)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 6:27pm UTC](https://discuss.elastic.co/t/add-custom-visualization-to-lens/255329 "2020-11-13T18:27:37Z")
Hello everyone :slight\_smile: I hope Elastic members safe with the Covid ! I just want to know if an API exists to add a custom visualization to Lens ? If not, did you plan something related to that ? --- ## [Elastic Query Regex](https://discuss.elastic.co/t/elastic-query-regex/254973)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 5:20pm UTC](https://discuss.elastic.co/t/elastic-query-regex/254973 "2020-11-13T17:20:33Z")
I am trying to write a regex query but it is not working accordingly am i doing something wrong I need to get back all values that ends in SVC in the field winlog.event\_data.TargetUserName { "query": { "regexp": { "… --- ## [Remove results with same id from Elasticsearch search result](https://discuss.elastic.co/t/remove-results-with-same-id-from-elasticsearch-search-result/255365)
**Author:** [@philippd](https://discuss.elastic.co/u/philippd)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 4:37pm UTC](https://discuss.elastic.co/t/remove-results-with-same-id-from-elasticsearch-search-result/255365 "2020-11-13T16:37:43Z")
Hi, I just joined the forums and would like to aks you for a little help with the following question I am not able to answer reading the docs: Let's assume we have a search result with 3 documents. Two of them share a … --- ## [How to check number of IPs being resolved by the Elasticsearch client?](https://discuss.elastic.co/t/how-to-check-number-of-ips-being-resolved-by-the-elasticsearch-client/255318)
**Author:** [@prasad98](https://discuss.elastic.co/u/prasad98)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 3:53pm UTC](https://discuss.elastic.co/t/how-to-check-number-of-ips-being-resolved-by-the-elasticsearch-client/255318 "2020-11-13T15:53:38Z")
I've some microservices which are dependent on the Elasticsearch cluster in production. This cluster is common for every microservice. Just to verify how many services IP is being resolved by Elasticsearch cluster, is t… --- ## [Different modules?](https://discuss.elastic.co/t/different-modules/255345)
**Author:** [@MichaelJ](https://discuss.elastic.co/u/MichaelJ)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 2:48pm UTC](https://discuss.elastic.co/t/different-modules/255345 "2020-11-13T14:48:15Z")
We're using filebeat to ingest all kinds of logfiles: application logs, apache, nginx, whatever. For specifix log files, like apache, we want to parse the message into different fields. The apache module with its ingest… --- ## [Unable to post the client's cert to Webapi](https://discuss.elastic.co/t/unable-to-post-the-clients-cert-to-webapi/255076)
**Author:** [@sdr](https://discuss.elastic.co/u/sdr)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 2:31pm UTC](https://discuss.elastic.co/t/unable-to-post-the-clients-cert-to-webapi/255076 "2020-11-13T14:31:40Z")
I'm using elk version 6.7.0. Issue with sending the client certificate using logstash output plugin to webapi. --- ## [How to manage long length text in data table viz?](https://discuss.elastic.co/t/how-to-manage-long-length-text-in-data-table-viz/255339)
**Author:** [@vidhyadhara](https://discuss.elastic.co/u/vidhyadhara)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 2:31pm UTC](https://discuss.elastic.co/t/how-to-manage-long-length-text-in-data-table-viz/255339 "2020-11-13T14:31:28Z")
Hello everyone, Below is the screenshot of a data table viz. here I have a lengthy column (story.keyword), I don't want to display so much of text. Is there any way to wrap it up and display only when needed ?? … --- ## [Beats Internal monitoring](https://discuss.elastic.co/t/beats-internal-monitoring/255350)
**Author:** [@MR\_551](https://discuss.elastic.co/u/MR_551)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 2:25pm UTC](https://discuss.elastic.co/t/beats-internal-monitoring/255350 "2020-11-13T14:25:03Z")
Hello everyone, Has anyone experienced an issue with internal monitoring for Beats? When i ship internal metrics for each Beat to a monitoring cluster, they get grouped under the same Beat on Stack Monitoring tab. I am … --- ## [Make visualizations (metric and table/topn) with weighted average](https://discuss.elastic.co/t/make-visualizations-metric-and-table-topn-with-weighted-average/255316)
**Author:** [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 1:48pm UTC](https://discuss.elastic.co/t/make-visualizations-metric-and-table-topn-with-weighted-average/255316 "2020-11-13T13:48:52Z")
Hi there, I'm struggling trying to visualize a Metric visualization with a given weighted average over a certain timespan. Let's consider the following scenario: index: sample\_shops\_data fields of interest: day (date)… --- ## [Optimize bool filter query](https://discuss.elastic.co/t/optimize-bool-filter-query/255348)
**Author:** [@Daniele\_Renda](https://discuss.elastic.co/u/Daniele_Renda)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 1:45pm UTC](https://discuss.elastic.co/t/optimize-bool-filter-query/255348 "2020-11-13T13:45:39Z")
Hi, I'm using ES 7.10 deployed in Elastic Cloud. I developed a Java application that use High Level REST client. I want to make a query like this: index \>=1.67 && index\<=1.67 AND (sphereMin\<=-7 && sphereMax \>=-7 && cyl… --- ## [Filebeat logs not making it to Logstash](https://discuss.elastic.co/t/filebeat-logs-not-making-it-to-logstash/252941)
**Author:** [@allieg](https://discuss.elastic.co/u/allieg)\ **Replies:** 4\ **Last updated:** [November 13, 2020, 1:43pm UTC](https://discuss.elastic.co/t/filebeat-logs-not-making-it-to-logstash/252941 "2020-11-13T13:43:37Z")
Hi, for some reason remote server that has Filebeat setup on it is not being able to successfully send logs to my elk server that has logstash running on it. I ran filebeat -e -d "publish,logstash and got the following w… --- ## [Auditbeat 7.8.0 stuck: process running but no data sent to Elasticsearch](https://discuss.elastic.co/t/auditbeat-7-8-0-stuck-process-running-but-no-data-sent-to-elasticsearch/255323)
**Author:** [@sonickenbaker](https://discuss.elastic.co/u/sonickenbaker)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 10:22am UTC](https://discuss.elastic.co/t/auditbeat-7-8-0-stuck-process-running-but-no-data-sent-to-elasticsearch/255323 "2020-11-13T10:22:19Z")
Hello, I have auditbeat v7.8.0 installed on a Ubuntu 18.04.5 LTS VM that is currently in a weird state. Process is up and running but no data is sent to Elasticsearch. According to systemd the process is fine (and it i… --- ## [How to get entire enriched "event" size (not message size)](https://discuss.elastic.co/t/how-to-get-entire-enriched-event-size-not-message-size/255184)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 10:05am UTC](https://discuss.elastic.co/t/how-to-get-entire-enriched-event-size-not-message-size/255184 "2020-11-13T10:05:53Z")
hi As per the link below, we got the size of the "message" field But after doing transformation, we need to have the size of the final event also. How to get it? I tried event.set('event\_size', event.get('\_source').… --- ## [How to schedule logstash pipelibe to consume kafka input logs every 15min](https://discuss.elastic.co/t/how-to-schedule-logstash-pipelibe-to-consume-kafka-input-logs-every-15min/255320)
**Author:** [@soufian.eldouqe](https://discuss.elastic.co/u/soufian.eldouqe)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 10:04am UTC](https://discuss.elastic.co/t/how-to-schedule-logstash-pipelibe-to-consume-kafka-input-logs-every-15min/255320 "2020-11-13T10:04:13Z")
Hello, I want to developp a logstash pipeline that consume heathbeat logs from filebeat that are generated every 5 minutes. the goal is to be sure that everything is working fine from my filebeat inputs to kibana. how … --- ## [How to sort aggregated based on sub aggregation's doc\_count?](https://discuss.elastic.co/t/how-to-sort-aggregated-based-on-sub-aggregations-doc-count/255195)
**Author:** [@lessless](https://discuss.elastic.co/u/lessless)\ **Replies:** 6\ **Last updated:** [November 13, 2020, 9:41am UTC](https://discuss.elastic.co/t/how-to-sort-aggregated-based-on-sub-aggregations-doc-count/255195 "2020-11-13T09:41:22Z")
Hi, I'm completely stuck trying to figure out how to sort "users" based on "status\_codes" in the following query: { "size": 0, "aggs": { "users": { "terms": { "field": "user\_id" }, … --- ## [Apply scoring after filter](https://discuss.elastic.co/t/apply-scoring-after-filter/255309)
**Author:** [@Jalendra\_Bhanarkar](https://discuss.elastic.co/u/Jalendra_Bhanarkar)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 9:28am UTC](https://discuss.elastic.co/t/apply-scoring-after-filter/255309 "2020-11-13T09:28:45Z")
I want the score to be calculated after the filter is applied. Right now, the score gets calculated considering all the documents in the index. And other entity documents interfere with the score calculation which I don'… --- ## [How to create aging column for number of days wanted](https://discuss.elastic.co/t/how-to-create-aging-column-for-number-of-days-wanted/254556)
**Author:** [@khairul\_faiz](https://discuss.elastic.co/u/khairul_faiz)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 8:52am UTC](https://discuss.elastic.co/t/how-to-create-aging-column-for-number-of-days-wanted/254556 "2020-11-13T08:52:20Z")
hi can anyone share their experience in creating aging column? the process is i need to calculate aging of days that a payment made between number of days invoice date - invoice due date thank you for sharing --- ## [java.lang.NoClassDefFoundError: org/elasticsearch/script/mustache/SearchTemplateRequestBuilder](https://discuss.elastic.co/t/java-lang-noclassdeffounderror-org-elasticsearch-script-mustache-searchtemplaterequestbuilder/255306)
**Author:** [@rakeshdkaatkar](https://discuss.elastic.co/u/rakeshdkaatkar)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 8:44am UTC](https://discuss.elastic.co/t/java-lang-noclassdeffounderror-org-elasticsearch-script-mustache-searchtemplaterequestbuilder/255306 "2020-11-13T08:44:43Z")
\[2020-11-13T14:09:32,460\]\[WARN \]\[o.e.h.AbstractHttpServerTransport\] \[L-356004158\] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/0:0:0:0:0:0:0:1:9200, remoteAddres… --- ## [Query - Regexp with special characters fails when used within multiple must\_not](https://discuss.elastic.co/t/query-regexp-with-special-characters-fails-when-used-within-multiple-must-not/255147)
**Author:** [@gonzalo2kx](https://discuss.elastic.co/u/gonzalo2kx)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 8:23am UTC](https://discuss.elastic.co/t/query-regexp-with-special-characters-fails-when-used-within-multiple-must-not/255147 "2020-11-13T08:23:49Z")
Good day, I am having issues when using regexp clause in my query for identifying and filtering out log messages produced by the filebeat, packetbeat, metricbeat, and auditbeat processes which contain the patterns below… --- ## [How to express SQL in Logstash](https://discuss.elastic.co/t/how-to-express-sql-in-logstash/255049)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 8:08am UTC](https://discuss.elastic.co/t/how-to-express-sql-in-logstash/255049 "2020-11-13T08:08:13Z")
Hello Team I am using Logstash and Elasticsearch version 7.8.0. I want to write query in logstash for input plugin of elasticsearch as select userId from myindex I am unable to achieve same in logstash ... --- ## [Logstash or straight?](https://discuss.elastic.co/t/logstash-or-straight/253722)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 7:57am UTC](https://discuss.elastic.co/t/logstash-or-straight/253722 "2020-11-13T07:57:51Z")
Hi, Simpel question, What would be the (dis)advantage of sending metrics via logstash to ES instead of directly? Cheers, --- ## [Kibana Heatmap: Use discrete values or labels in legend](https://discuss.elastic.co/t/kibana-heatmap-use-discrete-values-or-labels-in-legend/254548)
**Author:** [@FloJe](https://discuss.elastic.co/u/FloJe)\ **Replies:** 4\ **Last updated:** [November 13, 2020, 7:52am UTC](https://discuss.elastic.co/t/kibana-heatmap-use-discrete-values-or-labels-in-legend/254548 "2020-11-13T07:52:27Z")
Dear Forum, I have a question regarding Kibana Heatmap visualisation. I'm collecting logs from an application and these logs have an Apachee syslog level, i.e. error, warning, info. Now I want to visualize 10 minute bi… --- ## [Filebeat stop to send logs to Elastic after cluster\_block\_exception occurs](https://discuss.elastic.co/t/filebeat-stop-to-send-logs-to-elastic-after-cluster-block-exception-occurs/255301)
**Author:** [@mattia\_galati](https://discuss.elastic.co/u/mattia_galati)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 7:41am UTC](https://discuss.elastic.co/t/filebeat-stop-to-send-logs-to-elastic-after-cluster-block-exception-occurs/255301 "2020-11-13T07:41:20Z")
I'm containerizing a monolith application and in each current VM there is a Logstash process which harvests the logs that are being rotating by the application itself, through the python's RotatingFileHandler. In the cu… --- ## [Can't change port for logstash](https://discuss.elastic.co/t/cant-change-port-for-logstash/255299)
**Author:** [@Hi\_u\_Nguy\_n\_Dang](https://discuss.elastic.co/u/Hi_u_Nguy_n_Dang)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 7:18am UTC](https://discuss.elastic.co/t/cant-change-port-for-logstash/255299 "2020-11-13T07:18:33Z")
How can I change from port 5044 to 4000 for filebeat/logstash guys, it show this error Tks in advance --- ## [Kubernetes monitoring using metricbeat and ELK](https://discuss.elastic.co/t/kubernetes-monitoring-using-metricbeat-and-elk/255297)
**Author:** [@ruthvicsakre](https://discuss.elastic.co/u/ruthvicsakre)\ **Replies:** 0\ **Last updated:** [November 13, 2020, 7:06am UTC](https://discuss.elastic.co/t/kubernetes-monitoring-using-metricbeat-and-elk/255297 "2020-11-13T07:06:18Z")
Hi, We are using metricbeat to monitor kubernetes which is on AKS. We have installed Kube-state-metrics which is prerequisite for metricbeat and also metricbeat using the values.yaml present in below. But, in the default… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1697) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1699)