# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1911 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1912 --- ## [Server disk broken and recover, but failed to start](https://discuss.elastic.co/t/server-disk-broken-and-recover-but-failed-to-start/242140)
**Author:** [@jinjinbari](https://discuss.elastic.co/u/jinjinbari)\ **Replies:** 6\ **Last updated:** [July 23, 2020, 6:52am UTC](https://discuss.elastic.co/t/server-disk-broken-and-recover-but-failed-to-start/242140 "2020-07-23T06:52:06Z")
Hello. My server disk was broken and recover now. But the elasticsearch service isn\`t started. Here is my logs. \[2020-07-22T15:54:22,153\]\[INFO \]\[o.e.e.NodeEnvironment \] \[apm-data-node4\] using \[1\] data paths, mount… --- ## [Used heap behaves wired, JVM Heapsize problems](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871)
**Author:** [@defalt](https://discuss.elastic.co/u/defalt)\ **Replies:** 9\ **Last updated:** [July 23, 2020, 6:30am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871 "2020-07-23T06:30:21Z")
Hi, my used JVM Heap behaves very weird. I didn't change any settings. 2 Nodes work fine, the third has problems. (Orange Line) Settings: -Xms3g -Xmx3g The PC has a lot of free RAM left. What is happening he… --- ## [Issue with upsert document](https://discuss.elastic.co/t/issue-with-upsert-document/242279)
**Author:** [@siva4](https://discuss.elastic.co/u/siva4)\ **Replies:** 1\ **Last updated:** [July 23, 2020, 6:19am UTC](https://discuss.elastic.co/t/issue-with-upsert-document/242279 "2020-07-23T06:19:39Z")
What would be the correct api to perform upsert (insert if doc not exist, else update) in elasticsearch? ran into issues with below api. test.setId("test\_id"); test.setTitle("test\_title"); Stri… --- ## [How to load tables from mysql to elasticsearch](https://discuss.elastic.co/t/how-to-load-tables-from-mysql-to-elasticsearch/242275)
**Author:** [@Krishna\_Sai\_Nag\_G](https://discuss.elastic.co/u/Krishna_Sai_Nag_G)\ **Replies:** 5\ **Last updated:** [July 23, 2020, 6:14am UTC](https://discuss.elastic.co/t/how-to-load-tables-from-mysql-to-elasticsearch/242275 "2020-07-23T06:14:24Z")
Hi, I want load multiple tables from mysql to different indexes in elastic search(i.e; each table in mysql to different index in elastic search), Is it possible to load using single logstash config file. Anyone can hel… --- ## [Metricbeat for IBM MQ](https://discuss.elastic.co/t/metricbeat-for-ibm-mq/242276)
**Author:** [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\ **Replies:** 2\ **Last updated:** [July 23, 2020, 5:33am UTC](https://discuss.elastic.co/t/metricbeat-for-ibm-mq/242276 "2020-07-23T05:33:52Z")
Hi All, I am trying to use metricbeat for IBM MQ but I am getting below error No data has been received from this module yet Can someone please suggest what changes , do I need to make in below file 'modules.d/ibmmq.… --- ## [Is there any way to implement Discover as a Custom Plugin?](https://discuss.elastic.co/t/is-there-any-way-to-implement-discover-as-a-custom-plugin/242190)
**Author:** [@Akhil\_Kaithoju](https://discuss.elastic.co/u/Akhil_Kaithoju)\ **Replies:** 2\ **Last updated:** [July 23, 2020, 5:20am UTC](https://discuss.elastic.co/t/is-there-any-way-to-implement-discover-as-a-custom-plugin/242190 "2020-07-23T05:20:12Z")
Hi, I want to create an Discover Plugin as Custom Plugin for separate Modifications. Can you please share your thoughts here. Thanks --- ## [Filebeat Data transfer speed and size](https://discuss.elastic.co/t/filebeat-data-transfer-speed-and-size/242274)
**Author:** [@gangireddy\_l](https://discuss.elastic.co/u/gangireddy_l)\ **Replies:** 0\ **Last updated:** [July 23, 2020, 4:59am UTC](https://discuss.elastic.co/t/filebeat-data-transfer-speed-and-size/242274 "2020-07-23T04:59:59Z")
Hi Team How to find file-beat data transfer speed and data sent per hour or day..? Regards, Gangi Reddy --- ## [Is there any way to count two similar elements together in kibana visualization?](https://discuss.elastic.co/t/is-there-any-way-to-count-two-similar-elements-together-in-kibana-visualization/242118)
**Author:** [@myohyun](https://discuss.elastic.co/u/myohyun)\ **Replies:** 3\ **Last updated:** [July 23, 2020, 1:48am UTC](https://discuss.elastic.co/t/is-there-any-way-to-count-two-similar-elements-together-in-kibana-visualization/242118 "2020-07-23T01:48:04Z")
Hi, I'm curious if similar elements in the field are counted together in kibana visualization, like: In the field "request" there are two kinds of values - "discuss.elastic.co/c/elastic-stack/kibana/7" (count: 10) - … --- ## [Little confuse about registry file](https://discuss.elastic.co/t/little-confuse-about-registry-file/242031)
**Author:** [@Mrzhuang007](https://discuss.elastic.co/u/Mrzhuang007)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 1:07pm UTC](https://discuss.elastic.co/t/little-confuse-about-registry-file/242031 "2020-07-21T13:07:35Z")
I add some config like below filebeat.config: inputs: enabled: true path: /etc/filebeat/filebeat-2/\*.yml reload.enabled: true reload.period: 1s It enable me to add runtime yml configs. But when I remo… --- ## [Integer comparison not working](https://discuss.elastic.co/t/integer-comparison-not-working/242248)
**Author:** [@opoplawski](https://discuss.elastic.co/u/opoplawski)\ **Replies:** 3\ **Last updated:** [July 23, 2020, 3:35am UTC](https://discuss.elastic.co/t/integer-comparison-not-working/242248 "2020-07-23T03:35:39Z")
I'm trying to conditionally set some fields in my pipeline: filter { if \[log\]\[file\]\[path\] == "/var/log/e2guardian/access.log" { grok { match =\> { "message" =\> "(?\\\d\\d\\d\\d\\.\\d\\d\\.\\d\\d \\d\\d:\\d\\d:… --- ## [Filebeate can not remove the state of log in registry file](https://discuss.elastic.co/t/filebeate-can-not-remove-the-state-of-log-in-registry-file/242268)
**Author:** [@Mrzhuang007](https://discuss.elastic.co/u/Mrzhuang007)\ **Replies:** 0\ **Last updated:** [July 23, 2020, 3:23am UTC](https://discuss.elastic.co/t/filebeate-can-not-remove-the-state-of-log-in-registry-file/242268 "2020-07-23T03:23:20Z")
The Filebeat Version I used is 6.8.6. Running in Linux. The filebeat.yml I used is as fllow: #========================= Filebeat global options ============================ filebeat.config: inputs: enabled: true … --- ## [We are having performance issue in Elasticsearch/Kibana after migrating to another server host](https://discuss.elastic.co/t/we-are-having-performance-issue-in-elasticsearch-kibana-after-migrating-to-another-server-host/241771)
**Author:** [@\_kyllr](https://discuss.elastic.co/u/_kyllr)\ **Replies:** 7\ **Last updated:** [July 23, 2020, 2:49am UTC](https://discuss.elastic.co/t/we-are-having-performance-issue-in-elasticsearch-kibana-after-migrating-to-another-server-host/241771 "2020-07-23T02:49:29Z")
Hello, We are having issue in Kibana. We have AWS Proxy and Kibana 7.1 setup in the same server. After migrating both to another server host and make it live in the load balancer, we are facing request timeout issue wh… --- ## [Snapshot and archive old indices comparision](https://discuss.elastic.co/t/snapshot-and-archive-old-indices-comparision/242266)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 1\ **Last updated:** [July 23, 2020, 2:19am UTC](https://discuss.elastic.co/t/snapshot-and-archive-old-indices-comparision/242266 "2020-07-23T02:19:04Z")
Hi all I am using a script to take snapshot of an index and then using tar to compress them to save space. What i wanted to know that what compression is the best for this kind of data. gunzip, bzip2 or something else.… --- ## [PaloAlto ingest pipeline and geoip lookup](https://discuss.elastic.co/t/paloalto-ingest-pipeline-and-geoip-lookup/240869)
**Author:** [@rossw](https://discuss.elastic.co/u/rossw)\ **Replies:** 3\ **Last updated:** [July 23, 2020, 2:12am UTC](https://discuss.elastic.co/t/paloalto-ingest-pipeline-and-geoip-lookup/240869 "2020-07-23T02:12:57Z")
Hi there We are running Filebeat 7.8.0, and utilising the Palo Alto module to ingest firewalls using ECS. When I go to display the source or destination IP addresses on on a map, I get a failure and no data points. Ac… --- ## [I'm not able to see logs in \[Filebeat Logstash\] Logstash Logs ECS dashboard](https://discuss.elastic.co/t/im-not-able-to-see-logs-in-filebeat-logstash-logstash-logs-ecs-dashboard/241961)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 1\ **Last updated:** [July 23, 2020, 12:41am UTC](https://discuss.elastic.co/t/im-not-able-to-see-logs-in-filebeat-logstash-logstash-logs-ecs-dashboard/241961 "2020-07-23T00:41:08Z")
I installed both filebeat and logstash. i aslo create a pipeline in logstash which is input data from beats and output to elasticsearch. when i run pipeline the logs are succesfully index in logstash.index pattern cre… --- ## [Trying to create an index template to create a data stream](https://discuss.elastic.co/t/trying-to-create-an-index-template-to-create-a-data-stream/242089)
**Author:** [@randoran](https://discuss.elastic.co/u/randoran)\ **Replies:** 3\ **Last updated:** [July 22, 2020, 11:35pm UTC](https://discuss.elastic.co/t/trying-to-create-an-index-template-to-create-a-data-stream/242089 "2020-07-22T23:35:51Z")
I'm attempting to follow this tutorial https://www.elastic.co/guide/en/elasticsearch/reference/7.x/getting-started-index-lifecycle-management.html And I am at the point where it says to Create an index template PUT \_i… --- ## [Metricbeat capturing performance monitor terminal services](https://discuss.elastic.co/t/metricbeat-capturing-performance-monitor-terminal-services/242087)
**Author:** [@dklinkaz](https://discuss.elastic.co/u/dklinkaz)\ **Replies:** 0\ **Last updated:** [July 21, 2020, 6:58pm UTC](https://discuss.elastic.co/t/metricbeat-capturing-performance-monitor-terminal-services/242087 "2020-07-21T18:58:26Z")
I am trying to capture the WVD session host information, I am currently using the performance monitor terminal services to capture the information. I am having issues getting the windows module to properly run and send t… --- ## [Metricbeat from 6.5.2 to 7.8.0](https://discuss.elastic.co/t/metricbeat-from-6-5-2-to-7-8-0/242194)
**Author:** [@salvo](https://discuss.elastic.co/u/salvo)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 11:23pm UTC](https://discuss.elastic.co/t/metricbeat-from-6-5-2-to-7-8-0/242194 "2020-07-22T23:23:53Z")
I recently migrated metricbeat from 6.5.2 to 7.8.0 but for some reason, after I created the dashboard by running: metricbeat setup --dashboards and by navigating to the Kafka dashboard I am facing a strange error: Err… --- ## [Filebeat getting stopped automatically after sometime](https://discuss.elastic.co/t/filebeat-getting-stopped-automatically-after-sometime/242198)
**Author:** [@HariniGajendran](https://discuss.elastic.co/u/HariniGajendran)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 11:23pm UTC](https://discuss.elastic.co/t/filebeat-getting-stopped-automatically-after-sometime/242198 "2020-07-22T23:23:18Z")
Hi, I am using 7.4.2 version filebeat. Issue is my filebeat service gets stopped automatically after running for sometime. But whenever I start it will crawl properly and log data into elastic search properly for certai… --- ## [Cleanup: How to delete unused saved searches, visualizations and dashboards?](https://discuss.elastic.co/t/cleanup-how-to-delete-unused-saved-searches-visualizations-and-dashboards/242232)
**Author:** [@igorgatis](https://discuss.elastic.co/u/igorgatis)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 11:21pm UTC](https://discuss.elastic.co/t/cleanup-how-to-delete-unused-saved-searches-visualizations-and-dashboards/242232 "2020-07-22T23:21:36Z")
We've been using Kibana in production for a while now and we accumulated saved searches, visualizations and dashboards we're not used are actually being in use. Is there a way to figure out which saved objects are used … --- ## [Handling exception from HTTP logstash 6.8](https://discuss.elastic.co/t/handling-exception-from-http-logstash-6-8/242259)
**Author:** [@pvaithee07](https://discuss.elastic.co/u/pvaithee07)\ **Replies:** 4\ **Last updated:** [July 22, 2020, 11:18pm UTC](https://discuss.elastic.co/t/handling-exception-from-http-logstash-6-8/242259 "2020-07-22T23:18:17Z")
Hello - I am trying to handle exception in my logstash after HTTP filter plugin. I am using Logstash version 6.8. I have couple of question on this - How to handle exception if the HTTP response code is other than 200… --- ## [Elastic Rest-Client date format issues](https://discuss.elastic.co/t/elastic-rest-client-date-format-issues/242209)
**Author:** [@Dede\_Pessu](https://discuss.elastic.co/u/Dede_Pessu)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 10:51pm UTC](https://discuss.elastic.co/t/elastic-rest-client-date-format-issues/242209 "2020-07-22T22:51:39Z")
Sending data to elastic search via Rest client from a java application it it is complaining about the date format below is the stack trace that it being spit out. 1. ElasticsearchStatusException\[Elasticsearch excep… --- ## [Not able to get the CSV values in the filter columns](https://discuss.elastic.co/t/not-able-to-get-the-csv-values-in-the-filter-columns/242263)
**Author:** [@Harshil\_Narielwala](https://discuss.elastic.co/u/Harshil_Narielwala)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 10:40pm UTC](https://discuss.elastic.co/t/not-able-to-get-the-csv-values-in-the-filter-columns/242263 "2020-07-22T22:40:26Z")
My logstash configuration is as below : input { beats { port =\> "5044" } } filter { csv { columns =\> \["REQUEST\_ID","SERVICE\_NAME","OPERATION\_NAME","OPERATION\_TYPE","REQUEST\_PAYLOAD","LOGGED\_BY","LOGGED\_ON"\] } } … --- ## [Filebeats Getting Connection reset by peer error](https://discuss.elastic.co/t/filebeats-getting-connection-reset-by-peer-error/242250)
**Author:** [@deep\_sandhu](https://discuss.elastic.co/u/deep_sandhu)\ **Replies:** 0\ **Last updated:** [July 22, 2020, 8:58pm UTC](https://discuss.elastic.co/t/filebeats-getting-connection-reset-by-peer-error/242250 "2020-07-22T20:58:52Z")
I am trying to send logs from filebeat(from dev host) to logstash (running in kubernetes separate Node), but getting following error \[INFO \] 2020-07-22 20:42:19.309 \[defaultEventExecutorGroup-4-1\] BeatsHandler - \[local:… --- ## [Setting security for ElasticSearch 7.6.2](https://discuss.elastic.co/t/setting-security-for-elasticsearch-7-6-2/240834)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 2\ **Last updated:** [July 22, 2020, 8:51pm UTC](https://discuss.elastic.co/t/setting-security-for-elasticsearch-7-6-2/240834 "2020-07-22T20:51:02Z")
Hello. I am new to ElasticSearch. I need to make security settings for a elasticsearch cluster with 3 Master nodes and 5 data nodes. I was looking at the video under https://www.youtube.com/watch?v=nMh1HWWe6B4 This v… --- ## [Logstash config: Elastic search indices not being properly populated from multiple http\_poller inputs](https://discuss.elastic.co/t/logstash-config-elastic-search-indices-not-being-properly-populated-from-multiple-http-poller-inputs/241929)
**Author:** [@AnEpicK9](https://discuss.elastic.co/u/AnEpicK9)\ **Replies:** 8\ **Last updated:** [July 22, 2020, 7:25pm UTC](https://discuss.elastic.co/t/logstash-config-elastic-search-indices-not-being-properly-populated-from-multiple-http-poller-inputs/241929 "2020-07-22T19:25:07Z")
Currently, I have two http\_poller input blocks with a url for a live and backup instance of our app in my logstash config. What I'm then doing is I'm trying to output to two different elastic search indices: logstash\_pol… --- ## [Auto\_flush\_interval is invalid with Logstash 7.5.1](https://discuss.elastic.co/t/auto-flush-interval-is-invalid-with-logstash-7-5-1/242057)
**Author:** [@thanhlam](https://discuss.elastic.co/u/thanhlam)\ **Replies:** 16\ **Last updated:** [July 22, 2020, 7:09pm UTC](https://discuss.elastic.co/t/auto-flush-interval-is-invalid-with-logstash-7-5-1/242057 "2020-07-22T19:09:53Z")
Hello, I've been running Logstash 7.5.1 on IBM Power systems for a couple of months. But, the last time I restarted Logstash, it kept shutdown and restarting. So I run it with config.debug = true log.level: trace and … --- ## [Pipeline not discovering with correct paths](https://discuss.elastic.co/t/pipeline-not-discovering-with-correct-paths/242243)
**Author:** [@mattr](https://discuss.elastic.co/u/mattr)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 6:26pm UTC](https://discuss.elastic.co/t/pipeline-not-discovering-with-correct-paths/242243 "2020-07-22T18:26:35Z")
I am having trouble with the logstash pipeline. I've had the pipeline working with the exact same settings on a different server. The config works when I run it manually using the command line command. The discoverer d… --- ## [Status=403): {"type":"security\_exception","reason":"action \[indices:data/write/bulk\[s\]](https://discuss.elastic.co/t/status-403-type-security-exception-reason-action-indices-data-write-bulk-s/240299)
**Author:** [@mmk1995](https://discuss.elastic.co/u/mmk1995)\ **Replies:** 8\ **Last updated:** [July 22, 2020, 6:24pm UTC](https://discuss.elastic.co/t/status-403-type-security-exception-reason-action-indices-data-write-bulk-s/240299 "2020-07-22T18:24:34Z")
Hello, It took me the whole day and I still cant work it out. Please see below what is wrong My Kibana user setting: metricbeat.yml: metricbeat.config.modules: path: ${path.config}/modules.d/\*.yml reload.enab… --- ## [Timestamp conversion](https://discuss.elastic.co/t/timestamp-conversion/242136)
**Author:** [@ishank](https://discuss.elastic.co/u/ishank)\ **Replies:** 1\ **Last updated:** [July 22, 2020, 5:37pm UTC](https://discuss.elastic.co/t/timestamp-conversion/242136 "2020-07-22T17:37:57Z")
Hi The data in my csv file is having the timestamp in format: 2019-04-11 12:00 AM (There are 3 spaces between the date & time) My logstash configuration for same: filter{ date { match =\> \["Time", "yyyy-MM-… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1910) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1912)