# Latest **URL:** https://discuss.elastic.co/latest.md?page=1724 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1725 --- ## [Correlation analysis in elasticsearch](https://discuss.elastic.co/t/correlation-analysis-in-elasticsearch/270346)
**Author:** [@111387](https://discuss.elastic.co/u/111387)\ **Replies:** 0\ **Last updated:** [April 16, 2021, 8:14am UTC](https://discuss.elastic.co/t/correlation-analysis-in-elasticsearch/270346 "2021-04-16T08:14:22Z")
I am collecting suricata events with elasticsearch. I want to create an alert through correlation analysis for suricata event. For example, when the following events are saved { rule.event: "event a" src.ip: "… --- ## [Missing logs randomly on kibana](https://discuss.elastic.co/t/missing-logs-randomly-on-kibana/270329)
**Author:** [@danushkalakmina](https://discuss.elastic.co/u/danushkalakmina)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:32pm UTC](https://discuss.elastic.co/t/missing-logs-randomly-on-kibana/270329 "2021-04-19T23:32:37Z")
Hi My Kibana logs are missing randomly from every log bulk. But the raw log is in the original log location on the application server. --- ## [Filebeat -e running for a long time](https://discuss.elastic.co/t/filebeat-e-running-for-a-long-time/270292)
**Author:** [@thrak](https://discuss.elastic.co/u/thrak)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:31pm UTC](https://discuss.elastic.co/t/filebeat-e-running-for-a-long-time/270292 "2021-04-19T23:31:10Z")
Hello I am running a new installation of filebeat 7.12.0. I was initially getting an ILM error and so had the indices removed for filebeats. Now when I run the filebeat -e setup, it seems to run for ever. I let it go fo… --- ## [Cannot simulate watch - Request Timeout after 90000ms](https://discuss.elastic.co/t/cannot-simulate-watch-request-timeout-after-90000ms/270287)
**Author:** [@kksandesh1](https://discuss.elastic.co/u/kksandesh1)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:30pm UTC](https://discuss.elastic.co/t/cannot-simulate-watch-request-timeout-after-90000ms/270287 "2021-04-19T23:30:15Z")
I have watcher setup to send report. But getting the below error Cannot simulate watch Request Timeout after 90000ms. Please help --- ## [Stackmonitor production cluster on a different monitoring cluster](https://discuss.elastic.co/t/stackmonitor-production-cluster-on-a-different-monitoring-cluster/270277)
**Author:** [@Pooja](https://discuss.elastic.co/u/Pooja)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:29pm UTC](https://discuss.elastic.co/t/stackmonitor-production-cluster-on-a-different-monitoring-cluster/270277 "2021-04-19T23:29:25Z")
I would like to get help on monitoring production cluster on a separate monitoring cluster using metricbeat. As of now I am finding issues sending the data across clusters. I have the metricbeat setup on poduction clus… --- ## [Index\_closed\_exception on open index](https://discuss.elastic.co/t/index-closed-exception-on-open-index/270273)
**Author:** [@maigel](https://discuss.elastic.co/u/maigel)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:28pm UTC](https://discuss.elastic.co/t/index-closed-exception-on-open-index/270273 "2021-04-19T23:28:57Z")
Hi, We currently have a cluster with a lot of network security (Zeek) logs. I have a script written with elasticsearch-py which periodically searches all data for known IOCs (bad IPs, bad domains, etc.) Every once in a… --- ## [Many process](https://discuss.elastic.co/t/many-process/270257)
**Author:** [@gh\_darvishani](https://discuss.elastic.co/u/gh_darvishani)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:27pm UTC](https://discuss.elastic.co/t/many-process/270257 "2021-04-19T23:27:34Z")
i use magento as my cms and ES as the search engine. my server (apache) after a while created. i check the log and process and i saw many processes for ES how i can set a limit for processes or threads --- ## [Filebeat Harvester not being started for files](https://discuss.elastic.co/t/filebeat-harvester-not-being-started-for-files/270639)
**Author:** [@Usman18](https://discuss.elastic.co/u/Usman18)\ **Replies:** 0\ **Last updated:** [April 19, 2021, 11:27pm UTC](https://discuss.elastic.co/t/filebeat-harvester-not-being-started-for-files/270639 "2021-04-19T23:27:30Z")
I am running Filebeat inside docker container. My Filebeat service is working correctly on mac os and ubuntu machine but on another ubuntu vm it's not working. Harvesters are not started for the files neither connection … --- ## [Index created with too many fields, automatic creation](https://discuss.elastic.co/t/index-created-with-too-many-fields-automatic-creation/270245)
**Author:** [@PSTech](https://discuss.elastic.co/u/PSTech)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:26pm UTC](https://discuss.elastic.co/t/index-created-with-too-many-fields-automatic-creation/270245 "2021-04-19T23:26:51Z")
hi, I've been trying to setup apache module Filebeat, with automatic index creation. after my index was created I wasn't able to create an index pattern in my Kibana UI with the error code 413 Request Entity Too Large","… --- ## [Better define Kibana Watchers for unavailable Pod](https://discuss.elastic.co/t/better-define-kibana-watchers-for-unavailable-pod/270221)
**Author:** [@es-gabriele](https://discuss.elastic.co/u/es-gabriele)\ **Replies:** 0\ **Last updated:** [April 15, 2021, 11:05am UTC](https://discuss.elastic.co/t/better-define-kibana-watchers-for-unavailable-pod/270221 "2021-04-15T11:05:45Z")
2021-04-14T22:00:00Z (UTC) Good morning, I could need assistance with a particular task. Basically, we implemented a Watcher that is triggered whenever a Pod is unavailable, and we will export this trigger alarm to Ops… --- ## [ECE add extra storage disk and not seen by the instance](https://discuss.elastic.co/t/ece-add-extra-storage-disk-and-not-seen-by-the-instance/270118)
**Author:** [@Diana\_Dragoiu](https://discuss.elastic.co/u/Diana_Dragoiu)\ **Replies:** 0\ **Last updated:** [April 14, 2021, 1:34pm UTC](https://discuss.elastic.co/t/ece-add-extra-storage-disk-and-not-seen-by-the-instance/270118 "2021-04-14T13:34:20Z")
Hello there, I created a hot node with 30 GB and I need extra storage(100GB). I added(mounted) a disk but the ECE instance can not see the new extra storage. \[user@xxx~\]$ df -h Filesystem Size Used Avail Use% Mo… --- ## [Looking for updated geoip database](https://discuss.elastic.co/t/looking-for-updated-geoip-database/270450)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 11:14pm UTC](https://discuss.elastic.co/t/looking-for-updated-geoip-database/270450 "2021-04-19T23:14:41Z")
hi if somebody can help me to find a free database GeoIP updated thanks. --- ## [How to convert painless script to java api](https://discuss.elastic.co/t/how-to-convert-painless-script-to-java-api/270610)
**Author:** [@Rahul4](https://discuss.elastic.co/u/Rahul4)\ **Replies:** 0\ **Last updated:** [April 19, 2021, 5:07pm UTC](https://discuss.elastic.co/t/how-to-convert-painless-script-to-java-api/270610 "2021-04-19T17:07:12Z")
Below is the Elasticsearch painless script query , please suggest me the JAVA API or how to write the java code for below query I wanted to prepare the same query using Java 1.8 and RestHighLevelClient Request:- GET /x… --- ## [Logstash setting field to date without any options set](https://discuss.elastic.co/t/logstash-setting-field-to-date-without-any-options-set/270621)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 3\ **Last updated:** [April 19, 2021, 10:17pm UTC](https://discuss.elastic.co/t/logstash-setting-field-to-date-without-any-options-set/270621 "2021-04-19T22:17:56Z")
I am ingesting a csv file. The only options set in the filter are a gsub meant to remove double quotes; a csv filter which skips the header, the separator and the column/field names set; and a mutate to add one field an… --- ## [Dynamic Mapping for Nested Objects](https://discuss.elastic.co/t/dynamic-mapping-for-nested-objects/270628)
**Author:** [@schawla](https://discuss.elastic.co/u/schawla)\ **Replies:** 0\ **Last updated:** [April 19, 2021, 9:27pm UTC](https://discuss.elastic.co/t/dynamic-mapping-for-nested-objects/270628 "2021-04-19T21:27:24Z")
Is it possible to have dynamic mapping for a nested object like the following - employee { "id":"1", "name":"manager", "assistants":\[ { "id":"2", "name":"assistant1" }, { "id":"3… --- ## [Conversion of hexadecimal to integer](https://discuss.elastic.co/t/conversion-of-hexadecimal-to-integer/270615)
**Author:** [@Sruthi1](https://discuss.elastic.co/u/Sruthi1)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 9:14pm UTC](https://discuss.elastic.co/t/conversion-of-hexadecimal-to-integer/270615 "2021-04-19T21:14:01Z")
Hiiii I am using kibana 7.2.0. I have used the function Integer.parseInt() to convert the hexadecimal to integer value, but it is throwing runtime error. if(doc\['hex\_value'\]!=null){return Integer.parseInt(doc\['hex\_valu… --- ## [Import logs using filebeat for a java app running on kubernetes](https://discuss.elastic.co/t/import-logs-using-filebeat-for-a-java-app-running-on-kubernetes/270625)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 9:11pm UTC](https://discuss.elastic.co/t/import-logs-using-filebeat-for-a-java-app-running-on-kubernetes/270625 "2021-04-19T21:11:07Z")
Hello I am reading the documentation of now to format and read logs on a Java app that is running in kubernetes here. In this documentation said to enable hint based discovery and add the annotation that are using the l… --- ## [Elastic Agent Macbook M1](https://discuss.elastic.co/t/elastic-agent-macbook-m1/270578)
**Author:** [@fgierlinger](https://discuss.elastic.co/u/fgierlinger)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 8:52pm UTC](https://discuss.elastic.co/t/elastic-agent-macbook-m1/270578 "2021-04-19T20:52:07Z")
Are there any plans to release the elastic agent for the new m1 cpu from apple (Darwin aarch64)? Is anybody aware of an unofficial build of the elastic agent for the m1 chip? Currently, there are only builds for Linux a… --- ## [Ruby exception occurred: undefined method \`size' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-size-for-nil-nilclass/270409)
**Author:** [@Eduard\_Abril](https://discuss.elastic.co/u/Eduard_Abril)\ **Replies:** 2\ **Last updated:** [April 19, 2021, 8:24pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-size-for-nil-nilclass/270409 "2021-04-19T20:24:00Z")
Hi guys, I have the next error on a ruby filter. I'd really appreciate any suggest \[2021-04-16T11:28:06,202\]\[ERROR\]\[logstash.filters.ruby \]\[main-postilion-pipeline\]\[87afa373e1d0191b4f869238d2908162c237ad31fe6074ec0a… --- ## [Metricbeat 7.11.1 fail in system fsstat and filesystem: The device is not ready](https://discuss.elastic.co/t/metricbeat-7-11-1-fail-in-system-fsstat-and-filesystem-the-device-is-not-ready/265613)
**Author:** [@Sommerdal](https://discuss.elastic.co/u/Sommerdal)\ **Replies:** 8\ **Last updated:** [April 19, 2021, 8:04pm UTC](https://discuss.elastic.co/t/metricbeat-7-11-1-fail-in-system-fsstat-and-filesystem-the-device-is-not-ready/265613 "2021-04-19T20:04:16Z")
I keep getting this: module/wrapper.go:259 Error fetching data for metricset system.fsstat: filesystem list: GetFilesystemType failed: GetVolumeInformationW failed: The device is module/wrapper.go:259 Error fetchin… --- ## [Unable to connect to Enterprise App Search via Node.js](https://discuss.elastic.co/t/unable-to-connect-to-enterprise-app-search-via-node-js/270611)
**Author:** [@Shoto\_Todoroki](https://discuss.elastic.co/u/Shoto_Todoroki)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 7:59pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-enterprise-app-search-via-node-js/270611 "2021-04-19T19:59:08Z")
Hi, I am trying to connect to Enterprise App Search via node.js. I followed the below github code https://github.com/elastic/app-search-node but this is the error I am getting while connecting to the enterprise engine … --- ## [Kibana date shows up as plus one](https://discuss.elastic.co/t/kibana-date-shows-up-as-plus-one/270614)
**Author:** [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Replies:** 3\ **Last updated:** [April 19, 2021, 7:50pm UTC](https://discuss.elastic.co/t/kibana-date-shows-up-as-plus-one/270614 "2021-04-19T19:50:23Z")
Hello Team, My dates are showing up as minus one in comparison to the raw JSON, when I click on discover and search a record it shows up as 20210410 (see snippet attached) but when I check the raw JSON it is 20210411 … --- ## [Ingest JSON data into elasticsearch using logstash](https://discuss.elastic.co/t/ingest-json-data-into-elasticsearch-using-logstash/270558)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 4\ **Last updated:** [April 19, 2021, 6:33pm UTC](https://discuss.elastic.co/t/ingest-json-data-into-elasticsearch-using-logstash/270558 "2021-04-19T18:33:59Z")
HI All, I have a complicated json data to ingest into elasticsearch, need your suggestion to achieve this. Here is my data format, \ \ \/api/device/13766\ \ **Author:** [@accateo](https://discuss.elastic.co/u/accateo)\ **Replies:** 9\ **Last updated:** [April 19, 2021, 6:24pm UTC](https://discuss.elastic.co/t/canvas-conditional-statement/270495 "2021-04-19T18:24:15Z") Hi, Is it possible to add, on the canvas, an element that, based on the value of a field of an index, shows an image instead of another? Thank you --- ## [Need help with creating Kibana Alert](https://discuss.elastic.co/t/need-help-with-creating-kibana-alert/270508)
**Author:** [@Long\_Nguyen](https://discuss.elastic.co/u/Long_Nguyen)\ **Replies:** 8\ **Last updated:** [April 19, 2021, 6:19pm UTC](https://discuss.elastic.co/t/need-help-with-creating-kibana-alert/270508 "2021-04-19T18:19:50Z")
Hello everyone, I am new to the Elasticstack in general and I have encountered a problem with creating an Alert using the Elasticsearch Query. I am using the index from metricbeat that indexes various processes of a hos… --- ## [Restore kibana from a snapshot](https://discuss.elastic.co/t/restore-kibana-from-a-snapshot/270517)
**Author:** [@NightSpark](https://discuss.elastic.co/u/NightSpark)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 6:11pm UTC](https://discuss.elastic.co/t/restore-kibana-from-a-snapshot/270517 "2021-04-19T18:11:54Z")
Hi, Is it possible to just restore the .kibana system index from a snapshot? This is in the event that, through human error, an alert is modified and we wish to restore the original without losing other data that may h… --- ## [Kibana setting Alerts/Anomalies to email](https://discuss.elastic.co/t/kibana-setting-alerts-anomalies-to-email/270590)
**Author:** [@miklin](https://discuss.elastic.co/u/miklin)\ **Replies:** 1\ **Last updated:** [April 19, 2021, 6:01pm UTC](https://discuss.elastic.co/t/kibana-setting-alerts-anomalies-to-email/270590 "2021-04-19T18:01:35Z")
Hi, I'm taking the Kibana training courses and have a quick question. Is it possible to set an email alert on Kibana? If it's doable, how can I set up? (There are training sessions on Machine learning in Elastic Stack a… --- ## [Lucene Merge Thread causing high CPU usage](https://discuss.elastic.co/t/lucene-merge-thread-causing-high-cpu-usage/270599)
**Author:** [@souravsahoo](https://discuss.elastic.co/u/souravsahoo)\ **Replies:** 2\ **Last updated:** [April 19, 2021, 5:37pm UTC](https://discuss.elastic.co/t/lucene-merge-thread-causing-high-cpu-usage/270599 "2021-04-19T17:37:06Z")
Hi all, I have been facing issue with elasticsearch consuming a very high CPU i.e., 80-100% when a heavy data load is ingested. Below are few inputs:- Elasticsearch is running on. 7.10.2 version We have data tier arc… --- ## [Threat Intel module](https://discuss.elastic.co/t/threat-intel-module/269679)
**Author:** [@sbathla](https://discuss.elastic.co/u/sbathla)\ **Replies:** 2\ **Last updated:** [April 19, 2021, 5:15pm UTC](https://discuss.elastic.co/t/threat-intel-module/269679 "2021-04-19T17:15:39Z")
Hi all, Need one help. I tried to integrate threat intel module in 7.12 version. Post integration I am able to view dashboard for Abuse URL and Abuse malware but not getting results for MISP, Otx, alienvault..Did the re… --- ## [Unable to receive source fields over email from Watcher alerting](https://discuss.elastic.co/t/unable-to-receive-source-fields-over-email-from-watcher-alerting/269914)
**Author:** [@arunhk3](https://discuss.elastic.co/u/arunhk3)\ **Replies:** 5\ **Last updated:** [April 19, 2021, 3:44pm UTC](https://discuss.elastic.co/t/unable-to-receive-source-fields-over-email-from-watcher-alerting/269914 "2021-04-19T15:44:06Z")
Hi All, I have been trying for nearly 4 hours already to get an email alert with data table using Watcher, despite receiving the email it fails to send the data using the \_source fields specified. My Watcher alert is as… [Previous page](https://discuss.elastic.co/latest.md?page=1723) [Next page](https://discuss.elastic.co/latest.md?page=1725)