# Latest **URL:** https://discuss.elastic.co/latest.md?page=1690 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1691 --- ## [Filebeat Parsing issue with module aws](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374)
**Author:** [@AJ18](https://discuss.elastic.co/u/AJ18)\ **Replies:** 9\ **Last updated:** [May 7, 2021, 10:36am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374 "2021-05-07T10:36:29Z")
Hi, I had set up filebeat agent in my AWS account to ship logs from an S3 bucket. The s3 bucket contains Cloudtrail logs and VPC Flow Logs and has Access logging enabled. The file 'filebeat/modules.d/aws.yml' was modi… --- ## [Scan frequency working](https://discuss.elastic.co/t/scan-frequency-working/272406)
**Author:** [@Madhuri\_Penmatsa](https://discuss.elastic.co/u/Madhuri_Penmatsa)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 10:34am UTC](https://discuss.elastic.co/t/scan-frequency-working/272406 "2021-05-07T10:34:41Z")
I'm new to filebeat and wanted to know how scan\_frequency in filebeat actually works. When using the default scan frequency which is 10s, Our files of size around 400MB (sending the whole file at once using the multili… --- ## [Kibana not connecting after https setup](https://discuss.elastic.co/t/kibana-not-connecting-after-https-setup/272311)
**Author:** [@andcur](https://discuss.elastic.co/u/andcur)\ **Replies:** 6\ **Last updated:** [May 7, 2021, 10:26am UTC](https://discuss.elastic.co/t/kibana-not-connecting-after-https-setup/272311 "2021-05-07T10:26:14Z")
I have enabled https on elastic search and now kibana cannot connect. log \[15:28:25.571\] \[error\]\[elasticsearch\] Request error, retrying GET https://ukee-exp004:9200/\_xpack?accept\_enterprise=true =\> connect E… --- ## [Readiness probe failed: Waiting for elasticsearch cluster to become ready](https://discuss.elastic.co/t/readiness-probe-failed-waiting-for-elasticsearch-cluster-to-become-ready/272273)
**Author:** [@vbharath2007](https://discuss.elastic.co/u/vbharath2007)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 9:42am UTC](https://discuss.elastic.co/t/readiness-probe-failed-waiting-for-elasticsearch-cluster-to-become-ready/272273 "2021-05-07T09:42:59Z")
Hi, I'm running ELK stack using helm charts through the elastic community operator git repo. My ES is a single node cluster, today morning suddenly the pod failed and unable to bring up and receiving the below from the … --- ## [Registry.data.string is null](https://discuss.elastic.co/t/registry-data-string-is-null/272085)
**Author:** [@Nil\_Battey\_Sannata](https://discuss.elastic.co/u/Nil_Battey_Sannata)\ **Replies:** 3\ **Last updated:** [May 7, 2021, 9:39am UTC](https://discuss.elastic.co/t/registry-data-string-is-null/272085 "2021-05-07T09:39:36Z")
As per below logs from endpoint the field name from kibana registry.data.string should contain the value of data string from ImagePath, but it is giving Null value. the data which I want to be in registry.data.string it … --- ## [Mapping is different then discover shows](https://discuss.elastic.co/t/mapping-is-different-then-discover-shows/272330)
**Author:** [@alidrsn](https://discuss.elastic.co/u/alidrsn)\ **Replies:** 3\ **Last updated:** [May 7, 2021, 9:31am UTC](https://discuss.elastic.co/t/mapping-is-different-then-discover-shows/272330 "2021-05-07T09:31:05Z")
Hi there, I am trying to index data which is coming from logstash and ingesting with my ingest-pipeline. I am using index template to use some mapping fields. One of them is response\_time. When i check it from dev to… --- ## [Set Kibana index per Kibana space per beat via API](https://discuss.elastic.co/t/set-kibana-index-per-kibana-space-per-beat-via-api/272395)
**Author:** [@phlegx](https://discuss.elastic.co/u/phlegx)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 9:06am UTC](https://discuss.elastic.co/t/set-kibana-index-per-kibana-space-per-beat-via-api/272395 "2021-05-07T09:06:01Z")
It is possible to set the index per Kibana space for logAlias and metricAlias like this: $ curl -X POST http://your-kibana.example.com/s/\/api/saved\_objects/infrastructure-ui-source/default -H 'kbn-xsrf: true'… --- ## [Filebeat no connection could be made because the target machine actively refused it](https://discuss.elastic.co/t/filebeat-no-connection-could-be-made-because-the-target-machine-actively-refused-it/272302)
**Author:** [@Rashmika\_Gamage](https://discuss.elastic.co/u/Rashmika_Gamage)\ **Replies:** 9\ **Last updated:** [May 7, 2021, 8:55am UTC](https://discuss.elastic.co/t/filebeat-no-connection-could-be-made-because-the-target-machine-actively-refused-it/272302 "2021-05-07T08:55:23Z")
Hi all, I am trying to read logs from beats to logstash in the same machine. I am getting an error no connection could be made because the target machine actively refused it My logstash conf file input { beats … --- ## [How to filter out logs from modules.d configuration?](https://discuss.elastic.co/t/how-to-filter-out-logs-from-modules-d-configuration/272365)
**Author:** [@hazcod](https://discuss.elastic.co/u/hazcod)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 7:50am UTC](https://discuss.elastic.co/t/how-to-filter-out-logs-from-modules-d-configuration/272365 "2021-05-07T07:50:57Z")
Hi guys, So I am using the checkpoint module, configured in the modules.d directory. Is there a way to omit certain log lines using only the modules.d configuration? Thanks. --- ## [Installing Elasticsearch cluster in docker environment](https://discuss.elastic.co/t/installing-elasticsearch-cluster-in-docker-environment/272381)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 7:43am UTC](https://discuss.elastic.co/t/installing-elasticsearch-cluster-in-docker-environment/272381 "2021-05-07T07:43:25Z")
Would like to know the details related to installing the elasticsearch cluster in docker environment. Considering 3 VMs and having elasticsearch master and data nodes on each of these servers as containers, and also pos… --- ## [Nested Query summation aggregation for a subset received in the source](https://discuss.elastic.co/t/nested-query-summation-aggregation-for-a-subset-received-in-the-source/272038)
**Author:** [@RapTvin](https://discuss.elastic.co/u/RapTvin)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 7:16am UTC](https://discuss.elastic.co/t/nested-query-summation-aggregation-for-a-subset-received-in-the-source/272038 "2021-05-07T07:16:41Z")
Hello everyone, I am trying for a an aggregated summation query on a nested object on the basis of a variable inside it and the summation on another variable present inside it. Till now I have been able to segregate the … --- ## [Not able to query MongoDB.Bson.ObjectId using ElasticSearch Nest Client](https://discuss.elastic.co/t/not-able-to-query-mongodb-bson-objectid-using-elasticsearch-nest-client/272375)
**Author:** [@zeeshan](https://discuss.elastic.co/u/zeeshan)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 7:08am UTC](https://discuss.elastic.co/t/not-able-to-query-mongodb-bson-objectid-using-elasticsearch-nest-client/272375 "2021-05-07T07:08:58Z")
I have an field of type ObjectId (MongoDB.Bson) in my model, and when I index it elastic search server, it looks like following structure in Elastic Search: My question is, how can query (using NEST client) to match t… --- ## [Alternative for Delete By Query or Solution for Performance improvement](https://discuss.elastic.co/t/alternative-for-delete-by-query-or-solution-for-performance-improvement/272298)
**Author:** [@mrunalini](https://discuss.elastic.co/u/mrunalini)\ **Replies:** 5\ **Last updated:** [May 7, 2021, 7:06am UTC](https://discuss.elastic.co/t/alternative-for-delete-by-query-or-solution-for-performance-improvement/272298 "2021-05-07T07:06:29Z")
Hi , We have index in ELK ,where we are storing transaction details ,where we want to delete some records after some time. As quantity of requests is so high ,its taking so much time to complete the execution of the De… --- ## [How do I modify collect specific processes and top 10 processes in metricbeat's system module?](https://discuss.elastic.co/t/how-do-i-modify-collect-specific-processes-and-top-10-processes-in-metricbeats-system-module/272364)
**Author:** [@jongpyo.lee](https://discuss.elastic.co/u/jongpyo.lee)\ **Replies:** 2\ **Last updated:** [May 7, 2021, 7:01am UTC](https://discuss.elastic.co/t/how-do-i-modify-collect-specific-processes-and-top-10-processes-in-metricbeats-system-module/272364 "2021-05-07T07:01:51Z")
I have a question when collecting data from the system module. When collecting processes, I would like to collect the top 10 CPU and memory information. When collecting processes, I want to collect a specific process.n… --- ## [Help with grok pattern](https://discuss.elastic.co/t/help-with-grok-pattern/272351)
**Author:** [@Rahul\_A](https://discuss.elastic.co/u/Rahul_A)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 6:32am UTC](https://discuss.elastic.co/t/help-with-grok-pattern/272351 "2021-05-07T06:32:36Z")
Hi friends, I'm a newbie with grok filter and need help parsing the log message. Here is my log message: 2016-07-11T23:56:42.000+00:00 INFO ALERT|ECE002|5 Error with transaction for session -464410bf-37bf-475a-afc0-49… --- ## [How to solve Error no such index in Grafana](https://discuss.elastic.co/t/how-to-solve-error-no-such-index-in-grafana/272368)
**Author:** [@Arena\_Jo](https://discuss.elastic.co/u/Arena_Jo)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 6:15am UTC](https://discuss.elastic.co/t/how-to-solve-error-no-such-index-in-grafana/272368 "2021-05-07T06:15:33Z")
I try to add Data Sources / Elasticsearch on Grafana v6.5.2 But Error no such index. Nomallry I user kibana dashborad but I would like to use Grafana . please recommend to solve . --- ## [My Docker Elasticsearch Keeps Failing Looking For JAVA?](https://discuss.elastic.co/t/my-docker-elasticsearch-keeps-failing-looking-for-java/272334)
**Author:** [@cis4life](https://discuss.elastic.co/u/cis4life)\ **Replies:** 2\ **Last updated:** [May 7, 2021, 6:13am UTC](https://discuss.elastic.co/t/my-docker-elasticsearch-keeps-failing-looking-for-java/272334 "2021-05-07T06:13:12Z")
Good Day, I'm trying to stand up a demo / test install of ES on Docker for Mac. I have a M1 (ARM) MacBook. When I try to do the docker compose up I get the following error... Attaching to elasticsearch, kibana elast… --- ## [Ability to supply path to custom module?](https://discuss.elastic.co/t/ability-to-supply-path-to-custom-module/271615)
**Author:** [@hazcod](https://discuss.elastic.co/u/hazcod)\ **Replies:** 2\ **Last updated:** [May 7, 2021, 5:40am UTC](https://discuss.elastic.co/t/ability-to-supply-path-to-custom-module/271615 "2021-05-07T05:40:11Z")
Hi, So we developed a custom filebeat submodule which we'd like to use for filebeat. However there seems no way to specify the path to this module? We cannot copy it into /usr/share/filebeat/modules since that gets ov… --- ## [System module 추가시 모든 프로세스 정보 수집하면서 특정 프로세스 수집한 결과와 상위 10개의 프로세스 (Top 10 프로세스)를 함께 수집하고 싶습니다](https://discuss.elastic.co/t/system-module-10-top-10/272363)
**Author:** [@jongpyo.lee](https://discuss.elastic.co/u/jongpyo.lee)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 4:42am UTC](https://discuss.elastic.co/t/system-module-10-top-10/272363 "2021-05-07T04:42:32Z")
system module 의 데이터 수집시 궁금한 사항이 있어서 질문드립니다. processes 수집 할 때 CPU, Memory 정보 Top 10 상위 결과를 수집하고 싶습니다. processes 수집 할 때 특정 process.name 을 수집하고 싶습니다. (ex. Notepad\*, xyz.exe) 두가지의 수집 대상을 함께 수집하여 Visualize를 만들고자 하는데 System… --- ## [Logstash reading syslog from filebeat](https://discuss.elastic.co/t/logstash-reading-syslog-from-filebeat/272354)
**Author:** [@lachlan.simpson](https://discuss.elastic.co/u/lachlan.simpson)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 4:22am UTC](https://discuss.elastic.co/t/logstash-reading-syslog-from-filebeat/272354 "2021-05-07T04:22:11Z")
Using RHEL8, ELK 7.12, I have filebeat 7.12 set up on a RH7 workstation, configured with the syslog plugin and delivering data to logstash. The data is coming into logstash, but it's raw - the message field isn't being … --- ## [Elastic Agent fault by Fleet manager](https://discuss.elastic.co/t/elastic-agent-fault-by-fleet-manager/272347)
**Author:** [@root](https://discuss.elastic.co/u/root)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 1:44am UTC](https://discuss.elastic.co/t/elastic-agent-fault-by-fleet-manager/272347 "2021-05-07T01:44:19Z")
09:30:24.756 elastic\_agent \[elastic\_agent\]\[info\] The Elastic Agent is currently in BETA and should not be used in production 09:30:24.756 elastic\_agent \[elastic\_agent\]\[info\] The Elastic Agent is currently in BETA an… --- ## [How to logs from Azure microservice using logstash only when ELK stack is also in azure kubernetes](https://discuss.elastic.co/t/how-to-logs-from-azure-microservice-using-logstash-only-when-elk-stack-is-also-in-azure-kubernetes/272357)
**Author:** [@VBPrakash](https://discuss.elastic.co/u/VBPrakash)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 3:03am UTC](https://discuss.elastic.co/t/how-to-logs-from-azure-microservice-using-logstash-only-when-elk-stack-is-also-in-azure-kubernetes/272357 "2021-05-07T03:03:03Z")
I am trying to collect logs from a azure microservice using logstash. My entire ELK is running in azure kubernetes. Can you guys help me in achieving this ? Note: ELK on 7.12.1 --- ## [\[WARN \] 2021-05-06 23:01:18.054 \[\[marketdata\_fix\_pipeline\]\>worker0\] split - Only String and Array types are splittable. field:\[layers\]\[fix\] is of type = Hash](https://discuss.elastic.co/t/warn-2021-05-06-2318-054-marketdata-fix-pipeline-worker0-split-only-string-and-array-types-are-splittable-field-layers-fix-is-of-type-hash/272344)
**Author:** [@wmei](https://discuss.elastic.co/u/wmei)\ **Replies:** 3\ **Last updated:** [May 7, 2021, 12:16am UTC](https://discuss.elastic.co/t/warn-2021-05-06-2318-054-marketdata-fix-pipeline-worker0-split-only-string-and-array-types-are-splittable-field-layers-fix-is-of-type-hash/272344 "2021-05-07T00:16:08Z")
Hi, I am getting this warning message and I believe it is because some of the events do not have a \[layers\]\[fix\]. \[WARN \] 2021-05-06 23:01:18.054 \[\[marketdata\_fix\_pipeline\]\>worker0\] split - Only String and Array types … --- ## [How to discard specific event from storing or correlation in SIEM to save resources](https://discuss.elastic.co/t/how-to-discard-specific-event-from-storing-or-correlation-in-siem-to-save-resources/271716)
**Author:** [@zoot](https://discuss.elastic.co/u/zoot)\ **Replies:** 1\ **Last updated:** [May 6, 2021, 11:02pm UTC](https://discuss.elastic.co/t/how-to-discard-specific-event-from-storing-or-correlation-in-siem-to-save-resources/271716 "2021-05-06T23:02:20Z")
Hi, In SIEM systems events can be discard or filter to safe diskspace and avoid overloading SIEM from noisy events. This way SIEM resources can be managed at optimum levels. For example windows event 5156 from all agen… --- ## [Brute Force Detection Rule](https://discuss.elastic.co/t/brute-force-detection-rule/271713)
**Author:** [@filip.wozniak](https://discuss.elastic.co/u/filip.wozniak)\ **Replies:** 3\ **Last updated:** [May 6, 2021, 11:00pm UTC](https://discuss.elastic.co/t/brute-force-detection-rule/271713 "2021-05-06T23:00:32Z")
Greetings! I'm using WinlogBeats with configured Sysmon. I want to write my own rule: If any user in last 5 minutes failed to login 5 times, Detection should trigger. So I tried this: This is threshold rule, but I… --- ## [Create new issue in Jira for each event in a detection](https://discuss.elastic.co/t/create-new-issue-in-jira-for-each-event-in-a-detection/270744)
**Author:** [@Felipe\_Fuller](https://discuss.elastic.co/u/Felipe_Fuller)\ **Replies:** 10\ **Last updated:** [May 6, 2021, 10:52pm UTC](https://discuss.elastic.co/t/create-new-issue-in-jira-for-each-event-in-a-detection/270744 "2021-05-06T22:52:10Z")
Hi Community! I hope that you are doing great! I'm trying to send an issue per event in a detection, but I'm struggling to find any solution. So in this case, there should be 5 issues created in Jira. Thank you ver… --- ## [Filebeat: Fortinet Module not processing data as it should](https://discuss.elastic.co/t/filebeat-fortinet-module-not-processing-data-as-it-should/272102)
**Author:** [@b0r1s](https://discuss.elastic.co/u/b0r1s)\ **Replies:** 14\ **Last updated:** [May 6, 2021, 9:58pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-module-not-processing-data-as-it-should/272102 "2021-05-06T21:58:32Z")
I think I have a similar problem from this post but I can't solve with the solution on the post. Here is my filebeat.yml: # ============================== Filebeat modules ============================== filebeat.confi… --- ## [Logstash Elasticsearch Input](https://discuss.elastic.co/t/logstash-elasticsearch-input/272336)
**Author:** [@vishakh](https://discuss.elastic.co/u/vishakh)\ **Replies:** 3\ **Last updated:** [May 6, 2021, 9:52pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input/272336 "2021-05-06T21:52:21Z")
I'm trying to extract (or) export logs from Elasticsearch via Logstash I would like to write the extracted logs to a file input { elasticsearch { hosts =\> "elasticsearch.domain.com:9200" query =\> '{ "size": 5… --- ## [Create one custom document if no data is retrive from the JDBC input](https://discuss.elastic.co/t/create-one-custom-document-if-no-data-is-retrive-from-the-jdbc-input/272326)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [May 6, 2021, 6:12pm UTC](https://discuss.elastic.co/t/create-one-custom-document-if-no-data-is-retrive-from-the-jdbc-input/272326 "2021-05-06T18:12:39Z")
Hi, Its is posible to index one custom document if no data id retrieve from a database? how can logstash do this? --- ## [Watcher alerts 7.5 interval and bucket span](https://discuss.elastic.co/t/watcher-alerts-7-5-interval-and-bucket-span/272319)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 3\ **Last updated:** [May 6, 2021, 9:12pm UTC](https://discuss.elastic.co/t/watcher-alerts-7-5-interval-and-bucket-span/272319 "2021-05-06T21:12:54Z")
Why if its recomended that the interval of a watch has to be equal to twice the defined bucket\_span for the job. , when I create an anomaly detection alert in the kibana GUI, the interval is set to aprox. one minute? m… [Previous page](https://discuss.elastic.co/latest.md?page=1689) [Next page](https://discuss.elastic.co/latest.md?page=1691)