# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1426 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1427 --- ## [Alternative for Delete By Query or Solution for Performance improvement](https://discuss.elastic.co/t/alternative-for-delete-by-query-or-solution-for-performance-improvement/272298)
**Author:** [@mrunalini](https://discuss.elastic.co/u/mrunalini)\ **Replies:** 5\ **Last updated:** [May 7, 2021, 7:06am UTC](https://discuss.elastic.co/t/alternative-for-delete-by-query-or-solution-for-performance-improvement/272298 "2021-05-07T07:06:29Z")
Hi , We have index in ELK ,where we are storing transaction details ,where we want to delete some records after some time. As quantity of requests is so high ,its taking so much time to complete the execution of the De… --- ## [How do I modify collect specific processes and top 10 processes in metricbeat's system module?](https://discuss.elastic.co/t/how-do-i-modify-collect-specific-processes-and-top-10-processes-in-metricbeats-system-module/272364)
**Author:** [@jongpyo.lee](https://discuss.elastic.co/u/jongpyo.lee)\ **Replies:** 2\ **Last updated:** [May 7, 2021, 7:01am UTC](https://discuss.elastic.co/t/how-do-i-modify-collect-specific-processes-and-top-10-processes-in-metricbeats-system-module/272364 "2021-05-07T07:01:51Z")
I have a question when collecting data from the system module. When collecting processes, I would like to collect the top 10 CPU and memory information. When collecting processes, I want to collect a specific process.n… --- ## [Help with grok pattern](https://discuss.elastic.co/t/help-with-grok-pattern/272351)
**Author:** [@Rahul\_A](https://discuss.elastic.co/u/Rahul_A)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 6:32am UTC](https://discuss.elastic.co/t/help-with-grok-pattern/272351 "2021-05-07T06:32:36Z")
Hi friends, I'm a newbie with grok filter and need help parsing the log message. Here is my log message: 2016-07-11T23:56:42.000+00:00 INFO ALERT|ECE002|5 Error with transaction for session -464410bf-37bf-475a-afc0-49… --- ## [How to solve Error no such index in Grafana](https://discuss.elastic.co/t/how-to-solve-error-no-such-index-in-grafana/272368)
**Author:** [@Arena\_Jo](https://discuss.elastic.co/u/Arena_Jo)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 6:15am UTC](https://discuss.elastic.co/t/how-to-solve-error-no-such-index-in-grafana/272368 "2021-05-07T06:15:33Z")
I try to add Data Sources / Elasticsearch on Grafana v6.5.2 But Error no such index. Nomallry I user kibana dashborad but I would like to use Grafana . please recommend to solve . --- ## [My Docker Elasticsearch Keeps Failing Looking For JAVA?](https://discuss.elastic.co/t/my-docker-elasticsearch-keeps-failing-looking-for-java/272334)
**Author:** [@cis4life](https://discuss.elastic.co/u/cis4life)\ **Replies:** 2\ **Last updated:** [May 7, 2021, 6:13am UTC](https://discuss.elastic.co/t/my-docker-elasticsearch-keeps-failing-looking-for-java/272334 "2021-05-07T06:13:12Z")
Good Day, I'm trying to stand up a demo / test install of ES on Docker for Mac. I have a M1 (ARM) MacBook. When I try to do the docker compose up I get the following error... Attaching to elasticsearch, kibana elast… --- ## [Ability to supply path to custom module?](https://discuss.elastic.co/t/ability-to-supply-path-to-custom-module/271615)
**Author:** [@hazcod](https://discuss.elastic.co/u/hazcod)\ **Replies:** 2\ **Last updated:** [May 7, 2021, 5:40am UTC](https://discuss.elastic.co/t/ability-to-supply-path-to-custom-module/271615 "2021-05-07T05:40:11Z")
Hi, So we developed a custom filebeat submodule which we'd like to use for filebeat. However there seems no way to specify the path to this module? We cannot copy it into /usr/share/filebeat/modules since that gets ov… --- ## [Logstash reading syslog from filebeat](https://discuss.elastic.co/t/logstash-reading-syslog-from-filebeat/272354)
**Author:** [@lachlan.simpson](https://discuss.elastic.co/u/lachlan.simpson)\ **Replies:** 1\ **Last updated:** [May 7, 2021, 4:22am UTC](https://discuss.elastic.co/t/logstash-reading-syslog-from-filebeat/272354 "2021-05-07T04:22:11Z")
Using RHEL8, ELK 7.12, I have filebeat 7.12 set up on a RH7 workstation, configured with the syslog plugin and delivering data to logstash. The data is coming into logstash, but it's raw - the message field isn't being … --- ## [Elastic Agent fault by Fleet manager](https://discuss.elastic.co/t/elastic-agent-fault-by-fleet-manager/272347)
**Author:** [@root](https://discuss.elastic.co/u/root)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 1:44am UTC](https://discuss.elastic.co/t/elastic-agent-fault-by-fleet-manager/272347 "2021-05-07T01:44:19Z")
09:30:24.756 elastic\_agent \[elastic\_agent\]\[info\] The Elastic Agent is currently in BETA and should not be used in production 09:30:24.756 elastic\_agent \[elastic\_agent\]\[info\] The Elastic Agent is currently in BETA an… --- ## [How to logs from Azure microservice using logstash only when ELK stack is also in azure kubernetes](https://discuss.elastic.co/t/how-to-logs-from-azure-microservice-using-logstash-only-when-elk-stack-is-also-in-azure-kubernetes/272357)
**Author:** [@VBPrakash](https://discuss.elastic.co/u/VBPrakash)\ **Replies:** 0\ **Last updated:** [May 7, 2021, 3:03am UTC](https://discuss.elastic.co/t/how-to-logs-from-azure-microservice-using-logstash-only-when-elk-stack-is-also-in-azure-kubernetes/272357 "2021-05-07T03:03:03Z")
I am trying to collect logs from a azure microservice using logstash. My entire ELK is running in azure kubernetes. Can you guys help me in achieving this ? Note: ELK on 7.12.1 --- ## [\[WARN \] 2021-05-06 23:01:18.054 \[\[marketdata\_fix\_pipeline\]\>worker0\] split - Only String and Array types are splittable. field:\[layers\]\[fix\] is of type = Hash](https://discuss.elastic.co/t/warn-2021-05-06-2318-054-marketdata-fix-pipeline-worker0-split-only-string-and-array-types-are-splittable-field-layers-fix-is-of-type-hash/272344)
**Author:** [@wmei](https://discuss.elastic.co/u/wmei)\ **Replies:** 3\ **Last updated:** [May 7, 2021, 12:16am UTC](https://discuss.elastic.co/t/warn-2021-05-06-2318-054-marketdata-fix-pipeline-worker0-split-only-string-and-array-types-are-splittable-field-layers-fix-is-of-type-hash/272344 "2021-05-07T00:16:08Z")
Hi, I am getting this warning message and I believe it is because some of the events do not have a \[layers\]\[fix\]. \[WARN \] 2021-05-06 23:01:18.054 \[\[marketdata\_fix\_pipeline\]\>worker0\] split - Only String and Array types … --- ## [Filebeat: Fortinet Module not processing data as it should](https://discuss.elastic.co/t/filebeat-fortinet-module-not-processing-data-as-it-should/272102)
**Author:** [@b0r1s](https://discuss.elastic.co/u/b0r1s)\ **Replies:** 14\ **Last updated:** [May 6, 2021, 9:58pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-module-not-processing-data-as-it-should/272102 "2021-05-06T21:58:32Z")
I think I have a similar problem from this post but I can't solve with the solution on the post. Here is my filebeat.yml: # ============================== Filebeat modules ============================== filebeat.confi… --- ## [Logstash Elasticsearch Input](https://discuss.elastic.co/t/logstash-elasticsearch-input/272336)
**Author:** [@vishakh](https://discuss.elastic.co/u/vishakh)\ **Replies:** 3\ **Last updated:** [May 6, 2021, 9:52pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input/272336 "2021-05-06T21:52:21Z")
I'm trying to extract (or) export logs from Elasticsearch via Logstash I would like to write the extracted logs to a file input { elasticsearch { hosts =\> "elasticsearch.domain.com:9200" query =\> '{ "size": 5… --- ## [Create one custom document if no data is retrive from the JDBC input](https://discuss.elastic.co/t/create-one-custom-document-if-no-data-is-retrive-from-the-jdbc-input/272326)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [May 6, 2021, 6:12pm UTC](https://discuss.elastic.co/t/create-one-custom-document-if-no-data-is-retrive-from-the-jdbc-input/272326 "2021-05-06T18:12:39Z")
Hi, Its is posible to index one custom document if no data id retrieve from a database? how can logstash do this? --- ## [Watcher alerts 7.5 interval and bucket span](https://discuss.elastic.co/t/watcher-alerts-7-5-interval-and-bucket-span/272319)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 3\ **Last updated:** [May 6, 2021, 9:12pm UTC](https://discuss.elastic.co/t/watcher-alerts-7-5-interval-and-bucket-span/272319 "2021-05-06T21:12:54Z")
Why if its recomended that the interval of a watch has to be equal to twice the defined bucket\_span for the job. , when I create an anomaly detection alert in the kibana GUI, the interval is set to aprox. one minute? m… --- ## [Unable to index a file (Word document) greater than 45 MB](https://discuss.elastic.co/t/unable-to-index-a-file-word-document-greater-than-45-mb/272220)
**Author:** [@saqib619](https://discuss.elastic.co/u/saqib619)\ **Replies:** 5\ **Last updated:** [May 6, 2021, 6:59pm UTC](https://discuss.elastic.co/t/unable-to-index-a-file-word-document-greater-than-45-mb/272220 "2021-05-06T18:59:21Z")
We are trying to index a file (word document) which is \> 45 MB . It is failing throwing an exception as below: Maximum timeout reached while retrying request. Call: Status code unknown from: PUT /test\_index6/\_doc/16601?… --- ## [Index template order fields seems not to work](https://discuss.elastic.co/t/index-template-order-fields-seems-not-to-work/271778)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 6\ **Last updated:** [May 6, 2021, 6:07pm UTC](https://discuss.elastic.co/t/index-template-order-fields-seems-not-to-work/271778 "2021-05-06T18:07:33Z")
Hello, I have few templates each for every index name based on the module name for ilm management. I notice that after the update to 7.12.1 all my indices have applied templates with order value 10 event though there a… --- ## [No Data for Metricbeat AWS billing estimated charges](https://discuss.elastic.co/t/no-data-for-metricbeat-aws-billing-estimated-charges/272132)
**Author:** [@Sathish22](https://discuss.elastic.co/u/Sathish22)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 4:05pm UTC](https://discuss.elastic.co/t/no-data-for-metricbeat-aws-billing-estimated-charges/272132 "2021-05-06T16:05:00Z")
Dear Team, We have configured Metricbeat with AWS module for AWS billing metrics PFB configuration - module: aws period: 24h metricsets: - billing role\_arn: xxxxxxxxxxxxxxxx cost\_explorer\_config: group\_… --- ## [Heartbeat issue when matching on response body string negatively](https://discuss.elastic.co/t/heartbeat-issue-when-matching-on-response-body-string-negatively/270542)
**Author:** [@Nipun1](https://discuss.elastic.co/u/Nipun1)\ **Replies:** 1\ **Last updated:** [May 6, 2021, 3:26pm UTC](https://discuss.elastic.co/t/heartbeat-issue-when-matching-on-response-body-string-negatively/270542 "2021-05-06T15:26:16Z")
Hi all, I am setting up a heartbeat monitor (version 7.5.0) for an API whose response can be of the following types #response is okay, API can be considered up { "usersList": \[ { "firstName": "L… --- ## [Parsing nested json not happening properly](https://discuss.elastic.co/t/parsing-nested-json-not-happening-properly/272282)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 3:20pm UTC](https://discuss.elastic.co/t/parsing-nested-json-not-happening-properly/272282 "2021-05-06T15:20:51Z")
Hi Folks, I am trying to parse modsecurity audit logs which are natively being logged in JSON format. However when I am using logstash to ingest in elastic stack those appears like below and actual needed fields are not… --- ## [How to remove original message field after parsing using json filter](https://discuss.elastic.co/t/how-to-remove-original-message-field-after-parsing-using-json-filter/272306)
**Author:** [@Sandeep4](https://discuss.elastic.co/u/Sandeep4)\ **Replies:** 1\ **Last updated:** [May 6, 2021, 2:51pm UTC](https://discuss.elastic.co/t/how-to-remove-original-message-field-after-parsing-using-json-filter/272306 "2021-05-06T14:51:50Z")
First try:- filter { mutate { gsub =\> \[ "message", "^\\w+\`string\`", "" \] } json { source =\> "message" remove\_field =\> "message" } } Second try:- filter { mutate { gsub =\> \[ "message", "^\\w+\`string\`",… --- ## [Is it possible to replicate the index WITHOUT yellow status? (replication disabled due to faster reindex)](https://discuss.elastic.co/t/is-it-possible-to-replicate-the-index-without-yellow-status-replication-disabled-due-to-faster-reindex/272308)
**Author:** [@astrodi](https://discuss.elastic.co/u/astrodi)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 2:42pm UTC](https://discuss.elastic.co/t/is-it-possible-to-replicate-the-index-without-yellow-status-replication-disabled-due-to-faster-reindex/272308 "2021-05-06T14:42:15Z")
Hi there, due to speedup of re-indexing, I'm setting "number\_of\_replicas": 0 while re-indexing is running. Once finished, than I'm setting 1 replica: PUT bmpf\_claim\_docs\_v2/\_settings { "number\_of\_replicas": 1 } U… --- ## [Text Embedding in Elastic Search](https://discuss.elastic.co/t/text-embedding-in-elastic-search/272242)
**Author:** [@harry\_richard](https://discuss.elastic.co/u/harry_richard)\ **Replies:** 1\ **Last updated:** [May 6, 2021, 2:42pm UTC](https://discuss.elastic.co/t/text-embedding-in-elastic-search/272242 "2021-05-06T14:42:36Z")
Hi All, I have implemented text embedding using doc 2 vec and stored the vectors in elastic search , using cosine similarity I am trying to search for similar texts. But the issue is whenver I am giving a same text twi… --- ## [Filebeat dissect line break](https://discuss.elastic.co/t/filebeat-dissect-line-break/272274)
**Author:** [@mostpha456](https://discuss.elastic.co/u/mostpha456)\ **Replies:** 6\ **Last updated:** [May 6, 2021, 2:06pm UTC](https://discuss.elastic.co/t/filebeat-dissect-line-break/272274 "2021-05-06T14:06:11Z")
Hello, i am using dissect processor to parse a multiline log. i got the error dissect\_parsing\_error, i think it s because of the \\n. Do you have any idea where i can find any exemple of filebeat dissect for multiline. … --- ## [Cannot read property 'get' of undefined](https://discuss.elastic.co/t/cannot-read-property-get-of-undefined/272199)
**Author:** [@Aqib\_Rehman](https://discuss.elastic.co/u/Aqib_Rehman)\ **Replies:** 3\ **Last updated:** [May 6, 2021, 1:47pm UTC](https://discuss.elastic.co/t/cannot-read-property-get-of-undefined/272199 "2021-05-06T13:47:12Z")
Hi, I'm using Suricata to collect logs on Kibana, after connecting Suricata with Elastic via filebeat, Kibana cannot visualize logs with following error: Cannot read property 'get' of undefined. It receives data but cann… --- ## [Metricbeat in Kubernetes only get processes from the container and not the host (Windows)](https://discuss.elastic.co/t/metricbeat-in-kubernetes-only-get-processes-from-the-container-and-not-the-host-windows/272128)
**Author:** [@jproulx](https://discuss.elastic.co/u/jproulx)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 1:22pm UTC](https://discuss.elastic.co/t/metricbeat-in-kubernetes-only-get-processes-from-the-container-and-not-the-host-windows/272128 "2021-05-06T13:22:14Z")
Context I am trying to setup metricbeat 7.12 as a DaemonSet in an Azure Kubernetes Cluster on a Nano Windows Server host. I based my Kubernetes config of metricbeat on this document. In this document, the ConfigMap nam… --- ## [JSON filter swallowing events with no errors](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221)
**Author:** [@brian\_m](https://discuss.elastic.co/u/brian_m)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 1:14pm UTC](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221 "2021-05-06T13:14:45Z")
Hello, I'm trying to do some testing on ingesting JSON data, and having trouble with the JSON filter. My pipeline configuration is below (output filter has been modified to remove credentials and IPs) input { http {}… --- ## [Variable content from file](https://discuss.elastic.co/t/variable-content-from-file/272296)
**Author:** [@emilsvil](https://discuss.elastic.co/u/emilsvil)\ **Replies:** 0\ **Last updated:** [May 6, 2021, 12:50pm UTC](https://discuss.elastic.co/t/variable-content-from-file/272296 "2021-05-06T12:50:46Z")
Hi. Im using cidr filter to check if an IP si public or private. The list of cidrs to check is now hardcoded in the filter but I need to read it from a file or using a meta-variable loaded at runtime. cidr { … --- ## [Getting single line from multiline](https://discuss.elastic.co/t/getting-single-line-from-multiline/272292)
**Author:** [@Learn\_Mulesoft\_With](https://discuss.elastic.co/u/Learn_Mulesoft_With)\ **Replies:** 0\ **Last updated:** [May 6, 2021, 12:35pm UTC](https://discuss.elastic.co/t/getting-single-line-from-multiline/272292 "2021-05-06T12:35:32Z")
hi , my message is like below Error type : VALIDATION:INVALID\_NUMBER FlowStack : at db-error-logFlow(db-error-logFlow/processors/0 @ db-error-log:db-error-log.xml:18 i want just single line " E… --- ## [Configure Open ID connect using Azure AD roles](https://discuss.elastic.co/t/configure-open-id-connect-using-azure-ad-roles/272137)
**Author:** [@nityaraj06](https://discuss.elastic.co/u/nityaraj06)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 12:17pm UTC](https://discuss.elastic.co/t/configure-open-id-connect-using-azure-ad-roles/272137 "2021-05-06T12:17:19Z")
Hi, Is it possible to map the azure roles in kibana role mapping since my organisation does not allow azure groups to be enabled in the token response. The only possible fields seem to be username, groups, realm and dn.… --- ## [Last record for each unique value](https://discuss.elastic.co/t/last-record-for-each-unique-value/271077)
**Author:** [@JeremyLoyd](https://discuss.elastic.co/u/JeremyLoyd)\ **Replies:** 2\ **Last updated:** [May 6, 2021, 11:11am UTC](https://discuss.elastic.co/t/last-record-for-each-unique-value/271077 "2021-05-06T11:11:15Z")
Hello, Our company changed to Elastic few months ago. before then we used to log a lot of things to one simple SQL table. in SQL i was able to find for each unique identifier last value that was recorded with simple Que… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1425) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1427)