# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1446 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1447 --- ## [Element involves results from two KQLs?](https://discuss.elastic.co/t/element-involves-results-from-two-kqls/270915)
**Author:** [@tangkalo](https://discuss.elastic.co/u/tangkalo)\ **Replies:** 4\ **Last updated:** [April 25, 2021, 7:18pm UTC](https://discuss.elastic.co/t/element-involves-results-from-two-kqls/270915 "2021-04-25T19:18:30Z")
Hi, in Kibana Canvas, I want to have an image element that changes color according to results from two separate KQLs. Is it possible? If not, any suggestions on how I may be able to work around this limitation? Thanks a… --- ## [Split of JSON array into multiple events in Kibana](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018)
**Author:** [@pedro\_gonzalez](https://discuss.elastic.co/u/pedro_gonzalez)\ **Replies:** 7\ **Last updated:** [April 25, 2021, 5:55pm UTC](https://discuss.elastic.co/t/split-of-json-array-into-multiple-events-in-kibana/271018 "2021-04-25T17:55:28Z")
Hi, I am trying to split into different events (logs) this log schema: "\_index": "cocacola", "\_type": "raw", "\_id": "CqwJ63MBEQS11DmXsDyRZl", "\_score": 1, "\_source": { … --- ## [Filebeat 7.12 event collection rate drops after a period of time](https://discuss.elastic.co/t/filebeat-7-12-event-collection-rate-drops-after-a-period-of-time/271157)
**Author:** [@bhavaniprasad\_reddy](https://discuss.elastic.co/u/bhavaniprasad_reddy)\ **Replies:** 1\ **Last updated:** [April 24, 2021, 5:42pm UTC](https://discuss.elastic.co/t/filebeat-7-12-event-collection-rate-drops-after-a-period-of-time/271157 "2021-04-24T17:42:59Z")
I have deployed filebeat 7.12 on a kubernetes cluster for collecting log events. The deployment was smooth and started collecting events as expected. But after a day, after collecting 50K + events the number of events t… --- ## [Not sure if a field is nill, null or empty. Help please](https://discuss.elastic.co/t/not-sure-if-a-field-is-nill-null-or-empty-help-please/271123)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 4\ **Last updated:** [April 24, 2021, 6:55pm UTC](https://discuss.elastic.co/t/not-sure-if-a-field-is-nill-null-or-empty-help-please/271123 "2021-04-24T18:55:29Z")
Hi All, It's about barracuda FW log with the following entry +02:00 Security XXX Block: type=FWD|proto=TCP|srcIF=eth0|srcIP=0.0.0.0|srcPort=58570|srcMAC=00:00:00:00:00:00|dstIP=0.0.0.0|dstPort=3002|dstService=exlm-agen… --- ## [Filebeat AWS ,Azure Modules: Support Multiple Accounts](https://discuss.elastic.co/t/filebeat-aws-azure-modules-support-multiple-accounts/271177)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 1\ **Last updated:** [April 25, 2021, 1:46pm UTC](https://discuss.elastic.co/t/filebeat-aws-azure-modules-support-multiple-accounts/271177 "2021-04-25T13:46:47Z")
I have successfully started both the filebeat modules AWS and Azure for a single account. I want to know if Filebeat can support multiple accounts for AWS and Azure. I would like to achieve following scenarios. Get mul… --- ## [Is using "filter" inside "aggs" more efficient than "query" at the top level of an aggregation body?](https://discuss.elastic.co/t/is-using-filter-inside-aggs-more-efficient-than-query-at-the-top-level-of-an-aggregation-body/271153)
**Author:** [@mattwelke](https://discuss.elastic.co/u/mattwelke)\ **Replies:** 5\ **Last updated:** [April 25, 2021, 1:02pm UTC](https://discuss.elastic.co/t/is-using-filter-inside-aggs-more-efficient-than-query-at-the-top-level-of-an-aggregation-body/271153 "2021-04-25T13:02:02Z")
I'm working in a legacy code base and I think I've uncovered an inefficient way of doing a filter aggregation based on what I read in the docs. These are my assumptions: Using filters to eliminate documents from searc… --- ## [Non-nested query on copy\_to fields performs worse then a nested query](https://discuss.elastic.co/t/non-nested-query-on-copy-to-fields-performs-worse-then-a-nested-query/271186)
**Author:** [@Ron\_lusha](https://discuss.elastic.co/u/Ron_lusha)\ **Replies:** 0\ **Last updated:** [April 25, 2021, 11:50am UTC](https://discuss.elastic.co/t/non-nested-query-on-copy-to-fields-performs-worse-then-a-nested-query/271186 "2021-04-25T11:50:37Z")
Our service uses a lot of nested queries/data, in order to improve our response time I tried to rewrite our heaviest query and use copy\_to to copy the fields I need from inside the nested docs to the root doc and query t… --- ## [Cluster Health Events FROM Elastic TO Arcsight](https://discuss.elastic.co/t/cluster-health-events-from-elastic-to-arcsight/271185)
**Author:** [@yaharin\_ben\_ayon](https://discuss.elastic.co/u/yaharin_ben_ayon)\ **Replies:** 0\ **Last updated:** [April 25, 2021, 11:30am UTC](https://discuss.elastic.co/t/cluster-health-events-from-elastic-to-arcsight/271185 "2021-04-25T11:30:41Z")
Hello, is there a way to import the cluster health logs or any logs FROM Elastic TO Arcsight with the Logstash Arcsight module or other way? Thank you! --- ## [Aggregation on latest n records](https://discuss.elastic.co/t/aggregation-on-latest-n-records/271169)
**Author:** [@behzad\_dadashpour](https://discuss.elastic.co/u/behzad_dadashpour)\ **Replies:** 2\ **Last updated:** [April 25, 2021, 9:32am UTC](https://discuss.elastic.co/t/aggregation-on-latest-n-records/271169 "2021-04-25T09:32:14Z")
Hi Is there any way to do aggregation on latest N records? this solution didn't work { "query": {...}, "size": N, "order": ..., "aggs": { .... } } for more detail: I want to get last 10 records … --- ## [How to delete by query](https://discuss.elastic.co/t/how-to-delete-by-query/271170)
**Author:** [@Champion\_Xie](https://discuss.elastic.co/u/Champion_Xie)\ **Replies:** 1\ **Last updated:** [April 25, 2021, 9:15am UTC](https://discuss.elastic.co/t/how-to-delete-by-query/271170 "2021-04-25T09:15:05Z")
how to delete by query current index\_name :logstash-2021.04.23 actually Data entered the index of the current timestamp because of index definition error. How to delete contain 2021.04.22 in the index(logstash-2021.04.… --- ## [Perform a clear cache operation with no significant change in memory](https://discuss.elastic.co/t/perform-a-clear-cache-operation-with-no-significant-change-in-memory/271165)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 4\ **Last updated:** [April 25, 2021, 9:10am UTC](https://discuss.elastic.co/t/perform-a-clear-cache-operation-with-no-significant-change-in-memory/271165 "2021-04-25T09:10:55Z")
I executed the following request to clear the cache, but when I look at the percentage of memory used by the process on top, there is no change。What is the reason for this? curl -XPOST 'http://172.18.141.39:19200/\_cache… --- ## [Filebeat normal start but no update log to elasticsearch](https://discuss.elastic.co/t/filebeat-normal-start-but-no-update-log-to-elasticsearch/270733)
**Author:** [@WhyStart](https://discuss.elastic.co/u/WhyStart)\ **Replies:** 3\ **Last updated:** [April 25, 2021, 6:32am UTC](https://discuss.elastic.co/t/filebeat-normal-start-but-no-update-log-to-elasticsearch/270733 "2021-04-25T06:32:17Z")
I installed filebeta on macos localhost，elk installed on docker desktop for mac,start filebeat log normal output,doesn't seem to have any problems,but it is not transmitted to elasticsearch. filebeat.yml # ============… --- ## [Logstash cannot use docker metadata to build a dynamic index](https://discuss.elastic.co/t/logstash-cannot-use-docker-metadata-to-build-a-dynamic-index/271032)
**Author:** [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Replies:** 3\ **Last updated:** [April 25, 2021, 6:23am UTC](https://discuss.elastic.co/t/logstash-cannot-use-docker-metadata-to-build-a-dynamic-index/271032 "2021-04-25T06:23:05Z")
I use filebeat to collect docker logs and add a label when the docker container starts. - type: docker enabled: true containers.ids: - '\*' tail\_files: true The following configuration is made in logstash … --- ## [How to filter when dynamic index has no value](https://discuss.elastic.co/t/how-to-filter-when-dynamic-index-has-no-value/271166)
**Author:** [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Replies:** 3\ **Last updated:** [April 25, 2021, 6:19am UTC](https://discuss.elastic.co/t/how-to-filter-when-dynamic-index-has-no-value/271166 "2021-04-25T06:19:24Z")
I use the label in the docker metadata to build an index dynamically. When the label of some containers does not exist or the value is empty, I get two indexes at the same time. How can I solve this problem Logstash c… --- ## [Should A Full DLQ Queue Effect Logstash Performance Signifigantly?](https://discuss.elastic.co/t/should-a-full-dlq-queue-effect-logstash-performance-signifigantly/271036)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 7\ **Last updated:** [April 25, 2021, 3:49am UTC](https://discuss.elastic.co/t/should-a-full-dlq-queue-effect-logstash-performance-signifigantly/271036 "2021-04-25T03:49:08Z")
I'm sending logs from Filebeat -\> Logstash -\> ES with 7.10.2. I have enabled DLQ and set it to 50 MB's. Once something is in the DLQ I do not bother reprocessing it and I'm fine with the data loss. Once I start seeing… --- ## [Table column value update into existing index(pdf ingest) column](https://discuss.elastic.co/t/table-column-value-update-into-existing-index-pdf-ingest-column/271162)
**Author:** [@joseph-l.amalraj](https://discuss.elastic.co/u/joseph-l.amalraj)\ **Replies:** 0\ **Last updated:** [April 25, 2021, 1:40am UTC](https://discuss.elastic.co/t/table-column-value-update-into-existing-index-pdf-ingest-column/271162 "2021-04-25T01:40:23Z")
Hi, I am new to elasticsearch. I have set of pdf files, i did ingested through FSCrawler(index=pdfvalitate). so, i got details in ElasticSearch. Now, i have a Table in mySQL TMPLT\_META(tmpl\_id, eff\_frm\_date, doc\_number),… --- ## [How do you increase the GB storage without increasing the RAM and CPU. Is there a way?](https://discuss.elastic.co/t/how-do-you-increase-the-gb-storage-without-increasing-the-ram-and-cpu-is-there-a-way/271146)
**Author:** [@Yash\_Srivastava](https://discuss.elastic.co/u/Yash_Srivastava)\ **Replies:** 3\ **Last updated:** [April 24, 2021, 7:41pm UTC](https://discuss.elastic.co/t/how-do-you-increase-the-gb-storage-without-increasing-the-ram-and-cpu-is-there-a-way/271146 "2021-04-24T19:41:56Z")
how do you increase the GB storage without increasing the RAM and CPU. Is there a way? --- ## [Filebeat sonicwall module not parsing correctly many messages](https://discuss.elastic.co/t/filebeat-sonicwall-module-not-parsing-correctly-many-messages/268232)
**Author:** [@Rodrigo\_Bernardo](https://discuss.elastic.co/u/Rodrigo_Bernardo)\ **Replies:** 6\ **Last updated:** [April 24, 2021, 6:28pm UTC](https://discuss.elastic.co/t/filebeat-sonicwall-module-not-parsing-correctly-many-messages/268232 "2021-04-24T18:28:01Z")
Hello there, Our Sonicwall generates this message: \<110\> id=yyyy sn=C0EAE4F9FB00 time="2021-03-24 14:35:31 UTC" fw=xx.xx.xx.xx pri=6 c=262144 m=98 msg="Connection Opened" app=49177 appName="General HTTPS" n=2185104… --- ## [Get previous docs and following docs at innerHits](https://discuss.elastic.co/t/get-previous-docs-and-following-docs-at-innerhits/271156)
**Author:** [@Benjamin\_S](https://discuss.elastic.co/u/Benjamin_S)\ **Replies:** 0\ **Last updated:** [April 24, 2021, 4:29pm UTC](https://discuss.elastic.co/t/get-previous-docs-and-following-docs-at-innerhits/271156 "2021-04-24T16:29:44Z")
Hello, I have an index for comments. Comments can also get sub-comments. If I search my InnerHits query for the most popular one (most likes etc). Can I somehow get the information at which position this comment is. So … --- ## [Data transfer rates costs in ElasticCloud direct purchased and Azure Marketplace enabled purchase](https://discuss.elastic.co/t/data-transfer-rates-costs-in-elasticcloud-direct-purchased-and-azure-marketplace-enabled-purchase/271145)
**Author:** [@Yash\_Srivastava](https://discuss.elastic.co/u/Yash_Srivastava)\ **Replies:** 1\ **Last updated:** [April 24, 2021, 3:10pm UTC](https://discuss.elastic.co/t/data-transfer-rates-costs-in-elasticcloud-direct-purchased-and-azure-marketplace-enabled-purchase/271145 "2021-04-24T15:10:17Z")
data transfer rates costs in ElasticCloud direct purchased and Azure Marketplace enabled purchase --- ## [Kibana alerts: Calculate percentage of errors for a given period time](https://discuss.elastic.co/t/kibana-alerts-calculate-percentage-of-errors-for-a-given-period-time/271133)
**Author:** [@Rahul\_Miragi](https://discuss.elastic.co/u/Rahul_Miragi)\ **Replies:** 1\ **Last updated:** [April 24, 2021, 1:59pm UTC](https://discuss.elastic.co/t/kibana-alerts-calculate-percentage-of-errors-for-a-given-period-time/271133 "2021-04-24T13:59:20Z")
I’m looking for a solution to following problem: In ELK, I have to create an alert which will trigger - when an error field logged for an index is greater than 80% in last 30 mins The field name is status and it takes i… --- ## [How to Get all the Lens visualization using Rest API call](https://discuss.elastic.co/t/how-to-get-all-the-lens-visualization-using-rest-api-call/270987)
**Author:** [@Sagesh](https://discuss.elastic.co/u/Sagesh)\ **Replies:** 2\ **Last updated:** [April 24, 2021, 11:41am UTC](https://discuss.elastic.co/t/how-to-get-all-the-lens-visualization-using-rest-api-call/270987 "2021-04-24T11:41:52Z")
How to Get all the Lens visualization using Rest API call? --- ## [Automatic dashboards are not getting created when I change index name in filebeat.yml](https://discuss.elastic.co/t/automatic-dashboards-are-not-getting-created-when-i-change-index-name-in-filebeat-yml/271086)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 6\ **Last updated:** [April 24, 2021, 11:39am UTC](https://discuss.elastic.co/t/automatic-dashboards-are-not-getting-created-when-i-change-index-name-in-filebeat-yml/271086 "2021-04-24T11:39:08Z")
Hii.I am ingesting netflow data using filebeat netflow module.I have configured "filebeat.yml" file for automatic creation of dashboards.Filebeat creates index with name "filebeat-\*" and it also creates dashboards auto… --- ## [Filebeat & Windows DHCP Logging](https://discuss.elastic.co/t/filebeat-windows-dhcp-logging/271017)
**Author:** [@billkindle](https://discuss.elastic.co/u/billkindle)\ **Replies:** 1\ **Last updated:** [April 24, 2021, 9:45am UTC](https://discuss.elastic.co/t/filebeat-windows-dhcp-logging/271017 "2021-04-24T09:45:36Z")
I'm trying to locate some examples that are specific to elastic cloud when using filebeat to capture Microsoft DHCP logs. All the examples I find online aren't really matching what I'm seeing in 7.12.0 example files, or … --- ## [Monitoring Sync gateway using ELK](https://discuss.elastic.co/t/monitoring-sync-gateway-using-elk/271134)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 0\ **Last updated:** [April 24, 2021, 9:24am UTC](https://discuss.elastic.co/t/monitoring-sync-gateway-using-elk/271134 "2021-04-24T09:24:15Z")
Hi all, I am monitoring couchbase database using couchbase module of metricbeat. is there any way to monitor sync gateway ? Regards --- ## [java.io.StreamCorruptedException: invalid internal transport message format](https://discuss.elastic.co/t/java-io-streamcorruptedexception-invalid-internal-transport-message-format/269899)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 6\ **Last updated:** [April 24, 2021, 9:21am UTC](https://discuss.elastic.co/t/java-io-streamcorruptedexception-invalid-internal-transport-message-format/269899 "2021-04-24T09:21:45Z")
Hi all, I have a three node ES cluster with IP (10.0.1.10,10.0.1.11,10.0.1.12), following WARN has been found in the log. in the following log, a remote address has been mentioned which is not a member of cluster (10.0… --- ## [Can not add a node](https://discuss.elastic.co/t/can-not-add-a-node/271131)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 2\ **Last updated:** [April 24, 2021, 9:18am UTC](https://discuss.elastic.co/t/can-not-add-a-node/271131 "2021-04-24T09:18:24Z")
Hi there Unfortunately, I can not add another node to my single node cluster. The elasticsearch.yml of master node is: cluster.name: logServer node.name: elk1 node.master: true node.data: true path.data: /var/lib/elas… --- ## [(urgent) Converting a list of decimals to float in CSV](https://discuss.elastic.co/t/urgent-converting-a-list-of-decimals-to-float-in-csv/268772)
**Author:** [@shafeeka](https://discuss.elastic.co/u/shafeeka)\ **Replies:** 4\ **Last updated:** [April 24, 2021, 9:14am UTC](https://discuss.elastic.co/t/urgent-converting-a-list-of-decimals-to-float-in-csv/268772 "2021-04-24T09:14:11Z")
mutate { convert =\> { "Question\_Vector" =\> "float\_eu" } I am trying to push a CSV file to elasticsearch using logstash. One of the column contains a list of vectors and I am trying to conv… --- ## [Connecting filebeat to elastic cloud](https://discuss.elastic.co/t/connecting-filebeat-to-elastic-cloud/270788)
**Author:** [@Richard\_Phillips\_Roy](https://discuss.elastic.co/u/Richard_Phillips_Roy)\ **Replies:** 4\ **Last updated:** [April 24, 2021, 5:07am UTC](https://discuss.elastic.co/t/connecting-filebeat-to-elastic-cloud/270788 "2021-04-24T05:07:16Z")
Hi, I would i configure the output of this filebeat.yml file to send logs to elastic cloud filebeat.inputs: - enabled: true paths: - /data/fatt/log/fatt.log fields: type: Fatt fields\_under\_root… --- ## [Watcher that counts the documents that arrive to an index in kibana](https://discuss.elastic.co/t/watcher-that-counts-the-documents-that-arrive-to-an-index-in-kibana/270609)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 45\ **Last updated:** [April 24, 2021, 2:06am UTC](https://discuss.elastic.co/t/watcher-that-counts-the-documents-that-arrive-to-an-index-in-kibana/270609 "2021-04-24T02:06:21Z")
I have a problem with this watcher, I need to get the number of samples of an index, that is, the count of the documents sent by heartbeat to kibana and send an email every day with the number of documents that arrived t… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1445) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1447)