# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1223 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1224 --- ## [Logstash S3 Output plugin](https://discuss.elastic.co/t/logstash-s3-output-plugin/285276)
**Author:** [@John\_snow](https://discuss.elastic.co/u/John_snow)\ **Replies:** 0\ **Last updated:** [September 27, 2021, 7:58pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin/285276 "2021-09-27T19:58:25Z")
Hey all, Trying to send messages to S3 USING Logstash SQS as inout, but getting message in this format %{host} %{message} Also not able to see added filed in the logs. --- ## [How to use regex with vega](https://discuss.elastic.co/t/how-to-use-regex-with-vega/285237)
**Author:** [@forst](https://discuss.elastic.co/u/forst)\ **Replies:** 4\ **Last updated:** [September 27, 2021, 7:17pm UTC](https://discuss.elastic.co/t/how-to-use-regex-with-vega/285237 "2021-09-27T19:17:16Z")
I am new to vega lite / vega. I'm trying to write a regex query for a java log error as following with vega tree layout. A field 'message' contains : \[Poolthread\] com.xxxx.content.core-bundle com.xxxxx.content.model.… --- ## [Commenataries in kibana](https://discuss.elastic.co/t/commenataries-in-kibana/285198)
**Author:** [@ippo](https://discuss.elastic.co/u/ippo)\ **Replies:** 1\ **Last updated:** [September 27, 2021, 6:47pm UTC](https://discuss.elastic.co/t/commenataries-in-kibana/285198 "2021-09-27T18:47:44Z")
Hello guys i wanted to ask you if there is a way to add commentaries in a column to an enhanced table in kibana or if you think there is a way to implement this idea. any idea could help :pray: --- ## [Using Multiline for individual/specific Modules](https://discuss.elastic.co/t/using-multiline-for-individual-specific-modules/285264)
**Author:** [@User001](https://discuss.elastic.co/u/User001)\ **Replies:** 0\ **Last updated:** [September 27, 2021, 5:47pm UTC](https://discuss.elastic.co/t/using-multiline-for-individual-specific-modules/285264 "2021-09-27T17:47:39Z")
Hello, I am curious if you can set/override the multiline parameters within individual modules. I am trying to use a commonly configured filebeat instance within a larger project, of which, they currently and/or may not… --- ## [Configurable visualization](https://discuss.elastic.co/t/configurable-visualization/284990)
**Author:** [@PaulBuciuman](https://discuss.elastic.co/u/PaulBuciuman)\ **Replies:** 3\ **Last updated:** [September 27, 2021, 5:33pm UTC](https://discuss.elastic.co/t/configurable-visualization/284990 "2021-09-27T17:33:04Z")
I'm create a simple visualization that shows using a treemap, top 20 request times from an API. But I want that 20 to be configurable outside the settings page, similar to the way I can customize the time range. Is this … --- ## [Querying a list to check the size](https://discuss.elastic.co/t/querying-a-list-to-check-the-size/284969)
**Author:** [@eztask.io](https://discuss.elastic.co/u/eztask.io)\ **Replies:** 3\ **Last updated:** [September 27, 2021, 5:32pm UTC](https://discuss.elastic.co/t/querying-a-list-to-check-the-size/284969 "2021-09-27T17:32:35Z")
Hello, I am just new to Kibana query. I am trying to query a field, which is a list of objects "leadStages" : \[ { "leadStage" : 1, "inTime" : 1597832744832, "outTime… --- ## [Kibana scripted field: doc\['some\_field'\] doesnt work when the field has a lot of data](https://discuss.elastic.co/t/kibana-scripted-field-doc-some-field-doesnt-work-when-the-field-has-a-lot-of-data/285106)
**Author:** [@Felipsz](https://discuss.elastic.co/u/Felipsz)\ **Replies:** 3\ **Last updated:** [September 27, 2021, 5:31pm UTC](https://discuss.elastic.co/t/kibana-scripted-field-doc-some-field-doesnt-work-when-the-field-has-a-lot-of-data/285106 "2021-09-27T17:31:06Z")
Hi everyone, Im creating some scripted fields to improve the visualization of the data that comes into Kibana 7.5. I have a big field that its a JSON, and it has a lot of data, and i want to separate this data in multi… --- ## [Enable xpack on a running Elastic cluster](https://discuss.elastic.co/t/enable-xpack-on-a-running-elastic-cluster/284671)
**Author:** [@ls4074](https://discuss.elastic.co/u/ls4074)\ **Replies:** 9\ **Last updated:** [September 27, 2021, 5:28pm UTC](https://discuss.elastic.co/t/enable-xpack-on-a-running-elastic-cluster/284671 "2021-09-27T17:28:59Z")
I deployed Elastic using helm chart (7.13) without xpack enabled. The ES and Kibana pods are running fine. I logged in to one of the master pods and ran the command "Elasticsearch-certutil" to generate the certs and the… --- ## [ElasticSearch crashing - Magento 2.4.3](https://discuss.elastic.co/t/elasticsearch-crashing-magento-2-4-3/284228)
**Author:** [@pmono](https://discuss.elastic.co/u/pmono)\ **Replies:** 24\ **Last updated:** [September 27, 2021, 4:42pm UTC](https://discuss.elastic.co/t/elasticsearch-crashing-magento-2-4-3/284228 "2021-09-27T16:42:39Z")
We're having issues with Elasticsearch crashing from time to time. It also sometimes spikes up RAM + CPU and server becomes unresponsive. We have left most of the settings as is, but had to add more RAM to JVM heap (48G… --- ## [How to migrate data from db oracle to ES use Logstash?](https://discuss.elastic.co/t/how-to-migrate-data-from-db-oracle-to-es-use-logstash/285248)
**Author:** [@NinhNM](https://discuss.elastic.co/u/NinhNM)\ **Replies:** 1\ **Last updated:** [September 27, 2021, 4:37pm UTC](https://discuss.elastic.co/t/how-to-migrate-data-from-db-oracle-to-es-use-logstash/285248 "2021-09-27T16:37:28Z")
I have around 1 millions record in db oracle and I want migrate to ES. Logstash: input { jdbc { jdbc\_driver\_library =\> "/Users/ninhnguyen/Desktop/ojdbc7.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.Ora… --- ## [Creating choropleth maps using only geo locations](https://discuss.elastic.co/t/creating-choropleth-maps-using-only-geo-locations/284995)
**Author:** [@Daniil\_Naumov](https://discuss.elastic.co/u/Daniil_Naumov)\ **Replies:** 4\ **Last updated:** [September 27, 2021, 4:29pm UTC](https://discuss.elastic.co/t/creating-choropleth-maps-using-only-geo-locations/284995 "2021-09-27T16:29:19Z")
How can I visualise requests on choropleth map if I have only location data in logs? --- ## [Filebeat slows down as registry file grows](https://discuss.elastic.co/t/filebeat-slows-down-as-registry-file-grows/284903)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 2\ **Last updated:** [September 27, 2021, 4:22pm UTC](https://discuss.elastic.co/t/filebeat-slows-down-as-registry-file-grows/284903 "2021-09-27T16:22:40Z")
Filebeat version: 7.9.2 OS: Ubuntu 18.04.5 LTS VM: 16 CPU, 64 GB RAM Output: Kafka I have an issue with Filebeat's throughput. There are about 600 files are created every minute and those have around 4000 lines in th… --- ## [There is no match between one term and another in the result of a watcher](https://discuss.elastic.co/t/there-is-no-match-between-one-term-and-another-in-the-result-of-a-watcher/285254)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 2\ **Last updated:** [September 27, 2021, 3:53pm UTC](https://discuss.elastic.co/t/there-is-no-match-between-one-term-and-another-in-the-result-of-a-watcher/285254 "2021-09-27T15:53:49Z")
very good day, I have a problem with an alert when I get a result of several terms, i.e. the first term does not match the second, for example. As you can see in the red circle the term "ADUANILLA DE PAIBA" has the … --- ## [Unable to load Kibana UI from Internet Exlporer](https://discuss.elastic.co/t/unable-to-load-kibana-ui-from-internet-exlporer/284118)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 5\ **Last updated:** [September 27, 2021, 3:49pm UTC](https://discuss.elastic.co/t/unable-to-load-kibana-ui-from-internet-exlporer/284118 "2021-09-27T15:49:50Z")
I have completed installing Kibana, Logstash and ElasticStack all versions 7.13.4 (In that order) in the production environment. Each portion of the stack is installed on it's own Red Hat Enterprise 8 server, and Elasti… --- ## [Can I create a Data table which aggregates a top hit per ID?](https://discuss.elastic.co/t/can-i-create-a-data-table-which-aggregates-a-top-hit-per-id/285089)
**Author:** [@Joao\_Silva](https://discuss.elastic.co/u/Joao_Silva)\ **Replies:** 2\ **Last updated:** [September 27, 2021, 2:58pm UTC](https://discuss.elastic.co/t/can-i-create-a-data-table-which-aggregates-a-top-hit-per-id/285089 "2021-09-27T14:58:21Z")
Imagine that I have documents with the fields ID, ValueA, ValueB and that my index for each ID, let's say from 1 to 50 have some random values for A and B. Is it possible to build a visualization with 50 rows, one per I… --- ## [Detecting DNS Exfiltration](https://discuss.elastic.co/t/detecting-dns-exfiltration/285239)
**Author:** [@ksremo](https://discuss.elastic.co/u/ksremo)\ **Replies:** 1\ **Last updated:** [September 27, 2021, 2:47pm UTC](https://discuss.elastic.co/t/detecting-dns-exfiltration/285239 "2021-09-27T14:47:56Z")
Hello, we ingest dns logs into Elasticsearch. the format is domain -\> whole domain -\> deeper.cdn.example.com domain1 -\> com domain2 -\> example domain3 -\> cdn domain4 -\> deeper Now we want to create a siem detecti… --- ## [How to calculate the average count with term](https://discuss.elastic.co/t/how-to-calculate-the-average-count-with-term/284879)
**Author:** [@forst](https://discuss.elastic.co/u/forst)\ **Replies:** 7\ **Last updated:** [September 27, 2021, 2:31pm UTC](https://discuss.elastic.co/t/how-to-calculate-the-average-count-with-term/284879 "2021-09-27T14:31:44Z")
I am looking to make a heatmap type visualization, I wanted to put the time on x-axis and the team (name) on y-axis. a team has a number of different jobs I wanted to put the average count with : count total / count j… --- ## [Elastic Stack Mapping Error](https://discuss.elastic.co/t/elastic-stack-mapping-error/284871)
**Author:** [@dayajamal](https://discuss.elastic.co/u/dayajamal)\ **Replies:** 9\ **Last updated:** [September 27, 2021, 2:15pm UTC](https://discuss.elastic.co/t/elastic-stack-mapping-error/284871 "2021-09-27T14:15:13Z")
Hello, this my second time posting here with a hope to find a solution this time. I'm working on the Elastic Stack. I'm sending logs from Filebeat through Logstash to Elasticsearch. My logs get structured in Logstash, no… --- ## [Canvas - Values retrieved from ESQL aren't the same as the ones displayed in chart](https://discuss.elastic.co/t/canvas-values-retrieved-from-esql-arent-the-same-as-the-ones-displayed-in-chart/284640)
**Author:** [@Luis\_P](https://discuss.elastic.co/u/Luis_P)\ **Replies:** 6\ **Last updated:** [September 27, 2021, 2:13pm UTC](https://discuss.elastic.co/t/canvas-values-retrieved-from-esql-arent-the-same-as-the-ones-displayed-in-chart/284640 "2021-09-27T14:13:03Z")
I made a query which produces the following table: I've cropped out the other column, but it represents the processes I have documents on, being NrByProcess the count of documents for each process. However, when using… --- ## [Logstash not processing syslog data (logs not arriving in Elastic)](https://discuss.elastic.co/t/logstash-not-processing-syslog-data-logs-not-arriving-in-elastic/285177)
**Author:** [@hazenvs](https://discuss.elastic.co/u/hazenvs)\ **Replies:** 0\ **Last updated:** [September 26, 2021, 8:53pm UTC](https://discuss.elastic.co/t/logstash-not-processing-syslog-data-logs-not-arriving-in-elastic/285177 "2021-09-26T20:53:48Z")
Logstash is up and running: \[root@node1 logstash\]# systemctl status logstash.service ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled) Active: ac… --- ## [Cannot restore snapshot to new single node cluster](https://discuss.elastic.co/t/cannot-restore-snapshot-to-new-single-node-cluster/285025)
**Author:** [@cdalexndr](https://discuss.elastic.co/u/cdalexndr)\ **Replies:** 3\ **Last updated:** [September 27, 2021, 1:14pm UTC](https://discuss.elastic.co/t/cannot-restore-snapshot-to-new-single-node-cluster/285025 "2021-09-27T13:14:58Z")
I've made a snapshot for a single node cluster using default settings: PUT /\_snapshot/backup/%3Csnapshot-%7Bnow%2Fd%7D%3E Created a new single node cluster, and tried to restore it: POST /\_snapshot/backup/snapshot-202… --- ## [How many hosts I can list in logstash output](https://discuss.elastic.co/t/how-many-hosts-i-can-list-in-logstash-output/285093)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 6\ **Last updated:** [September 27, 2021, 1:00pm UTC](https://discuss.elastic.co/t/how-many-hosts-i-can-list-in-logstash-output/285093 "2021-09-27T13:00:04Z")
how many hosts I can list in output section of host part? can I use wildcard? something like this? hosts =\> \["host\_123\*"\] --- ## [Error fetching alert status Stack Monitoring](https://discuss.elastic.co/t/error-fetching-alert-status-stack-monitoring/283707)
**Author:** [@vikasgurlinka](https://discuss.elastic.co/u/vikasgurlinka)\ **Replies:** 1\ **Last updated:** [September 27, 2021, 12:51pm UTC](https://discuss.elastic.co/t/error-fetching-alert-status-stack-monitoring/283707 "2021-09-27T12:51:18Z")
We have been using Kibana 7.9.1 and when we open stack monitoring an error message saying Error fetching alert status is being popped up. But it seems the Stack monitoring displays all monitored data correctly (CPU, mem… --- ## [How to get e-mail notification with log file path og log file hostname](https://discuss.elastic.co/t/how-to-get-e-mail-notification-with-log-file-path-og-log-file-hostname/283890)
**Author:** [@amiraliw](https://discuss.elastic.co/u/amiraliw)\ **Replies:** 1\ **Last updated:** [September 27, 2021, 12:40pm UTC](https://discuss.elastic.co/t/how-to-get-e-mail-notification-with-log-file-path-og-log-file-hostname/283890 "2021-09-27T12:40:30Z")
Hi I am very new in elasticsearch and kibana. I have elasticsearch and kibana on the same on premise server. And filebeat agents are installed on several other servers (on premise). These agents send logs to elasticsea… --- ## [Upgrade Elasticsearch from 7.8.1 to 7.14.2](https://discuss.elastic.co/t/upgrade-elasticsearch-from-7-8-1-to-7-14-2/285217)
**Author:** [@Pete\_Watcharawit](https://discuss.elastic.co/u/Pete_Watcharawit)\ **Replies:** 4\ **Last updated:** [September 27, 2021, 10:33am UTC](https://discuss.elastic.co/t/upgrade-elasticsearch-from-7-8-1-to-7-14-2/285217 "2021-09-27T10:33:23Z")
Hello, I am having two issues with the upgrade from 7.8.1 to 7.14.2 I downloaded the tar.gz file for 7.14.2 and launched the Elasticsearch instance by running /bin/Elasticsearch. The instance updates existing indices a… --- ## [Packetbeat "dropping event, queue is blocked (seq=0)"](https://discuss.elastic.co/t/packetbeat-dropping-event-queue-is-blocked-seq-0/285219)
**Author:** [@guido.lamoto](https://discuss.elastic.co/u/guido.lamoto)\ **Replies:** 0\ **Last updated:** [September 27, 2021, 10:18am UTC](https://discuss.elastic.co/t/packetbeat-dropping-event-queue-is-blocked-seq-0/285219 "2021-09-27T10:18:54Z")
We have a Packetbeat 7.15.0 on Ubuntu 20.04 directly connected to a Debian 10 with Elasticsearch 7.14.1. Packetbeat is running on a a DNS server with an average of 4000 queries/s. This is the configuration file: packet… --- ## [Node Stats API in 7.15 broken?](https://discuss.elastic.co/t/node-stats-api-in-7-15-broken/285212)
**Author:** [@oliver](https://discuss.elastic.co/u/oliver)\ **Replies:** 2\ **Last updated:** [September 27, 2021, 10:12am UTC](https://discuss.elastic.co/t/node-stats-api-in-7-15-broken/285212 "2021-09-27T10:12:22Z")
Hello! In #75433 you added a shard count structure to the Node Stats API. That results in a new JSON key named shards. However, if you run the Node Stats API with level=shards, you already have a JSON key named shards. … --- ## [How to update values present in particular field for every docs in index using DSL query](https://discuss.elastic.co/t/how-to-update-values-present-in-particular-field-for-every-docs-in-index-using-dsl-query/284885)
**Author:** [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\ **Replies:** 1\ **Last updated:** [September 27, 2021, 9:02am UTC](https://discuss.elastic.co/t/how-to-update-values-present-in-particular-field-for-every-docs-in-index-using-dsl-query/284885 "2021-09-27T09:02:57Z")
Hi, I want to update a particular field in the index. Index contains 100 docs. I want to update field "name" with "condition\_system" in every doc where "Condition System" is present. GET part\_ther/\_search { "\_… --- ## [Elasticsearch install on MacOS USB stick fails - java.nio.channels.OverlappingFileLockException](https://discuss.elastic.co/t/elasticsearch-install-on-macos-usb-stick-fails-java-nio-channels-overlappingfilelockexception/285215)
**Author:** [@John\_Lewis](https://discuss.elastic.co/u/John_Lewis)\ **Replies:** 0\ **Last updated:** [September 27, 2021, 9:55am UTC](https://discuss.elastic.co/t/elasticsearch-install-on-macos-usb-stick-fails-java-nio-channels-overlappingfilelockexception/285215 "2021-09-27T09:55:55Z")
Running bin/Elasticsearch on 'tar -xf Elasticsearch-7.15.0-darwin-x86\_64.tar' install fails to start (base) ➜ Samsung\_T5 cd elasticsearch-7.15.0 (base) ➜ elasticsearch-7.15.0 pwd /Volumes/Samsung\_T5/elasticsearch-7.15… --- ## [No shard exception after logstash tries to index large data](https://discuss.elastic.co/t/no-shard-exception-after-logstash-tries-to-index-large-data/285193)
**Author:** [@yash\_chaudhary](https://discuss.elastic.co/u/yash_chaudhary)\ **Replies:** 4\ **Last updated:** [September 27, 2021, 9:43am UTC](https://discuss.elastic.co/t/no-shard-exception-after-logstash-tries-to-index-large-data/285193 "2021-09-27T09:43:49Z")
My goal is to migrate data from psql db to Elasticsearch. I am using jdbc plugin to achieve it. Issue: ES instance is breaking for larger db ( 700k records ) NOTE: For small db ( say 500 records ), it works perfectly … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1222) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1224)