# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=955 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 956 --- ## [Using a colon in a field name does not work?](https://discuss.elastic.co/t/using-a-colon-in-a-field-name-does-not-work/303143)
**Author:** [@johann-petrak](https://discuss.elastic.co/u/johann-petrak)\ **Replies:** 2\ **Last updated:** [May 3, 2022, 12:25am UTC](https://discuss.elastic.co/t/using-a-colon-in-a-field-name-does-not-work/303143 "2022-05-03T00:25:25Z")
I have the following as part of my mapping file: "entities.name1:Name2": { "type": "nested" }, But the field does not actually end up nested in the index. Is there a problem with the use of colons as part… --- ## [Grok pattern for date format](https://discuss.elastic.co/t/grok-pattern-for-date-format/302923)
**Author:** [@Garry](https://discuss.elastic.co/u/Garry)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 9:26pm UTC](https://discuss.elastic.co/t/grok-pattern-for-date-format/302923 "2022-05-02T21:26:39Z")
Hi, I am looking to ingest a custom set of logs and i need some help with a grok date format. I would like to convert the following date as the timestamp 19 April 2022 15:38:20 Any help would be appreciated. I have be… --- ## [Filebeat and Fortinet](https://discuss.elastic.co/t/filebeat-and-fortinet/303397)
**Author:** [@Christer\_Palmen](https://discuss.elastic.co/u/Christer_Palmen)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 8:56pm UTC](https://discuss.elastic.co/t/filebeat-and-fortinet/303397 "2022-05-02T20:56:49Z")
Blockquote Hello. Filebeat seems to have built in Index patterns and templates for some Fortinet products, which is good from my point of view. What I would like to do is to instead of using the default index that F… --- ## [GeoIP data not creating "location" field and duplicating data](https://discuss.elastic.co/t/geoip-data-not-creating-location-field-and-duplicating-data/303612)
**Author:** [@joshn](https://discuss.elastic.co/u/joshn)\ **Replies:** 7\ **Last updated:** [May 2, 2022, 8:29pm UTC](https://discuss.elastic.co/t/geoip-data-not-creating-location-field-and-duplicating-data/303612 "2022-05-02T20:29:39Z")
I'm ingesting logs from my firewall, and as part of that I thought it would be nice to look at geoip data. geoip { add\_tag =\> \[ "GeoIP" \] source =\> "src\_ip" } geoip { … --- ## [Logstash fine tuning for ingesting more events (s3 input)](https://discuss.elastic.co/t/logstash-fine-tuning-for-ingesting-more-events-s3-input/303681)
**Author:** [@antonisnyc94](https://discuss.elastic.co/u/antonisnyc94)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 7:48pm UTC](https://discuss.elastic.co/t/logstash-fine-tuning-for-ingesting-more-events-s3-input/303681 "2022-05-02T19:48:46Z")
Hello, We sending events for vpc flowlogs from multiple AWS accounts into a central s3 bucket and due to the large number of events we are always 5-6 days behind in Elasticsearch. I already set the batch.size to 6000 a… --- ## [How to get paid version of kibana (not cloud)](https://discuss.elastic.co/t/how-to-get-paid-version-of-kibana-not-cloud/303382)
**Author:** [@Rayhan\_Shaik](https://discuss.elastic.co/u/Rayhan_Shaik)\ **Replies:** 8\ **Last updated:** [May 2, 2022, 2:50pm UTC](https://discuss.elastic.co/t/how-to-get-paid-version-of-kibana-not-cloud/303382 "2022-05-02T14:50:02Z")
hi team, We are using the free version of kibana(not cloud) in our project, but now as per our project requirements ( automating the reports generation of dashboard and sending them over email ), we came to know that t… --- ## [Clean method for adding field when the values may be missing?](https://discuss.elastic.co/t/clean-method-for-adding-field-when-the-values-may-be-missing/303769)
**Author:** [@jbrowe](https://discuss.elastic.co/u/jbrowe)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 7:42pm UTC](https://discuss.elastic.co/t/clean-method-for-adding-field-when-the-values-may-be-missing/303769 "2022-05-02T19:42:28Z")
I have logs with variable field names. For example: {"field\_one":"first", "field\_two":"second"} ("field\_two":"second", "field\_three": "third"} To get the correct index mapping, I currently I use: mutate { add\_fi… --- ## [Retrieving DataStream list in Elasticsearch using python API](https://discuss.elastic.co/t/retrieving-datastream-list-in-elasticsearch-using-python-api/303705)
**Author:** [@curious\_ands](https://discuss.elastic.co/u/curious_ands)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 7:36pm UTC](https://discuss.elastic.co/t/retrieving-datastream-list-in-elasticsearch-using-python-api/303705 "2022-05-02T19:36:17Z")
In Kibana dev tools, When I am using the API call GET /\_data\_stream/ I am getting the list of DataStream. In similar way I want to retrieve it using the Elasticsearch Python API, I am not able to find the way. Can anyone… --- ## [Testing Logs in Logstash |Automation with Ansible](https://discuss.elastic.co/t/testing-logs-in-logstash-automation-with-ansible/303768)
**Author:** [@Amanda\_Ruzza](https://discuss.elastic.co/u/Amanda_Ruzza)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 7:24pm UTC](https://discuss.elastic.co/t/testing-logs-in-logstash-automation-with-ansible/303768 "2022-05-02T19:24:32Z")
Dear all, My team is currently working on deploying an ELK stack on AWS using Ansible and GitHub Actions. My question is: Is there a way to test that the the Log files sent from MetricBeats are being properly collecte… --- ## [Data node CPU constantly 90-98% on one node](https://discuss.elastic.co/t/data-node-cpu-constantly-90-98-on-one-node/302720)
**Author:** [@Damelon](https://discuss.elastic.co/u/Damelon)\ **Replies:** 5\ **Last updated:** [May 2, 2022, 5:46pm UTC](https://discuss.elastic.co/t/data-node-cpu-constantly-90-98-on-one-node/302720 "2022-05-02T17:46:22Z")
I currently have an Elasticsearch cluster (7.15.1) Which has 3 masters, 6 data nodes, 2 ingest nodes which is deployed onto Kubernetes in AWS. Each data node is using an m5.2xlarge AWS instance and a 1.5TB storage volu… --- ## [Offline Mapping](https://discuss.elastic.co/t/offline-mapping/303756)
**Author:** [@millie123](https://discuss.elastic.co/u/millie123)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 6:49pm UTC](https://discuss.elastic.co/t/offline-mapping/303756 "2022-05-02T18:49:39Z")
I'm trying to get offline maps working in Kibana, but running into problems. For context, I want a standard OSM style base map, similar to the default base map, that doesn’t require an internet connection. I’ve tried a c… --- ## [Alert config](https://discuss.elastic.co/t/alert-config/303697)
**Author:** [@inwoo1](https://discuss.elastic.co/u/inwoo1)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 6:25pm UTC](https://discuss.elastic.co/t/alert-config/303697 "2022-05-02T18:25:28Z")
Hello, I always get a lot of help. If you specify one host.name in the filter, the value of the host is applied (here, 87.5%), and you can set an alarm based on that number. . I'm trying to set an alarm when any ass… --- ## [Vega autosize does not work](https://discuss.elastic.co/t/vega-autosize-does-not-work/303745)
**Author:** [@karlanakamura](https://discuss.elastic.co/u/karlanakamura)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 6:18pm UTC](https://discuss.elastic.co/t/vega-autosize-does-not-work/303745 "2022-05-02T18:18:40Z")
Since I updated elastic to v8.1.2 autosize is not working. I'm using "autosize": {"type": "fit", "contains": "padding"}, but even if I increase or decrease the size of the chart on the dashboard, the scrollbar always app… --- ## [\[NEST\] Bulk Ingest Not Utilizing Max System Resources](https://discuss.elastic.co/t/nest-bulk-ingest-not-utilizing-max-system-resources/303638)
**Author:** [@cj\_hillbrand](https://discuss.elastic.co/u/cj_hillbrand)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 5:50pm UTC](https://discuss.elastic.co/t/nest-bulk-ingest-not-utilizing-max-system-resources/303638 "2022-05-02T17:50:23Z")
Hello Elastic Community, We have multiple environments set up for our Elasticsearch system, and I am experiencing some odd behavior. In the dev environment, when we perform a bulk update, the CPU utilization reaches to… --- ## [Lens: Top values bar diagram with percentage](https://discuss.elastic.co/t/lens-top-values-bar-diagram-with-percentage/303591)
**Author:** [@tomhe](https://discuss.elastic.co/u/tomhe)\ **Replies:** 5\ **Last updated:** [May 2, 2022, 5:09pm UTC](https://discuss.elastic.co/t/lens-top-values-bar-diagram-with-percentage/303591 "2022-05-02T17:09:35Z")
I would like to be able to see percentages for each bar in a bar graph for top values, just like a donut chart displays it. Is this planned to be added in the future or is the donut chart my only option if I want percent… --- ## [Logstash best practices for multiple sources](https://discuss.elastic.co/t/logstash-best-practices-for-multiple-sources/303722)
**Author:** [@fropa](https://discuss.elastic.co/u/fropa)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 4:27pm UTC](https://discuss.elastic.co/t/logstash-best-practices-for-multiple-sources/303722 "2022-05-02T16:27:21Z")
Hi folks, I'm starting to use ELK, I've multiple source servers, some of them have also multiple logs to send. Now I'm trying to set up logstash sample configuration to just receive and send logs to the elasticsearch I… --- ## [Packetbeat setup fail](https://discuss.elastic.co/t/packetbeat-setup-fail/303406)
**Author:** [@OBT](https://discuss.elastic.co/u/OBT)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 4:21pm UTC](https://discuss.elastic.co/t/packetbeat-setup-fail/303406 "2022-05-02T16:21:34Z")
im trying to use packetbeat but i have some problem about the packetbeat.yml can someone help me thx! --- ## [Using elasticsearch with remote storage](https://discuss.elastic.co/t/using-elasticsearch-with-remote-storage/303631)
**Author:** [@James\_Hsia](https://discuss.elastic.co/u/James_Hsia)\ **Replies:** 2\ **Last updated:** [May 2, 2022, 3:21pm UTC](https://discuss.elastic.co/t/using-elasticsearch-with-remote-storage/303631 "2022-05-02T15:21:13Z")
We have an AKS cluster in the US running an Elasticsearch pod (version 7.16.0) tied to an Azure File Share storage account in Europe. We are getting continuous issues in the Elasticsearch logs similar to the below upon … --- ## [Visualize unique count of IDs values by a date field](https://discuss.elastic.co/t/visualize-unique-count-of-ids-values-by-a-date-field/303616)
**Author:** [@H0tmilk](https://discuss.elastic.co/u/H0tmilk)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 2:51pm UTC](https://discuss.elastic.co/t/visualize-unique-count-of-ids-values-by-a-date-field/303616 "2022-05-02T14:51:31Z")
Hello, This is my data (for example) in Elasticsearch : { "id": 1, "status": "Active", "title": "Incident 1", "@timestamp": "28/04/22" }, { "id": 2, "status": "Active", "title": "Incident 2", "@timestamp": "28/… --- ## [I am trying to convert unix timestamp(string) coming from my dynamo db by using lambda function to date format to visualize in kibana](https://discuss.elastic.co/t/i-am-trying-to-convert-unix-timestamp-string-coming-from-my-dynamo-db-by-using-lambda-function-to-date-format-to-visualize-in-kibana/303622)
**Author:** [@Kedhar1006](https://discuss.elastic.co/u/Kedhar1006)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 2:35pm UTC](https://discuss.elastic.co/t/i-am-trying-to-convert-unix-timestamp-string-coming-from-my-dynamo-db-by-using-lambda-function-to-date-format-to-visualize-in-kibana/303622 "2022-05-02T14:35:44Z")
'use strict'; var moment = require('moment'); //var Elasticsearch = require('Elasticsearch'); //var client = new Elasticsearch.Clinet({ // host: 'https://.us-east-2.es.amazonaws.com', // log: 'error' //}); const… --- ## [Error changing certificate in Elasticsearch](https://discuss.elastic.co/t/error-changing-certificate-in-elasticsearch/303675)
**Author:** [@jorge.visentini](https://discuss.elastic.co/u/jorge.visentini)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 2:29pm UTC](https://discuss.elastic.co/t/error-changing-certificate-in-elasticsearch/303675 "2022-05-02T14:29:11Z")
I'm trying to change Elasticsearch SSL certificates but I'm getting some errors and it's not starting. I would like to change the CA and certificate to one generated by me, which I use for testing, to learn how to chang… --- ## [The label of kibanaAddFilter doesn't work](https://discuss.elastic.co/t/the-label-of-kibanaaddfilter-doesnt-work/301795)
**Author:** [@karlanakamura](https://discuss.elastic.co/u/karlanakamura)\ **Replies:** 7\ **Last updated:** [May 2, 2022, 1:50pm UTC](https://discuss.elastic.co/t/the-label-of-kibanaaddfilter-doesnt-work/301795 "2022-05-02T13:50:37Z")
Hi, I am using Vega to create an age pyramid and I would like to filter the data by clicking on a bar. Thus, I created a signal and used kibanaAddFilter to define the query, as shown in the code below. "signals": \[ {… --- ## [Logstash not receiving messages when started via docker-compose (Docker is ok)](https://discuss.elastic.co/t/logstash-not-receiving-messages-when-started-via-docker-compose-docker-is-ok/303645)
**Author:** [@alex\_london](https://discuss.elastic.co/u/alex_london)\ **Replies:** 2\ **Last updated:** [May 2, 2022, 1:41pm UTC](https://discuss.elastic.co/t/logstash-not-receiving-messages-when-started-via-docker-compose-docker-is-ok/303645 "2022-05-02T13:41:30Z")
I've encountered strange behaviour when running Logstash via docker-compose which I have been unable to figure out. Logstash starts (seemingly) fine, but it does not receive any events from Beats or other sources. Events… --- ## [\[Vega/EMS\] Problems using EMS map World Countries](https://discuss.elastic.co/t/vega-ems-problems-using-ems-map-world-countries/302507)
**Author:** [@Kayak007](https://discuss.elastic.co/u/Kayak007)\ **Replies:** 2\ **Last updated:** [May 2, 2022, 1:34pm UTC](https://discuss.elastic.co/t/vega-ems-problems-using-ems-map-world-countries/302507 "2022-05-02T13:34:44Z")
Hi, Trying to use the World Countries map from the EMS Boundaries with Vega like it's shown in the documentation here. The (very simple) code is the following under Kibana 7.16.3: { $schema: https://vega.github.io/s… --- ## [Redis Cache keyspace miss issue](https://discuss.elastic.co/t/redis-cache-keyspace-miss-issue/303584)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 0\ **Last updated:** [April 29, 2022, 8:25am UTC](https://discuss.elastic.co/t/redis-cache-keyspace-miss-issue/303584 "2022-04-29T08:25:59Z")
Hi everyone, We're using an ELK stack with redis buffer on kubernetes. Lately there has been an increase in traffic and we've noticed that some logs are not registered to Elasticsearch. Redis-cli shows steadily increasi… --- ## [Derive index with reversed nesting](https://discuss.elastic.co/t/derive-index-with-reversed-nesting/303393)
**Author:** [@denbo](https://discuss.elastic.co/u/denbo)\ **Replies:** 5\ **Last updated:** [May 2, 2022, 12:24pm UTC](https://discuss.elastic.co/t/derive-index-with-reversed-nesting/303393 "2022-05-02T12:24:01Z")
I have an index of products with a (denormalized) m:n relationship to categories. My goal is to derive a categories index from it which contains the same information, but with the relationship reversed. What I have # … --- ## [Is it possible to calculate the number of results by grouping them by several criteria?](https://discuss.elastic.co/t/is-it-possible-to-calculate-the-number-of-results-by-grouping-them-by-several-criteria/303497)
**Author:** [@Denys\_Lukash](https://discuss.elastic.co/u/Denys_Lukash)\ **Replies:** 2\ **Last updated:** [May 2, 2022, 11:53am UTC](https://discuss.elastic.co/t/is-it-possible-to-calculate-the-number-of-results-by-grouping-them-by-several-criteria/303497 "2022-05-02T11:53:24Z")
Hi dear community. I'm a new user of elastic and I'm having trouble building a query, I don't know if it's possible to do what I want. My task is to get the number of products in each category. The problem is that the g… --- ## [Log ES inserts / operations](https://discuss.elastic.co/t/log-es-inserts-operations/303738)
**Author:** [@webfr](https://discuss.elastic.co/u/webfr)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 10:55am UTC](https://discuss.elastic.co/t/log-es-inserts-operations/303738 "2022-05-02T10:55:21Z")
Hello guys, I would like to know how to log Elasticsearch operations like inserts with parameters for URL called, Ex: http://cloud:9200/myindex/\_doc Using free ES 7.15.1 Thanks. --- ## [Hunspell fails to stem correctly](https://discuss.elastic.co/t/hunspell-fails-to-stem-correctly/303739)
**Author:** [@mekanix](https://discuss.elastic.co/u/mekanix)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 10:56am UTC](https://discuss.elastic.co/t/hunspell-fails-to-stem-correctly/303739 "2022-05-02T10:56:02Z")
We are a bit puzzled about some shortcomings when using hunspell for stemming and can't figure out why it fails. Examples: gulerod (singular), gulerødder (plural) mand (singular), mænd (plural) mønster (singular), møns… --- ## [Elasticsearch login](https://discuss.elastic.co/t/elasticsearch-login/303527)
**Author:** [@Roshan1](https://discuss.elastic.co/u/Roshan1)\ **Replies:** 4\ **Last updated:** [May 2, 2022, 10:23am UTC](https://discuss.elastic.co/t/elasticsearch-login/303527 "2022-05-02T10:23:32Z")
Hello Team, kindly advise why I am being prompted to enter username/password when accessing ES. PLease find attached image. I have installed latest version of Kibana and ES 8.1 Thanks, Roshan [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=954) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=956)