# Latest **URL:** https://discuss.elastic.co/latest.md?page=1602 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1603 --- ## [Signs of too few shards? Number of documents per shard?](https://discuss.elastic.co/t/signs-of-too-few-shards-number-of-documents-per-shard/276972)
**Author:** [@egalpin](https://discuss.elastic.co/u/egalpin)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 1:13pm UTC](https://discuss.elastic.co/t/signs-of-too-few-shards-number-of-documents-per-shard/276972 "2021-06-25T13:13:49Z")
Hi all, I'm experiencing performance challenges with an ES cluster. The cluster consists of 6 data nodes (3 hot w/ ~30GB heap, 3 warm w/ ~15GB heap). The cluster holds around 4TB of data total, with a total of 20,000,00… --- ## [Add\_kubernetes\_metadata : handle logs from containerd and docker](https://discuss.elastic.co/t/add-kubernetes-metadata-handle-logs-from-containerd-and-docker/276390)
**Author:** [@orgoz](https://discuss.elastic.co/u/orgoz)\ **Replies:** 0\ **Last updated:** [June 18, 2021, 12:34pm UTC](https://discuss.elastic.co/t/add-kubernetes-metadata-handle-logs-from-containerd-and-docker/276390 "2021-06-18T12:34:56Z")
Hello, I am using filebeat 7.9.0. I have a k8s cluster with multiple node pools using containerd and docker runtime. I want my logs from these two runtime correctly parsed and with kubernetes metadata. I am using thi… --- ## [Using "message" in custom alert rule](https://discuss.elastic.co/t/using-message-in-custom-alert-rule/275899)
**Author:** [@EvanGertis](https://discuss.elastic.co/u/EvanGertis)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 12:51pm UTC](https://discuss.elastic.co/t/using-message-in-custom-alert-rule/275899 "2021-06-25T12:51:31Z")
I would like to create an alert that follows: "message: some search string" When I create the alert and save it. I receive the following error "Bulk Indexing of signals failed: object mapping for \[source\] tried to par… --- ## [Is it possible to reindex using a source index including a wildcard?](https://discuss.elastic.co/t/is-it-possible-to-reindex-using-a-source-index-including-a-wildcard/277059)
**Author:** [@Amphagory](https://discuss.elastic.co/u/Amphagory)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 12:47pm UTC](https://discuss.elastic.co/t/is-it-possible-to-reindex-using-a-source-index-including-a-wildcard/277059 "2021-06-25T12:47:02Z")
Hello, I'm trying to perform a reindex where the source index is split up into a number of indices all starting with my\_index. Also, I'm limiting the data I want between a certain range. The syntax I have is as follow… --- ## [Should I route "get by \_id" queries to improve performance?](https://discuss.elastic.co/t/should-i-route-get-by-id-queries-to-improve-performance/277046)
**Author:** [@Alexander\_Gamanyuk1](https://discuss.elastic.co/u/Alexander_Gamanyuk1)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 12:33pm UTC](https://discuss.elastic.co/t/should-i-route-get-by-id-queries-to-improve-performance/277046 "2021-06-25T12:33:17Z")
I have an Elasticsearch index with hundreds of millions of documents that is used mostly for the "get by id" queries. I consider adding a route to documents when indexing. The route will be a random number from 0 to 9. … --- ## [Unable to use new multi\_terms aggregation via Java Client](https://discuss.elastic.co/t/unable-to-use-new-multi-terms-aggregation-via-java-client/277050)
**Author:** [@IliaIsakhin](https://discuss.elastic.co/u/IliaIsakhin)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 11:19am UTC](https://discuss.elastic.co/t/unable-to-use-new-multi-terms-aggregation-via-java-client/277050 "2021-06-25T11:19:00Z")
Hello! I\`ve updated my ES cluster to the version 7.12 to use brand new multi-terms aggregations I want to do something like this in my Java Rest Client AggregationBuilders.multiTerms(field) But as i see, there is no b… --- ## [Logstash with Point to Point](https://discuss.elastic.co/t/logstash-with-point-to-point/277047)
**Author:** [@Michael\_Dylan\_McAloo](https://discuss.elastic.co/u/Michael_Dylan_McAloo)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 11:03am UTC](https://discuss.elastic.co/t/logstash-with-point-to-point/277047 "2021-06-25T11:03:07Z")
In Kibana I try to use the "Point to Point" map but it doesn't show me anything because I only have two fields called "geoip.location" and "location" and I don't have the destination field to filter and "geoip.location" … --- ## [Removing granularity information from a TSVB graph](https://discuss.elastic.co/t/removing-granularity-information-from-a-tsvb-graph/277045)
**Author:** [@mdgElastic](https://discuss.elastic.co/u/mdgElastic)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 11:01am UTC](https://discuss.elastic.co/t/removing-granularity-information-from-a-tsvb-graph/277045 "2021-06-25T11:01:54Z")
Hi all, Is it possible to hide the granularity information of a graph created using a TSBV? As an example: Many thanks in advance! Best! --- ## [ThreatIntel + module configuration](https://discuss.elastic.co/t/threatintel-module-configuration/276652)
**Author:** [@malvivent7](https://discuss.elastic.co/u/malvivent7)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 10:18am UTC](https://discuss.elastic.co/t/threatintel-module-configuration/276652 "2021-06-25T10:18:18Z")
Hi to all, i have enabled through filebeat modules the threatintel module and after that i have configured threatintel.yml activating otx and abusemalware than filebeat -e setup . So far so good but till now i dont see … --- ## [Kibana Hyperlink](https://discuss.elastic.co/t/kibana-hyperlink/277042)
**Author:** [@Mathu](https://discuss.elastic.co/u/Mathu)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 10:34am UTC](https://discuss.elastic.co/t/kibana-hyperlink/277042 "2021-06-25T10:34:07Z")
I have a data table. I need to apply hyperlink to values in the table. User Jan Feb March Sam 11 22 33 Rob 44 55 66 Kim 77 88 99 Say for examp… --- ## [Threat detection rules VS beats](https://discuss.elastic.co/t/threat-detection-rules-vs-beats/275311)
**Author:** [@farciarz121](https://discuss.elastic.co/u/farciarz121)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 10:26am UTC](https://discuss.elastic.co/t/threat-detection-rules-vs-beats/275311 "2021-06-25T10:26:30Z")
Can someone tell me if default detection rules are capable of triggering alerts base on logs from events (not from fleet). I.e. new user creation. I can not make it work. Also, how do I define my own custom policies that… --- ## [Connecting to elasticsearch through kibana using NEST](https://discuss.elastic.co/t/connecting-to-elasticsearch-through-kibana-using-nest/277030)
**Author:** [@mityai013](https://discuss.elastic.co/u/mityai013)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 8:39am UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-through-kibana-using-nest/277030 "2021-06-25T08:39:02Z")
Hello everyone . We are using a kibana connection to elasticsearch using NEST. This is necessary to monitor the generated indexes. We updated our ELK stack to 7.13.2 and this stopped working. Before the update in version… --- ## [Logstash does not read 1st line of file which the header](https://discuss.elastic.co/t/logstash-does-not-read-1st-line-of-file-which-the-header/277035)
**Author:** [@Julius\_Gamboa](https://discuss.elastic.co/u/Julius_Gamboa)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 9:58am UTC](https://discuss.elastic.co/t/logstash-does-not-read-1st-line-of-file-which-the-header/277035 "2021-06-25T09:58:05Z")
Hi, I am using logstash to ingest a pipe delimited file into elasticsearch. I am using the autodetect\_column\_names in the csv filter. Noticed when I run logstash, I get the following warnings below: \[2021-06-25T16:48:3… --- ## [Index update](https://discuss.elastic.co/t/index-update/277025)
**Author:** [@Abhishek\_Shinde](https://discuss.elastic.co/u/Abhishek_Shinde)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 9:23am UTC](https://discuss.elastic.co/t/index-update/277025 "2021-06-25T09:23:56Z")
I would need to know, if we can take Snapshot of Indices during Index update operation? I know restore demands to keep indices in the close state. --- ## [Last and sum in Kibana ESSQL](https://discuss.elastic.co/t/last-and-sum-in-kibana-essql/277036)
**Author:** [@xquest](https://discuss.elastic.co/u/xquest)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 9:17am UTC](https://discuss.elastic.co/t/last-and-sum-in-kibana-essql/277036 "2021-06-25T09:17:52Z")
Hi! I am using Kiban canvas and sometimes Kibana canvas expression editor. I got some search: SELECT host.name as n, system.diskio.name as s, last(system.diskio.io.time,"@timestamp") as v FROM "\ **Author:** [@ORich](https://discuss.elastic.co/u/ORich)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 9:15am UTC](https://discuss.elastic.co/t/index-lifecycle-error-2-index-templates-matching/276809 "2021-06-25T09:15:58Z") Dear experts, here is my filebeat setup: # Set the prefix used in the index lifecycle write alias name. The default alias # name is 'filebeat-%{\[agent.version\]}'. setup.ilm.rollover\_alias: 'filebeat-%{\[agent.version\]}-… --- ## [Tooltip filter in TSVB](https://discuss.elastic.co/t/tooltip-filter-in-tsvb/277034)
**Author:** [@Krunal](https://discuss.elastic.co/u/Krunal)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 8:54am UTC](https://discuss.elastic.co/t/tooltip-filter-in-tsvb/277034 "2021-06-25T08:54:07Z")
Hello Expert, I would like to remove all the 0% value which shows in the tooltip, means I would like to apply filter not equal to 0. Is it possible to remove it? --- ## [Logstash "Received fatal alert : bad\_certificate"](https://discuss.elastic.co/t/logstash-received-fatal-alert-bad-certificate/277032)
**Author:** [@diegz](https://discuss.elastic.co/u/diegz)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 8:51am UTC](https://discuss.elastic.co/t/logstash-received-fatal-alert-bad-certificate/277032 "2021-06-25T08:51:50Z")
Hello, I setup TLS on Elasticsearch, kibana, logstash and filebeat. I don't know if it comes from the configuration of logstash or filebeat. I have a cluster of 3 elastic nodes, logstash and filebeat are on node1. Th… --- ## [How to obtain the list of \ associated to a \](https://discuss.elastic.co/t/how-to-obtain-the-list-of-forecast-id-associated-to-a-job-name/276942)
**Author:** [@StephaneB](https://discuss.elastic.co/u/StephaneB)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 8:28am UTC](https://discuss.elastic.co/t/how-to-obtain-the-list-of-forecast-id-associated-to-a-job-name/276942 "2021-06-25T08:28:15Z")
Hi, I would like to get the list of all the \ associated to a specific \. I tried: GET \_ml/anomaly\_detectors/\ but I don't obtain any information about associated forecast GET \_ml/anoma… --- ## [Filebeat decode\_json\_fields processor keeps giving convert error for GCP module](https://discuss.elastic.co/t/filebeat-decode-json-fields-processor-keeps-giving-convert-error-for-gcp-module/277024)
**Author:** [@stevensim226](https://discuss.elastic.co/u/stevensim226)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 7:55am UTC](https://discuss.elastic.co/t/filebeat-decode-json-fields-processor-keeps-giving-convert-error-for-gcp-module/277024 "2021-06-25T07:55:06Z")
I keep getting an error that says GoError: failed in processor.convert: conversion of field \[json.protoPayload\] to type \[\[unset\]\] with target field \[json\] failed: field \[json.protoPayload\] is missing: key not found when … --- ## [How can i replace multiple special characters in all fields?](https://discuss.elastic.co/t/how-can-i-replace-multiple-special-characters-in-all-fields/276922)
**Author:** [@alprde](https://discuss.elastic.co/u/alprde)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 7:58am UTC](https://discuss.elastic.co/t/how-can-i-replace-multiple-special-characters-in-all-fields/276922 "2021-06-25T07:58:24Z")
Ç =\> Ç ç =\> ç Ş =\> Ş ş =\> ş İ =\> İ ı =\> ı Ü =\> Ü ü =\> ü Ö =\> Ö ö =\> ö Ğ =\> Ğ ğ =\> ğ I want to replace these characters in all fields. how can I … --- ## [Elasticsearch not saving all fields from logstash](https://discuss.elastic.co/t/elasticsearch-not-saving-all-fields-from-logstash/276957)
**Author:** [@mlassnig\_no](https://discuss.elastic.co/u/mlassnig_no)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 7:50am UTC](https://discuss.elastic.co/t/elasticsearch-not-saving-all-fields-from-logstash/276957 "2021-06-25T07:50:14Z")
I got following situation: Im sending logs from my servers to logstash using filebeat. Im running a logstash pipeline that looks like this: input { beats { port =\> 5003 } } filter { grok { match =\> … --- ## [Elasticsearch deployment](https://discuss.elastic.co/t/elasticsearch-deployment/276700)
**Author:** [@porscheme](https://discuss.elastic.co/u/porscheme)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 7:27am UTC](https://discuss.elastic.co/t/elasticsearch-deployment/276700 "2021-06-25T07:27:46Z")
We wanted to build Elasticsearch on cloud Azure. Source data size: 50TB Replicas count: 3 Shard count: 1000 For doing this, we are considering below two VM SKUs. L32sv2 is very processing we wanted to know SLAs for… --- ## [Duplication while logs are appended in Filebeat](https://discuss.elastic.co/t/duplication-while-logs-are-appended-in-filebeat/277016)
**Author:** [@connectgeeks](https://discuss.elastic.co/u/connectgeeks)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 6:32am UTC](https://discuss.elastic.co/t/duplication-while-logs-are-appended-in-filebeat/277016 "2021-06-25T06:32:05Z")
I'm using Filebeat-\> Logstash-\> ES setup. 7.x While paring the apache and json logs using, I notice every time the file is appended with new logs I see old logs also getting re-created in elasticsearch. How can I avoid … --- ## [Count unique customers based on date range](https://discuss.elastic.co/t/count-unique-customers-based-on-date-range/277014)
**Author:** [@Sanjay\_Agnani1](https://discuss.elastic.co/u/Sanjay_Agnani1)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 5:55am UTC](https://discuss.elastic.co/t/count-unique-customers-based-on-date-range/277014 "2021-06-25T05:55:53Z")
Need the stats in unique customer: index contains customers orders data. Ex: { "\_index" : "customers\_orders", "\_type" : "\_doc", "\_id" : "109126", "\_score" : 1.0, "\_source" : … --- ## [How can I start logstash from a script](https://discuss.elastic.co/t/how-can-i-start-logstash-from-a-script/277010)
**Author:** [@Mbartlett413](https://discuss.elastic.co/u/Mbartlett413)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 5:30am UTC](https://discuss.elastic.co/t/how-can-i-start-logstash-from-a-script/277010 "2021-06-25T05:30:51Z")
Background: I have logstash and elasticsearch running on a server. I have a rails application which uploads a CSV to the server and logstash then processes. It works if I manually execute the generated script. If I try t… --- ## [Elasticsearch \_update\_by\_query in logstash error \[HTTP Output Failure\] Encountered non-2xx HTTP code 400](https://discuss.elastic.co/t/elasticsearch-update-by-query-in-logstash-error-http-output-failure-encountered-non-2xx-http-code-400/274405)
**Author:** [@priyaankaa](https://discuss.elastic.co/u/priyaankaa)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 5:40am UTC](https://discuss.elastic.co/t/elasticsearch-update-by-query-in-logstash-error-http-output-failure-encountered-non-2xx-http-code-400/274405 "2021-06-25T05:40:42Z")
Properly working \_update\_by\_query call - POST /s1test-demo7/\_update\_by\_query { "script": { "source": "ctx.\_source.externaldata = params.externaldata", "lang": "painless", "params": { "externa… --- ## [Question about update sorting](https://discuss.elastic.co/t/question-about-update-sorting/277002)
**Author:** [@111486](https://discuss.elastic.co/u/111486)\ **Replies:** 1\ **Last updated:** [June 25, 2021, 5:07am UTC](https://discuss.elastic.co/t/question-about-update-sorting/277002 "2021-06-25T05:07:32Z")
When updating the content, it was found that the updated content and the read content fields were not in the same order. version is: // 20210625115137 { "name": "372aac1d4b15", "cluster\_name": "tommy-es", "clu… --- ## [Which 'permission' for 'create user patternn'?](https://discuss.elastic.co/t/which-permission-for-create-user-patternn/277006)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 0\ **Last updated:** [June 25, 2021, 4:13am UTC](https://discuss.elastic.co/t/which-permission-for-create-user-patternn/277006 "2021-06-25T04:13:13Z")
Hi there, when I (as an cluster admin) enter the index name in the search field at "create index pattern" the index is displayed. When the customer does this, no index is found. So, which permission for him is needed t… --- ## [How do elastic Apply Severity to pre-built rules?](https://discuss.elastic.co/t/how-do-elastic-apply-severity-to-pre-built-rules/276437)
**Author:** [@Oliver2](https://discuss.elastic.co/u/Oliver2)\ **Replies:** 2\ **Last updated:** [June 25, 2021, 4:20am UTC](https://discuss.elastic.co/t/how-do-elastic-apply-severity-to-pre-built-rules/276437 "2021-06-25T04:20:21Z")
How is the pre-built detection rules severity worked out, is it a random number or is there some documented framework i cant seem to find any notes on it? [Previous page](https://discuss.elastic.co/latest.md?page=1601) [Next page](https://discuss.elastic.co/latest.md?page=1603)