# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=671 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 672 --- ## [Logstash filter with grok](https://discuss.elastic.co/t/logstash-filter-with-grok/269056)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 1\ **Last updated:** [April 1, 2021, 4:39pm UTC](https://discuss.elastic.co/t/logstash-filter-with-grok/269056 "2021-04-01T16:39:29Z")
I have used grok debugger and I've identified the pattern i need for getting the filed id.orig\_h My log looks like this: 1295981543.337241 CCBr9k1C1bSaHQbUhh 192.168.3.131 55954 65.55.17.37 80 tcp http 1.776718 1837 24… --- ## [Shrink How to avoid stop writing](https://discuss.elastic.co/t/shrink-how-to-avoid-stop-writing/269154)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 2:27am UTC](https://discuss.elastic.co/t/shrink-how-to-avoid-stop-writing/269154 "2021-04-04T02:27:30Z")
I have 1 index with 2 primary shards, I want to narrow it down to 1 shard, I learned through search that it is recommended to achieve through shrink api, but shrink will stop writing, is there any solution to stop writin… --- ## [Count array size in all document](https://discuss.elastic.co/t/count-array-size-in-all-document/269108)
**Author:** [@Giovanni\_Di\_Lembo](https://discuss.elastic.co/u/Giovanni_Di_Lembo)\ **Replies:** 7\ **Last updated:** [April 3, 2021, 8:49pm UTC](https://discuss.elastic.co/t/count-array-size-in-all-document/269108 "2021-04-03T20:49:19Z")
Hi all and sorry for my poor english I have a document like { "uuid": "SkDHvXr9ES", "userRole": "9qvfWg3isp", "devices":\[ {"uuid": "hwb258pc7x", "macAddress":\["9DGSsuAX4M", "ChypSa8tWx", "fIWrArQyDA", "PQjcvOkHYh… --- ## [Aggregation by host displays hash instead of hostname](https://discuss.elastic.co/t/aggregation-by-host-displays-hash-instead-of-hostname/269158)
**Author:** [@desoul](https://discuss.elastic.co/u/desoul)\ **Replies:** 0\ **Last updated:** [April 3, 2021, 4:21pm UTC](https://discuss.elastic.co/t/aggregation-by-host-displays-hash-instead-of-hostname/269158 "2021-04-03T16:21:19Z")
Hello, i try to get unique agent hostnames, grouped with its logfile paths (i have Filebeat agents with system.yml configuration). Here is a query: curl -u myUser:myPassword-X GET "myELKhostname:9237/filebeat\*/\_search?p… --- ## [How to simulate the failure of elasticsearch？](https://discuss.elastic.co/t/how-to-simulate-the-failure-of-elasticsearch/269149)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 0\ **Last updated:** [April 3, 2021, 12:30pm UTC](https://discuss.elastic.co/t/how-to-simulate-the-failure-of-elasticsearch/269149 "2021-04-03T12:30:44Z")
I found that for some simple failures, I can do it easily with the help of tools, such as network partitioning, nodes being killed, but there are still many abnormal cases that are not good to simulate, for example, ther… --- ## [Latency in indexing using client.update() of the high level rest client](https://discuss.elastic.co/t/latency-in-indexing-using-client-update-of-the-high-level-rest-client/269142)
**Author:** [@Mahesh\_H\_Viraktamath](https://discuss.elastic.co/u/Mahesh_H_Viraktamath)\ **Replies:** 2\ **Last updated:** [April 3, 2021, 10:13am UTC](https://discuss.elastic.co/t/latency-in-indexing-using-client-update-of-the-high-level-rest-client/269142 "2021-04-03T10:13:54Z")
We have started using the high level REST client finally, to ease the development of queries from backend engineering perspective. For indexing, we are using the client.update(request, RequestOptions.DEFAULT) so that new… --- ## [QueryShardException No mapping found for \[failed\]](https://discuss.elastic.co/t/queryshardexception-no-mapping-found-for-failed/269111)
**Author:** [@MdRashid](https://discuss.elastic.co/u/MdRashid)\ **Replies:** 1\ **Last updated:** [April 2, 2021, 8:47pm UTC](https://discuss.elastic.co/t/queryshardexception-no-mapping-found-for-failed/269111 "2021-04-02T20:47:56Z")
Hi All, I have deploy my ES on Azure Kubernetes Services AKS after deploying i got error all shards failed Caused by: org.elasticsearch.index.query.QueryShardException: No mapping found for \[failed\] in order to sort o… --- ## [How to know Elasticsearch daily log intake?](https://discuss.elastic.co/t/how-to-know-elasticsearch-daily-log-intake/269104)
**Author:** [@mahesh.yerawar](https://discuss.elastic.co/u/mahesh.yerawar)\ **Replies:** 3\ **Last updated:** [April 2, 2021, 8:47pm UTC](https://discuss.elastic.co/t/how-to-know-elasticsearch-daily-log-intake/269104 "2021-04-02T20:47:01Z")
Hi, I'm looking to understand how much data is getting ingested to our ELK cluster on a daily basis. anyone has any idea how to get these details? Thank you. --- ## [Pipeline has lesser fields than Filebeat directly to Elasticsearch](https://discuss.elastic.co/t/pipeline-has-lesser-fields-than-filebeat-directly-to-elasticsearch/269054)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 0\ **Last updated:** [April 1, 2021, 4:04pm UTC](https://discuss.elastic.co/t/pipeline-has-lesser-fields-than-filebeat-directly-to-elasticsearch/269054 "2021-04-01T16:04:17Z")
Hi team, I have a pipeline like this: filebeat =\>kafka =\>logstask=\>elasticsearch (ELK7.12) Why I have a smaller amounts of fileds than filebeat directly =\>elasticsearch (ELK7.11.1) Whenever I check the fields from ECS… --- ## [Ngram vs wildcard (Using ngrams instead of wildcards)](https://discuss.elastic.co/t/ngram-vs-wildcard-using-ngrams-instead-of-wildcards/260809)
**Author:** [@mohsin](https://discuss.elastic.co/u/mohsin)\ **Replies:** 1\ **Last updated:** [April 2, 2021, 1:56pm UTC](https://discuss.elastic.co/t/ngram-vs-wildcard-using-ngrams-instead-of-wildcards/260809 "2021-04-02T13:56:06Z")
We are using spring-data-elasticsearch in our Project we have the autocomplete function for the search textbox. When we type into the search text field, we will fetch suggestions by performing a wild card search with t… --- ## [Is there any option to define multiple key value pairs based on delimeter while ingesting files through FSCrawler in Elasticsearch?](https://discuss.elastic.co/t/is-there-any-option-to-define-multiple-key-value-pairs-based-on-delimeter-while-ingesting-files-through-fscrawler-in-elasticsearch/261383)
**Author:** [@Nirmal\_Raj](https://discuss.elastic.co/u/Nirmal_Raj)\ **Replies:** 3\ **Last updated:** [April 2, 2021, 1:49pm UTC](https://discuss.elastic.co/t/is-there-any-option-to-define-multiple-key-value-pairs-based-on-delimeter-while-ingesting-files-through-fscrawler-in-elasticsearch/261383 "2021-04-02T13:49:14Z")
I'm trying to ingest multiple extension files like (.pdf, .doc, .docx, .csv) using FSCrawler in Elasticsearch . After ingesting, all the details inside the files mapped under one key value pair called content. Is there … --- ## [Elasticsearch thread\_pool is growing over and over](https://discuss.elastic.co/t/elasticsearch-thread-pool-is-growing-over-and-over/266078)
**Author:** [@nuzolx](https://discuss.elastic.co/u/nuzolx)\ **Replies:** 3\ **Last updated:** [April 2, 2021, 1:39pm UTC](https://discuss.elastic.co/t/elasticsearch-thread-pool-is-growing-over-and-over/266078 "2021-04-02T13:39:17Z")
Hi all, I am a newbie with Elastic. I cannot find all my documents in my research. If I repeat the same search, after a while, it's okay. I understood that it could come from the 'refresh'. But sometimes the waiting tim… --- ## [Change ILM policy](https://discuss.elastic.co/t/change-ilm-policy/269105)
**Author:** [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\ **Replies:** 0\ **Last updated:** [April 2, 2021, 12:49pm UTC](https://discuss.elastic.co/t/change-ilm-policy/269105 "2021-04-02T12:49:17Z")
Hello, I have a cluster of 2 nodes HOT and 1 DATA warm, and my ILM configuration is set to 1 shard and 1 replica I checked today my indexes and I found out that My HOT data node is going to hit 90% of DISK, and after … --- ## [Elasticsearch crashing with 'no space available', while disk still has 75% free storage](https://discuss.elastic.co/t/elasticsearch-crashing-with-no-space-available-while-disk-still-has-75-free-storage/269094)
**Author:** [@Kristijan](https://discuss.elastic.co/u/Kristijan)\ **Replies:** 1\ **Last updated:** [April 2, 2021, 11:10am UTC](https://discuss.elastic.co/t/elasticsearch-crashing-with-no-space-available-while-disk-still-has-75-free-storage/269094 "2021-04-02T11:10:08Z")
The elasticsearch(v5.6.12) is deployed as a single pod in a Kubernetes cluster with attached storage through a PVC. Once every couple of months it will go into a crash loop state and complain that there is no storage le… --- ## [How to create mulitple indices](https://discuss.elastic.co/t/how-to-create-mulitple-indices/268920)
**Author:** [@Shushil\_Shankar](https://discuss.elastic.co/u/Shushil_Shankar)\ **Replies:** 5\ **Last updated:** [April 2, 2021, 10:40am UTC](https://discuss.elastic.co/t/how-to-create-mulitple-indices/268920 "2021-04-02T10:40:48Z")
Good morning eveyyone! I have a question about the creation of a multiple indices. Currenlty in my project i have one index mutileple types. Now i need to migrate new version 7.12. How to do i create multiple indicies … --- ## [Slower Query Response Times - Intermittent](https://discuss.elastic.co/t/slower-query-response-times-intermittent/266634)
**Author:** [@israni.amit](https://discuss.elastic.co/u/israni.amit)\ **Replies:** 6\ **Last updated:** [April 2, 2021, 9:41am UTC](https://discuss.elastic.co/t/slower-query-response-times-intermittent/266634 "2021-04-02T09:41:46Z")
Hi there, We recently noticed that some of the queries were having high response times like 12-15 seconds compared to others which were in miliseconds. We have a query layer (Spring Boot App) which uses RestHighLevelCl… --- ## [ElasticSearch WARN after installing kibana](https://discuss.elastic.co/t/elasticsearch-warn-after-installing-kibana/268428)
**Author:** [@kylesu](https://discuss.elastic.co/u/kylesu)\ **Replies:** 10\ **Last updated:** [April 2, 2021, 9:07am UTC](https://discuss.elastic.co/t/elasticsearch-warn-after-installing-kibana/268428 "2021-04-02T09:07:47Z")
After installing kibana,when i operate(GET/POST/DELETE and so on) Elastic Search with High Level Rest Clinet comes up this warning: Each request comes up with one warning,i have 500 millions documents to process every … --- ## [Ingest-attachment sometimes recognizes files as empty](https://discuss.elastic.co/t/ingest-attachment-sometimes-recognizes-files-as-empty/268238)
**Author:** [@dimaPust](https://discuss.elastic.co/u/dimaPust)\ **Replies:** 1\ **Last updated:** [April 2, 2021, 8:32am UTC](https://discuss.elastic.co/t/ingest-attachment-sometimes-recognizes-files-as-empty/268238 "2021-04-02T08:32:10Z")
Hello, I am using elasticsearch:7.9.0 with the ingest-attachment plugin for indexing docx files, created in an external editor. However, sometimes files are indexed as 'empty', for example next file: { \_index: 'loc… --- ## [Adding objects to an Array filed in Elastic Search index using Java High level Rest Client](https://discuss.elastic.co/t/adding-objects-to-an-array-filed-in-elastic-search-index-using-java-high-level-rest-client/268418)
**Author:** [@dekay14](https://discuss.elastic.co/u/dekay14)\ **Replies:** 7\ **Last updated:** [April 2, 2021, 8:14am UTC](https://discuss.elastic.co/t/adding-objects-to-an-array-filed-in-elastic-search-index-using-java-high-level-rest-client/268418 "2021-04-02T08:14:27Z")
I am trying to add a list or a single object to a Array field on a Elastic Search Index, the index is as shown below: "\_index" : "class\_index", "\_id" : "1234", "\_source" : { "externalId" : … --- ## [Question about threadpools](https://discuss.elastic.co/t/question-about-threadpools/268888)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 13\ **Last updated:** [April 2, 2021, 8:08am UTC](https://discuss.elastic.co/t/question-about-threadpools/268888 "2021-04-02T08:08:31Z")
Hi, It's a simple question but I couldn't be sure and couldn't find anything that specifies this. When using Kibana on discover page what operation is used on elasticsearch to show logs? Is it Get or Search? --- ## [Aggregation size](https://discuss.elastic.co/t/aggregation-size/269067)
**Author:** [@Giovanni\_Di\_Lembo](https://discuss.elastic.co/u/Giovanni_Di_Lembo)\ **Replies:** 3\ **Last updated:** [April 2, 2021, 7:01am UTC](https://discuss.elastic.co/t/aggregation-size/269067 "2021-04-02T07:01:11Z")
Hi all and sorry for my poor english I have tons of data with different field value 'fastCategory', but whe i do an aggragation on fastCategory, I found just 10 aggregation? Why that? POST statistic/\_search { "aggs"… --- ## [How to get a specific cluster setting?](https://discuss.elastic.co/t/how-to-get-a-specific-cluster-setting/269081)
**Author:** [@smlbiobot](https://discuss.elastic.co/u/smlbiobot)\ **Replies:** 2\ **Last updated:** [April 2, 2021, 6:40am UTC](https://discuss.elastic.co/t/how-to-get-a-specific-cluster-setting/269081 "2021-04-02T06:40:27Z")
I know that I can get the cluster settings with this: GET \_cluster/settings What I don’t know is how I can get a specific setting, especially when it‘s at the default and so it doesn’t show up on the GET. For example,… --- ## [Elasticsearch & elasticflow](https://discuss.elastic.co/t/elasticsearch-elasticflow/269062)
**Author:** [@atamc3rb3r](https://discuss.elastic.co/u/atamc3rb3r)\ **Replies:** 1\ **Last updated:** [April 1, 2021, 9:12pm UTC](https://discuss.elastic.co/t/elasticsearch-elasticflow/269062 "2021-04-01T21:12:41Z")
Dear community, I have decided to install the Elasticflow version 5.0.1 on my ubuntu server 20.04 LTS and integrate it with the current installed Elasticsearch 7.10. I did it step by step based on below documentation. … --- ## [Does Elasticsearch 7.11 not supported by ECK operator](https://discuss.elastic.co/t/does-elasticsearch-7-11-not-supported-by-eck-operator/268984)
**Author:** [@NK2812](https://discuss.elastic.co/u/NK2812)\ **Replies:** 3\ **Last updated:** [April 1, 2021, 2:44pm UTC](https://discuss.elastic.co/t/does-elasticsearch-7-11-not-supported-by-eck-operator/268984 "2021-04-01T14:44:07Z")
Hi, I trying to deploy the Elasticsearch 7.11.0 version cluster using eck operator 1.3 version, it fails in init container elastic-internal-init-filesystem . It says unsupported\_distribution Is 7.11 version not suppor… --- ## [Stopwords are not working in custom tokenizer](https://discuss.elastic.co/t/stopwords-are-not-working-in-custom-tokenizer/269033)
**Author:** [@tusharl](https://discuss.elastic.co/u/tusharl)\ **Replies:** 2\ **Last updated:** [April 1, 2021, 2:23pm UTC](https://discuss.elastic.co/t/stopwords-are-not-working-in-custom-tokenizer/269033 "2021-04-01T14:23:47Z")
Consider below example GET \_analyze { "tokenizer": { "pattern" : ",", "type" : "pattern" }, "filter": \[ { "type": "edge\_ngram", "min\_gram": 2, "max\_g… --- ## [Is it bad to disable mmapfs when running elasticsearch in fargate](https://discuss.elastic.co/t/is-it-bad-to-disable-mmapfs-when-running-elasticsearch-in-fargate/269044)
**Author:** [@Heshan\_Anupama](https://discuss.elastic.co/u/Heshan_Anupama)\ **Replies:** 0\ **Last updated:** [April 1, 2021, 2:28pm UTC](https://discuss.elastic.co/t/is-it-bad-to-disable-mmapfs-when-running-elasticsearch-in-fargate/269044 "2021-04-01T14:28:19Z")
Hello, I am trying to get elasticsearch with fargate and I am getting the error of \[2\]: max virtual memory areas vm.max\_map\_count \[65530\] is too low, increase to at least \[262144\]. I understand that we could not change … --- ## [How to monitor restore from snapshot - failure case?](https://discuss.elastic.co/t/how-to-monitor-restore-from-snapshot-failure-case/269012)
**Author:** [@Lavanya](https://discuss.elastic.co/u/Lavanya)\ **Replies:** 5\ **Last updated:** [April 1, 2021, 1:00pm UTC](https://discuss.elastic.co/t/how-to-monitor-restore-from-snapshot-failure-case/269012 "2021-04-01T13:00:06Z")
I found that /\_cat/recovery API gives information about ongoing and completed snapshot recovery activity. In this response body, there is an attribute called stage whose possible values are { init, index, start, translo… --- ## [Rollover is taking time](https://discuss.elastic.co/t/rollover-is-taking-time/268927)
**Author:** [@Narayan\_Banik](https://discuss.elastic.co/u/Narayan_Banik)\ **Replies:** 1\ **Last updated:** [April 1, 2021, 10:58am UTC](https://discuss.elastic.co/t/rollover-is-taking-time/268927 "2021-04-01T10:58:23Z")
My Policy is supposed to roll over indices if size reaches to 500mb like below: GET \_ilm/policy/my-policy { "my-policy" : { "version" : 4, "modified\_date" : "2021-03-31T14:34:45.932Z", "policy" : { "… --- ## [How to solve hard and soft limit machine learning jobs](https://discuss.elastic.co/t/how-to-solve-hard-and-soft-limit-machine-learning-jobs/268933)
**Author:** [@Abdelhalim](https://discuss.elastic.co/u/Abdelhalim)\ **Replies:** 2\ **Last updated:** [April 1, 2021, 9:28am UTC](https://discuss.elastic.co/t/how-to-solve-hard-and-soft-limit-machine-learning-jobs/268933 "2021-04-01T09:28:18Z")
Hello; I have created a machine learning job to detect port scanner using the packetbeat-\* index. type of job: Population Population field: destination.ip metric: distinct count(destination port) influencers: destin… --- ## [Execute aggs query for each unqiue value?](https://discuss.elastic.co/t/execute-aggs-query-for-each-unqiue-value/268997)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 0\ **Last updated:** [April 1, 2021, 8:38am UTC](https://discuss.elastic.co/t/execute-aggs-query-for-each-unqiue-value/268997 "2021-04-01T08:38:49Z")
Hi, I have the following script which gives me the 10 most used applications. I want to expand this by getting the top 10 applications for each unique value in the customers field. Is this possible? The goal is to appl… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=670) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=672)