# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=317 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 318 --- ## [Space and data retention question](https://discuss.elastic.co/t/space-and-data-retention-question/322403)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 11\ **Last updated:** [January 12, 2023, 4:20pm UTC](https://discuss.elastic.co/t/space-and-data-retention-question/322403 "2023-01-12T16:20:47Z")
Good Afternoon and Happy New Year to all my Elastic Stack friends and Guru's. I have been having an issue with my Elasticsearch and syslog retention. In the /var/lib/elasticsearch/log folder I have 4.3 Terrabytes of sp… --- ## [Heapdump without timeouts](https://discuss.elastic.co/t/heapdump-without-timeouts/322850)
**Author:** [@frens](https://discuss.elastic.co/u/frens)\ **Replies:** 8\ **Last updated:** [January 12, 2023, 1:12pm UTC](https://discuss.elastic.co/t/heapdump-without-timeouts/322850 "2023-01-12T13:12:32Z")
Hi! We're running nodes with a substantial amount of heap (150+Gb). I know how to take a dump the heap with jmap. However, dumping a heap this size on our machines will probably take several minutes (if I extrapolate li… --- ## [How to get ML Categorization results break down by users](https://discuss.elastic.co/t/how-to-get-ml-categorization-results-break-down-by-users/320283)
**Author:** [@CamillePapillon](https://discuss.elastic.co/u/CamillePapillon)\ **Replies:** 8\ **Last updated:** [January 12, 2023, 12:45pm UTC](https://discuss.elastic.co/t/how-to-get-ml-categorization-results-break-down-by-users/320283 "2023-01-12T12:45:33Z")
Hi! I have an input index with tons of different logs messages coming from many different devices (same product type = similar logs, different users). My source index is break down with : timestamp, serial\_number, messa… --- ## [Logstash is not ingesting data from all nodes](https://discuss.elastic.co/t/logstash-is-not-ingesting-data-from-all-nodes/322981)
**Author:** [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\ **Replies:** 5\ **Last updated:** [January 12, 2023, 11:58am UTC](https://discuss.elastic.co/t/logstash-is-not-ingesting-data-from-all-nodes/322981 "2023-01-12T11:58:46Z")
I am using filebeat to ship logs to logstash and using it to ingest data to Index. From 2 days i am seeing multiple issues. from Kibana dashboard i am not seeing logs on regular basis of the particular index. // for t… --- ## [Max\_docs\_per\_index exceeded](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994)
**Author:** [@Dimitri\_F](https://discuss.elastic.co/u/Dimitri_F)\ **Replies:** 7\ **Last updated:** [January 12, 2023, 9:25am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994 "2023-01-12T09:25:07Z")
Hello. I use Elasticsearch with Graylog. During this week I noticed that my Elasticsearch server is running out of disk space. It’s configured to store 240 indices. Max\_docs\_per\_index is 20 million documents. Having che… --- ## [ELasticsearch 7.3.2 on ubuntu 20.04](https://discuss.elastic.co/t/elasticsearch-7-3-2-on-ubuntu-20-04/322984)
**Author:** [@Siva\_Karan](https://discuss.elastic.co/u/Siva_Karan)\ **Replies:** 3\ **Last updated:** [January 12, 2023, 6:47am UTC](https://discuss.elastic.co/t/elasticsearch-7-3-2-on-ubuntu-20-04/322984 "2023-01-12T06:47:40Z")
Hi Team, Can we install elasticsearch 7.3.2 in ubunutu 20.04. In the support matrix elasticsearch team no supported in to install on ubunut 20.04 But for our need shall we install 7.3.2 in ubuntu 20.0.4. is this cause… --- ## [Unable to access elasticsearch on a particular network](https://discuss.elastic.co/t/unable-to-access-elasticsearch-on-a-particular-network/322878)
**Author:** [@Jay\_Ostwal](https://discuss.elastic.co/u/Jay_Ostwal)\ **Replies:** 8\ **Last updated:** [January 12, 2023, 6:40am UTC](https://discuss.elastic.co/t/unable-to-access-elasticsearch-on-a-particular-network/322878 "2023-01-12T06:40:48Z")
I am using elasticsearch since more than one year on a particular wifi network. Suddenly since 2 days I am unable to access the data from elasticsearch on this particular network and I am getting error as ConnectionError… --- ## [Elasticsearch not starting after RHEL kernel upgrade](https://discuss.elastic.co/t/elasticsearch-not-starting-after-rhel-kernel-upgrade/322076)
**Author:** [@hasith-adl](https://discuss.elastic.co/u/hasith-adl)\ **Replies:** 7\ **Last updated:** [January 12, 2023, 6:22am UTC](https://discuss.elastic.co/t/elasticsearch-not-starting-after-rhel-kernel-upgrade/322076 "2023-01-12T06:22:42Z")
Hi, Elasticsearch (v7.5.2) does not start after performing a kernel upgrade from 3.10.0-957.el7 to 3.10.0-1160.80.1.el7. \[2022-12-28T12:30:27,741\]\[INFO \]\[o.e.b.BootstrapChecks \] \[igw-edb3\] bound or publishing to a n… --- ## [Error: Could not find or load main class org.elasticsearch.launcher.CliToolLauncher](https://discuss.elastic.co/t/error-could-not-find-or-load-main-class-org-elasticsearch-launcher-clitoollauncher/321572)
**Author:** [@dziugitaz](https://discuss.elastic.co/u/dziugitaz)\ **Replies:** 3\ **Last updated:** [January 12, 2023, 3:26am UTC](https://discuss.elastic.co/t/error-could-not-find-or-load-main-class-org-elasticsearch-launcher-clitoollauncher/321572 "2023-01-12T03:26:27Z")
I'm using windows and when I'm trying to run any other command than bin/elasticsearch.bat I get this strange error, maybe do you know why? I've checked the logs but I didn't found anything other than this dziugas@DESKTO… --- ## [Snapshot multiple indexes on cold tier only](https://discuss.elastic.co/t/snapshot-multiple-indexes-on-cold-tier-only/322923)
**Author:** [@ShubhamKumarJena](https://discuss.elastic.co/u/ShubhamKumarJena)\ **Replies:** 1\ **Last updated:** [January 11, 2023, 10:28pm UTC](https://discuss.elastic.co/t/snapshot-multiple-indexes-on-cold-tier-only/322923 "2023-01-11T22:28:08Z")
Hello guys, we have couple of our end users requesting to store their data in Elasticsearch for auditing purpose for 3 years. As data grows we have the bottleneck in our resources, so we have decided snapshot the data … --- ## [Cannot form cluster of Elasticsearch on EKS](https://discuss.elastic.co/t/cannot-form-cluster-of-elasticsearch-on-eks/322186)
**Author:** [@Zeynal\_Hajili](https://discuss.elastic.co/u/Zeynal_Hajili)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 9:58pm UTC](https://discuss.elastic.co/t/cannot-form-cluster-of-elasticsearch-on-eks/322186 "2023-01-11T21:58:43Z")
Hello, dear community, I am very new to Elasticsearch. I have been asked to deploy the ELK stack on AWS EKS and unfortunately, I am facing some issues. I am deploying the ELK stack in the following order via Helm chart… --- ## [How to Mask Card Number](https://discuss.elastic.co/t/how-to-mask-card-number/322914)
**Author:** [@Anil0110](https://discuss.elastic.co/u/Anil0110)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 9:55pm UTC](https://discuss.elastic.co/t/how-to-mask-card-number/322914 "2023-01-11T21:55:35Z")
How to Mask Card Number OF XML MESSAGE --- ## [Checksum failed (hardware problem?)](https://discuss.elastic.co/t/checksum-failed-hardware-problem/322500)
**Author:** [@yang\_peng](https://discuss.elastic.co/u/yang_peng)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 9:11pm UTC](https://discuss.elastic.co/t/checksum-failed-hardware-problem/322500 "2023-01-11T21:11:03Z")
A two-node cluster was restarted. The shard of one index failed to merge. The error is as follows： failed shard on node \[VosGChW1QTudYo9Tcl5sqg\]: shard failure, reason \[merge failed\], failure NotSerializableExceptionWra… --- ## [When I try to index a jason file using "es.helpers.bulk" the error "TypeError: pop expected at most 1 argument, got 2" is occurring](https://discuss.elastic.co/t/when-i-try-to-index-a-jason-file-using-es-helpers-bulk-the-error-typeerror-pop-expected-at-most-1-argument-got-2-is-occurring/322828)
**Author:** [@pand](https://discuss.elastic.co/u/pand)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 4:54pm UTC](https://discuss.elastic.co/t/when-i-try-to-index-a-jason-file-using-es-helpers-bulk-the-error-typeerror-pop-expected-at-most-1-argument-got-2-is-occurring/322828 "2023-01-11T16:54:09Z")
When I execute the following code the error "TypeError: pop expected at most 1 argument, got 2" is occuring, please provide solutions to fix this. def creatdocs(filename,pdfcontent): docs = \[{ 'content':… --- ## [Failed indices in snapshot since upgrade to 8.5.3](https://discuss.elastic.co/t/failed-indices-in-snapshot-since-upgrade-to-8-5-3/322802)
**Author:** [@QuLh44](https://discuss.elastic.co/u/QuLh44)\ **Replies:** 7\ **Last updated:** [January 11, 2023, 3:55pm UTC](https://discuss.elastic.co/t/failed-indices-in-snapshot-since-upgrade-to-8-5-3/322802 "2023-01-11T15:55:22Z")
Hello, Since we upgraded Elasticsearch/Logstash/Kibana to 8.5.3 version yesterday, we have lot of failed indices in snapshots (559 indices are OK, but 168 failed). Here is the error for each of them : INTERNAL\_SERVER\_… --- ## [How to stagger results by same source id but default sorted by date?](https://discuss.elastic.co/t/how-to-stagger-results-by-same-source-id-but-default-sorted-by-date/322275)
**Author:** [@Urban\_B](https://discuss.elastic.co/u/Urban_B)\ **Replies:** 3\ **Last updated:** [January 11, 2023, 3:37pm UTC](https://discuss.elastic.co/t/how-to-stagger-results-by-same-source-id-but-default-sorted-by-date/322275 "2023-01-11T15:37:54Z")
I have a situation where I have a set of documents that by default sorts by datetime so most resent documents are top listed. The dilemma is that a company can send us a feed of data so that companies results can take u… --- ## [Migrating code from ElasticSearch.Net 7 to Elastic.Clients.Elasticsearch 8](https://discuss.elastic.co/t/migrating-code-from-elasticsearch-net-7-to-elastic-clients-elasticsearch-8/322937)
**Author:** [@mbooh](https://discuss.elastic.co/u/mbooh)\ **Replies:** 0\ **Last updated:** [January 11, 2023, 2:03pm UTC](https://discuss.elastic.co/t/migrating-code-from-elasticsearch-net-7-to-elastic-clients-elasticsearch-8/322937 "2023-01-11T14:03:48Z")
Hi! I’m rewriting our code for the new Elastic.Client.Elasticsearch and I’m trying to find some code examples how to do that. For example we have a lot of queries like this one: \> var response = \_client.Search\ **Author:** [@fesiqueira](https://discuss.elastic.co/u/fesiqueira)\ **Replies:** 0\ **Last updated:** [January 11, 2023, 1:52pm UTC](https://discuss.elastic.co/t/get-elastic-shards-failure-upon-search-query-across-multiple-indices-while-creating-indices/322935 "2023-01-11T13:52:37Z") I'm using Elastic 7.13.1. I create indices dynamically using a template based on the year of a timestamp field. So when I try to index a document with a timestamp for 2023-01-01 into index index-2023 the index will be cr… --- ## [Elasticsearch 7.7 , cpu usage is very high](https://discuss.elastic.co/t/elasticsearch-7-7-cpu-usage-is-very-high/322726)
**Author:** [@gilsagiv](https://discuss.elastic.co/u/gilsagiv)\ **Replies:** 10\ **Last updated:** [January 11, 2023, 1:00pm UTC](https://discuss.elastic.co/t/elasticsearch-7-7-cpu-usage-is-very-high/322726 "2023-01-11T13:00:36Z")
Hi, My server is working on Elasticsearch 7.7, Recently the CPU usage is reaching 100% when Kibana is running. Elasticsearch process CPU usage is very high. GET \_cat/nodes?v=true&s=cpu:desc GET /\_cluster/setti… --- ## [Elasticsearch gives execption while report download\["message":"Request cannot be executed; I/O reactor status: STOPPED","errorstacktrace":"java.lang.RuntimeException\]](https://discuss.elastic.co/t/elasticsearch-gives-execption-while-report-download-message-request-cannot-be-executed-i-o-reactor-status-stopped-errorstacktrace-java-lang-runtimeexception/322825)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 12:03pm UTC](https://discuss.elastic.co/t/elasticsearch-gives-execption-while-report-download-message-request-cannot-be-executed-i-o-reactor-status-stopped-errorstacktrace-java-lang-runtimeexception/322825 "2023-01-11T12:03:13Z")
Hello All, I'm getting following exception while downloading report by query elastic.Unable to understand resolution for this.Tried increasing the jvm heap to 12 gb ,still same exception. Exception: {"statusCode":"REP… --- ## [Parse PDF catalogs to extract products informations](https://discuss.elastic.co/t/parse-pdf-catalogs-to-extract-products-informations/322145)
**Author:** [@Valentin\_Hirson](https://discuss.elastic.co/u/Valentin_Hirson)\ **Replies:** 3\ **Last updated:** [January 11, 2023, 11:22am UTC](https://discuss.elastic.co/t/parse-pdf-catalogs-to-extract-products-informations/322145 "2023-01-11T11:22:52Z")
Hello, I need to parse some PDF files to find the most relevant words and trying to find what PDF are talking about. Those PDF are vendors catalog, I need for example to extract products to propose them in a web search… --- ## [ES default date type not taking nanos](https://discuss.elastic.co/t/es-default-date-type-not-taking-nanos/322614)
**Author:** [@samee\_alam](https://discuss.elastic.co/u/samee_alam)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 11:19am UTC](https://discuss.elastic.co/t/es-default-date-type-not-taking-nanos/322614 "2023-01-11T11:19:58Z")
ES version 7.17.3 Fluentd config is as below: time\_key timestamp time\_format %Y-%m-%dT%H:%M:%S.%N%Z which takes nano seconds into consideration and data is stored properly in elasticsearch as: But default … --- ## [Cat Nodes API returning CPU usage as -1. Elasticsearch version:7.11.2 OS Deb 11](https://discuss.elastic.co/t/cat-nodes-api-returning-cpu-usage-as-1-elasticsearch-version-7-11-2-os-deb-11/322912)
**Author:** [@Sagar\_Shivani1](https://discuss.elastic.co/u/Sagar_Shivani1)\ **Replies:** 1\ **Last updated:** [January 11, 2023, 10:59am UTC](https://discuss.elastic.co/t/cat-nodes-api-returning-cpu-usage-as-1-elasticsearch-version-7-11-2-os-deb-11/322912 "2023-01-11T10:59:42Z")
Cat Nodes API returning CPU usage as -1. Elasticsearch version:7.11.2 OS Deb 11.. What can be the case? --- ## [Is it okay to keep too frequent rollups?](https://discuss.elastic.co/t/is-it-okay-to-keep-too-frequent-rollups/322909)
**Author:** [@mafia-task-pasta](https://discuss.elastic.co/u/mafia-task-pasta)\ **Replies:** 0\ **Last updated:** [January 11, 2023, 10:48am UTC](https://discuss.elastic.co/t/is-it-okay-to-keep-too-frequent-rollups/322909 "2023-01-11T10:48:21Z")
I have a lots of data in elasticsearch for which I am adding a new rollups. It is taking a lot of time to catch up on all historical data. I am running it at 1 hour interval using the cron expression and the page size if… --- ## [Facing issues with ILM on auto indexes](https://discuss.elastic.co/t/facing-issues-with-ilm-on-auto-indexes/322789)
**Author:** [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 8:58am UTC](https://discuss.elastic.co/t/facing-issues-with-ilm-on-auto-indexes/322789 "2023-01-11T08:58:50Z")
I am playing with ILM on one index named as filebeat-7.17.8-2023.01.09. This is the auto index. i want to create ILM on this so i have created alias for this index. curl -XPUT 'ip/filebeat-7.17.8-\*/\_alias/filebeat-000… --- ## [How to calculate disk space if 1tb of data ingested every day and log reteyis 180 days](https://discuss.elastic.co/t/how-to-calculate-disk-space-if-1tb-of-data-ingested-every-day-and-log-reteyis-180-days/322893)
**Author:** [@happylearning](https://discuss.elastic.co/u/happylearning)\ **Replies:** 1\ **Last updated:** [January 11, 2023, 9:06am UTC](https://discuss.elastic.co/t/how-to-calculate-disk-space-if-1tb-of-data-ingested-every-day-and-log-reteyis-180-days/322893 "2023-01-11T09:06:15Z")
How to calculate disk space if 1tb of data ingested every day and log reteyis 180 days --- ## [Combining phonetic filter with synonym\_graph filter](https://discuss.elastic.co/t/combining-phonetic-filter-with-synonym-graph-filter/322838)
**Author:** [@rune](https://discuss.elastic.co/u/rune)\ **Replies:** 0\ **Last updated:** [January 10, 2023, 1:12pm UTC](https://discuss.elastic.co/t/combining-phonetic-filter-with-synonym-graph-filter/322838 "2023-01-10T13:12:25Z")
So - I want to use the phonetic filter to transform the user query into phonetic hashes as it's supposed to do. F.ex. with "refined\_soundex" i get: pacemaker -\> P103080309 Then I want to take this hash value to my syno… --- ## [Different Doc count after reindexing](https://discuss.elastic.co/t/different-doc-count-after-reindexing/322826)
**Author:** [@cr\_168328](https://discuss.elastic.co/u/cr_168328)\ **Replies:** 5\ **Last updated:** [January 11, 2023, 8:10am UTC](https://discuss.elastic.co/t/different-doc-count-after-reindexing/322826 "2023-01-11T08:10:11Z")
I have an index user1. To add some fields, I created a new index and did reindexing using the query: POST /\_reindex?scroll=30m { "source": { "index": "user1" }, "dest": { "index": "user2" } } After rei… --- ## [Need Some Help Understanding Match Query Behavior](https://discuss.elastic.co/t/need-some-help-understanding-match-query-behavior/322870)
**Author:** [@denvaar](https://discuss.elastic.co/u/denvaar)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 7:10am UTC](https://discuss.elastic.co/t/need-some-help-understanding-match-query-behavior/322870 "2023-01-11T07:10:43Z")
I'm confused why the match query seen below is matching two documents rather than just one. I thought using the "and" operator would require all terms to be present in order for it to match. When I hit the explain endpo… --- ## [Search is not working properly on cluster 2 node setup](https://discuss.elastic.co/t/search-is-not-working-properly-on-cluster-2-node-setup/322851)
**Author:** [@arjunmalu](https://discuss.elastic.co/u/arjunmalu)\ **Replies:** 2\ **Last updated:** [January 11, 2023, 5:39am UTC](https://discuss.elastic.co/t/search-is-not-working-properly-on-cluster-2-node-setup/322851 "2023-01-11T05:39:36Z")
Hi Elastic team, I am using Elasticsearch service on 2 node setup. Both the nodes have default configuration (elasticsearch.yml). Case : Suppose there are 50 entries in a list to search from. A new entry ("newentry51")… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=316) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=318)