# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=236 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 237 --- ## [NullPointerException when performing a Completion Suggester query with synonym analyzer, v8.5](https://discuss.elastic.co/t/nullpointerexception-when-performing-a-completion-suggester-query-with-synonym-analyzer-v8-5/336674)
**Author:** [@jacoMet](https://discuss.elastic.co/u/jacoMet)\ **Replies:** 3\ **Last updated:** [June 29, 2023, 10:47am UTC](https://discuss.elastic.co/t/nullpointerexception-when-performing-a-completion-suggester-query-with-synonym-analyzer-v8-5/336674 "2023-06-29T10:47:37Z")
I need to be able to perform Completion Suggest queries that are context dependent. When executing the query below: POST synonym\_file\_test/\_search { "\_source": "suggest", "suggest": { "my-suggest": { "prefix… --- ## [Use aggregate filter of logatash to find dynamic task-id](https://discuss.elastic.co/t/use-aggregate-filter-of-logatash-to-find-dynamic-task-id/337177)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 10:33am UTC](https://discuss.elastic.co/t/use-aggregate-filter-of-logatash-to-find-dynamic-task-id/337177 "2023-06-29T10:33:22Z")
Hi I want to use aggregate filter to find dynamic task-id https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html Here is the scenario I have log like below need to extract "Send&Receive dura… --- ## [Index historical time-series data into a data stream - ILM](https://discuss.elastic.co/t/index-historical-time-series-data-into-a-data-stream-ilm/337167)
**Author:** [@qcha](https://discuss.elastic.co/u/qcha)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 8:51am UTC](https://discuss.elastic.co/t/index-historical-time-series-data-into-a-data-stream-ilm/337167 "2023-06-29T08:51:54Z")
Hi everyone, My use case is the following : I have continuously produced time-series data + one year history (both outside Elastic). I want to index them into Elastic in such a way that data is deleted after one year (a… --- ## [Calculate total duration](https://discuss.elastic.co/t/calculate-total-duration/337148)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 5:00am UTC](https://discuss.elastic.co/t/calculate-total-duration/337148 "2023-06-29T05:00:19Z")
Hi How can I calculate duration of below log: 2021-07-15 00:00:01,869 INFO CUS.AbCD-AppService1-1234567 \[AppListener\] Receive Packet\[00\*\]: Kafka\[AppService1.APP1\] 2021-07-15 00:00:01,988 INFO CUS.AbCD-AppService1-1234… --- ## [Calculate transaction duration](https://discuss.elastic.co/t/calculate-transaction-duration/337147)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 4:53am UTC](https://discuss.elastic.co/t/calculate-transaction-duration/337147 "2023-06-29T04:53:54Z")
Hi i have log file like this: 2021-07-15 00:00:01,869 INFO client.InEE-server1-1234567 \[AppListener\] Receive Message\[A123\]: Q\[p1.APP\], IID\[null\], Cookie\[{"NODE\_SRC":"server0"}\] 2021-07-15 00:00:01,871 INFO client.InEE… --- ## [Find time gaps](https://discuss.elastic.co/t/find-time-gaps/337146)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 4:44am UTC](https://discuss.elastic.co/t/find-time-gaps/337146 "2023-06-29T04:44:33Z")
Hi i have log like this, need to find where unusuall time gap between "Packet Processed" and "Send Packet" that exist this is normal 001 2021-10-25 08:59:50,725 INFO CUS.AbCD-VW2-1234567890 \[FlowProcessorService\] Packe… --- ## [Find transaction in log](https://discuss.elastic.co/t/find-transaction-in-log/337143)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 4:34am UTC](https://discuss.elastic.co/t/find-transaction-in-log/337143 "2023-06-29T04:34:49Z")
Hi I have log like below need to extract "Send&Receive duration" and "send that has not respond". this is send 2021-07-15 00:00:01,800 INFO CUST.InAB-ServerApp-1234567 \[MyService\] Packet Processed: A\[50\] B\[0000211\] t… --- ## [Heartbeat on Kubernetes cluster](https://discuss.elastic.co/t/heartbeat-on-kubernetes-cluster/337140)
**Author:** [@jam\_mahmoudi](https://discuss.elastic.co/u/jam_mahmoudi)\ **Replies:** 0\ **Last updated:** [June 29, 2023, 4:17am UTC](https://discuss.elastic.co/t/heartbeat-on-kubernetes-cluster/337140 "2023-06-29T04:17:24Z")
Hi guys I have a question How should I define a heartbeat to monitor all pods in a Kubernetes cluster? Do I only need to install and configure Heartbeat on the worker nodes, or do I need to install and configure it on… --- ## [Segments info In Indics Stats](https://discuss.elastic.co/t/segments-info-in-indics-stats/337133)
**Author:** [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 11:14pm UTC](https://discuss.elastic.co/t/segments-info-in-indics-stats/337133 "2023-06-28T23:14:37Z")
I use ES 8.6. I created a lot indexes and added a lot data. but i don't know why segments's memory\_in\_bytes is zero. Can the value be always zero In ES 8.6 ??? --- ## [If there are multiple \`order\`, what is the priority?](https://discuss.elastic.co/t/if-there-are-multiple-order-what-is-the-priority/337045)
**Author:** [@kimjinyoung](https://discuss.elastic.co/u/kimjinyoung)\ **Replies:** 2\ **Last updated:** [June 28, 2023, 11:51pm UTC](https://discuss.elastic.co/t/if-there-are-multiple-order-what-is-the-priority/337045 "2023-06-28T23:51:25Z")
Hello, I'm using a translation because I can't speak English, so please teach me gently. Currently, I have something I would like to ask the order, so I am in a community. GET /\_search { "track\_scores": true, "sort" :… --- ## [How to check the default value of \`dynamic\_date\_formats\` and other mapping settings?](https://discuss.elastic.co/t/how-to-check-the-default-value-of-dynamic-date-formats-and-other-mapping-settings/337125)
**Author:** [@Mike\_Z](https://discuss.elastic.co/u/Mike_Z)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 6:53pm UTC](https://discuss.elastic.co/t/how-to-check-the-default-value-of-dynamic-date-formats-and-other-mapping-settings/337125 "2023-06-28T18:53:04Z")
New to Elasticsearch, and I am wondering how to check the current value of dynamic\_date\_formats. The online document, here, provides an example of setting customized value, but I did not find how to check its value. P… --- ## [Cluster shards unbalanced and keep moving shards around after upgrade to 8.8.1](https://discuss.elastic.co/t/cluster-shards-unbalanced-and-keep-moving-shards-around-after-upgrade-to-8-8-1/336573)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 18\ **Last updated:** [June 28, 2023, 3:56pm UTC](https://discuss.elastic.co/t/cluster-shards-unbalanced-and-keep-moving-shards-around-after-upgrade-to-8-8-1/336573 "2023-06-28T15:56:20Z")
Hello, Yesterday we upgraded our cluster from 8.5.1 to 8.8.1 and now the shards are unbalacend between the nodes and the cluster keeps moving shards around to try to balance it. I have a hot/warm architecture with 4 ho… --- ## [Is it possible to send pfsense syslogs from firewall to elastic agent on windows 11 home to my discover page?](https://discuss.elastic.co/t/is-it-possible-to-send-pfsense-syslogs-from-firewall-to-elastic-agent-on-windows-11-home-to-my-discover-page/337020)
**Author:** [@synthallthetime](https://discuss.elastic.co/u/synthallthetime)\ **Replies:** 0\ **Last updated:** [June 27, 2023, 6:55pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-pfsense-syslogs-from-firewall-to-elastic-agent-on-windows-11-home-to-my-discover-page/337020 "2023-06-27T18:55:48Z")
Hi there, I'm looking to see if it's possible to configure pfsense to send its syslogs into the pfsense integrations addin into my elastic agent on my windows 11 home endpoint. I have managed to set up logging for sysm… --- ## [Unable to do match query with new Java API client](https://discuss.elastic.co/t/unable-to-do-match-query-with-new-java-api-client/337065)
**Author:** [@p4charu](https://discuss.elastic.co/u/p4charu)\ **Replies:** 4\ **Last updated:** [June 28, 2023, 2:37pm UTC](https://discuss.elastic.co/t/unable-to-do-match-query-with-new-java-api-client/337065 "2023-06-28T14:37:03Z")
Hello! I am trying to do a match query similar to one below: "query": { "bool": { "must": \[ { "match": { "detected.tag": "chair" } } \] } } } This query gives me expec… --- ## [Store apm agent log in file](https://discuss.elastic.co/t/store-apm-agent-log-in-file/337099)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 2:33pm UTC](https://discuss.elastic.co/t/store-apm-agent-log-in-file/337099 "2023-06-28T14:33:41Z")
Hi Normally APM work like this: APMAgent\>APMServer\>Elastic\>kibana 1-Is it possible to store APMAgent log in file after that import in Elastic? Like this APMAgent\>file 2-Then feed log file to APMServer file\>APMServe… --- ## [Missing some machine learning jobs](https://discuss.elastic.co/t/missing-some-machine-learning-jobs/335678)
**Author:** [@queried1](https://discuss.elastic.co/u/queried1)\ **Replies:** 1\ **Last updated:** [June 28, 2023, 2:28pm UTC](https://discuss.elastic.co/t/missing-some-machine-learning-jobs/335678 "2023-06-28T14:28:40Z")
Hello! Recently I noticed that some ML jobs that start with "v3..." are missing. I can find other jobs that start with "v2..." under Machine Learning \> Anomaly Detection \> Jobs, but not "v3...". Some rules complain tha… --- ## [Rsyslog logs stop when any security is enabled](https://discuss.elastic.co/t/rsyslog-logs-stop-when-any-security-is-enabled/335869)
**Author:** [@Ryan\_Caputo](https://discuss.elastic.co/u/Ryan_Caputo)\ **Replies:** 7\ **Last updated:** [June 28, 2023, 2:04pm UTC](https://discuss.elastic.co/t/rsyslog-logs-stop-when-any-security-is-enabled/335869 "2023-06-28T14:04:29Z")
I have installed ELK 7.17.10 with podman, it works until I turn on security, even minimal security seems to block rsyslog from being received. What am I missing? --- ## [ES goes out of heap when issuing clusterstats (caused by CompletionStats)](https://discuss.elastic.co/t/es-goes-out-of-heap-when-issuing-clusterstats-caused-by-completionstats/336866)
**Author:** [@mgsag](https://discuss.elastic.co/u/mgsag)\ **Replies:** 11\ **Last updated:** [June 28, 2023, 1:21pm UTC](https://discuss.elastic.co/t/es-goes-out-of-heap-when-issuing-clusterstats-caused-by-completionstats/336866 "2023-06-28T13:21:56Z")
We observed this behavior in several of our production ElasticSearches and we were also able to reproduce it locally. If a database contains a lot of data for the completion-suggester, issuing a "\_cluster/stats?pretty"… --- ## [I want to know why the indices.id\_field\_data.enabled configuration is turned off by default](https://discuss.elastic.co/t/i-want-to-know-why-the-indices-id-field-data-enabled-configuration-is-turned-off-by-default/336834)
**Author:** [@gaorui](https://discuss.elastic.co/u/gaorui)\ **Replies:** 1\ **Last updated:** [June 28, 2023, 1:12pm UTC](https://discuss.elastic.co/t/i-want-to-know-why-the-indices-id-field-data-enabled-configuration-is-turned-off-by-default/336834 "2023-06-28T13:12:43Z")
I have a 200 million index, and I need to find out about 1 million of them based on certain conditions. I used the scroll api to query before, but I found that in the 8.x version, the scroll api is no longer recommended,… --- ## [Alerting in Marvel (or any other way)](https://discuss.elastic.co/t/alerting-in-marvel-or-any-other-way/337071)
**Author:** [@Mark\_S](https://discuss.elastic.co/u/Mark_S)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 12:01pm UTC](https://discuss.elastic.co/t/alerting-in-marvel-or-any-other-way/337071 "2023-06-28T12:01:36Z")
Is it possible to create alerts for various metrics (eg JVM usage, nodes volume usage, etc?) There have been similar questions a couple of years ago but I did not find a positive answer. Perhaps installing also Promethe… --- ## [Elastic Search Error after altering/modifying the elasticsearch.yml file](https://discuss.elastic.co/t/elastic-search-error-after-altering-modifying-the-elasticsearch-yml-file/337031)
**Author:** [@Zu\_kun](https://discuss.elastic.co/u/Zu_kun)\ **Replies:** 4\ **Last updated:** [June 28, 2023, 11:37am UTC](https://discuss.elastic.co/t/elastic-search-error-after-altering-modifying-the-elasticsearch-yml-file/337031 "2023-06-28T11:37:42Z")
Hi, I have installed Elasticsearch version 7.17.7 on ubuntu and I have noticed an issue whenever I alter or modify the elasticsearch.yml file within /etc/elasticsearch, In the elasticsearch.yml file, I want to bind the… --- ## [Elasticsearch with Docker](https://discuss.elastic.co/t/elasticsearch-with-docker/337068)
**Author:** [@anderstr1](https://discuss.elastic.co/u/anderstr1)\ **Replies:** 1\ **Last updated:** [June 28, 2023, 11:30am UTC](https://discuss.elastic.co/t/elasticsearch-with-docker/337068 "2023-06-28T11:30:39Z")
We have already set up Elasticsearch version 7.10.2 using the official docker image, and our single-node cluster is running fine. We need to use this specific Elasticsearch version since it is required by the Mediawiki a… --- ## [Dose Low Level Rest Client not cause the same connection pressure as Transport Client](https://discuss.elastic.co/t/dose-low-level-rest-client-not-cause-the-same-connection-pressure-as-transport-client/337056)
**Author:** [@emmning](https://discuss.elastic.co/u/emmning)\ **Replies:** 2\ **Last updated:** [June 28, 2023, 10:08am UTC](https://discuss.elastic.co/t/dose-low-level-rest-client-not-cause-the-same-connection-pressure-as-transport-client/337056 "2023-06-28T10:08:38Z")
We are using flink to write data to ES cluster.The transport client is used as the client in the flink task and the sniffer is configured.Each parallism of flink task will create a transport client, and each transport cl… --- ## [Empty aggregations, ES 6.7 nodes, RHLC upgraded from v6.3 to v6.8](https://discuss.elastic.co/t/empty-aggregations-es-6-7-nodes-rhlc-upgraded-from-v6-3-to-v6-8/337062)
**Author:** [@Hi\_Jonk](https://discuss.elastic.co/u/Hi_Jonk)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 9:37am UTC](https://discuss.elastic.co/t/empty-aggregations-es-6-7-nodes-rhlc-upgraded-from-v6-3-to-v6-8/337062 "2023-06-28T09:37:50Z")
As part of a rolling upgrade, I am trying upgrade my high level client from v6.3 to v6.8 while talking to v6.7 nodes. Search is working fine, however, the aggregation queries we were using with the v6.3 client have stopp… --- ## [Elastic 8.X license](https://discuss.elastic.co/t/elastic-8-x-license/337052)
**Author:** [@Norsu296](https://discuss.elastic.co/u/Norsu296)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 8:23am UTC](https://discuss.elastic.co/t/elastic-8-x-license/337052 "2023-06-28T08:23:25Z")
Hello, I'm trying to use elastic 8.8.0 on kubernetes with authentication using file realm, but it doesn;t work. In previous version (7.17) it was working, I didn't change anything in configuration. I checked password sh… --- ## [Total hits showing up as 0, but documents exist. 6.3 Rest high level client, 7.1 cluster](https://discuss.elastic.co/t/total-hits-showing-up-as-0-but-documents-exist-6-3-rest-high-level-client-7-1-cluster/337006)
**Author:** [@Hi\_Jonk](https://discuss.elastic.co/u/Hi_Jonk)\ **Replies:** 3\ **Last updated:** [June 28, 2023, 7:29am UTC](https://discuss.elastic.co/t/total-hits-showing-up-as-0-but-documents-exist-6-3-rest-high-level-client-7-1-cluster/337006 "2023-06-28T07:29:55Z")
Hi, I had a cluster that was running on ES v6.7 with a REST high level client v6.3. We want to upgrade, so as per this suggestion: Clarify high level REST client compatibility between 6 and 7 created a test cluster at … --- ## [CancelException with AsyncBulkLoad (Python helper)](https://discuss.elastic.co/t/cancelexception-with-asyncbulkload-python-helper/337038)
**Author:** [@ionFreeman](https://discuss.elastic.co/u/ionFreeman)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 5:58am UTC](https://discuss.elastic.co/t/cancelexception-with-asyncbulkload-python-helper/337038 "2023-06-28T05:58:19Z")
Hello! I have a little action generator, actually a bunch of coroutines I stich together with aiostream.merge(). I pass it into the AsyncBulkLoad. I haven't reproduced the behavior when I have a small number of test acti… --- ## [Elastic node crashing due to java.lang.OutOfMemoryError: Java heap space](https://discuss.elastic.co/t/elastic-node-crashing-due-to-java-lang-outofmemoryerror-java-heap-space/337034)
**Author:** [@sasvmware](https://discuss.elastic.co/u/sasvmware)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 5:22am UTC](https://discuss.elastic.co/t/elastic-node-crashing-due-to-java-lang-outofmemoryerror-java-heap-space/337034 "2023-06-28T05:22:56Z")
Hi Elasticsearch nodes are crashing due to java.lang.OutOfMemoryError: Java heap space. We have 10 GB memory in each node and as per formula total memory/2 -1 we have set JVM as 4. ava.lang.OutOfMemoryError: Java heap… --- ## [How to migrate data older than 30 day from a cluster to another cluster](https://discuss.elastic.co/t/how-to-migrate-data-older-than-30-day-from-a-cluster-to-another-cluster/337029)
**Author:** [@Tai\_Nguyen\_Huu](https://discuss.elastic.co/u/Tai_Nguyen_Huu)\ **Replies:** 0\ **Last updated:** [June 28, 2023, 2:48am UTC](https://discuss.elastic.co/t/how-to-migrate-data-older-than-30-day-from-a-cluster-to-another-cluster/337029 "2023-06-28T02:48:51Z")
Hi guys, I have a elasticsearch cluster, I wan to migrate data older than 30 day from a cluster to another cluster by automatic . --- ## [How to check total required heap for ES 8.8](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821)
**Author:** [@Geunmoon\_Oh](https://discuss.elastic.co/u/Geunmoon_Oh)\ **Replies:** 13\ **Last updated:** [June 28, 2023, 2:46am UTC](https://discuss.elastic.co/t/how-to-check-total-required-heap-for-es-8-8/336821 "2023-06-28T02:46:06Z")
To set my ES node's heap minimum, I referenced Size your shards (Size your shards | Elasticsearch Guide \[8.8\] | Elastic). "total\_deduplicated\_mapping\_size" : "4.1kb“ "total\_estimated\_overhead" : “13.5mb“ "extra heap f… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=235) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=237)