# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=528 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 529 --- ## [ML job behavior changed](https://discuss.elastic.co/t/ml-job-behavior-changed/291230)
**Author:** [@dao](https://discuss.elastic.co/u/dao)\ **Replies:** 6\ **Last updated:** [December 9, 2021, 3:30pm UTC](https://discuss.elastic.co/t/ml-job-behavior-changed/291230 "2021-12-09T15:30:46Z")
Hello, I have a ML jon that is started for a long time, but since a few monthes, there is a change (is it a bug?) Explanations: Here is what I have As you can see, It seems that the ML runs OK and detects some ano… --- ## [I want to set up the initial password without any environment variables after "xpack.security.enabled: true"](https://discuss.elastic.co/t/i-want-to-set-up-the-initial-password-without-any-environment-variables-after-xpack-security-enabled-true/291209)
**Author:** [@tadashi.oya](https://discuss.elastic.co/u/tadashi.oya)\ **Replies:** 1\ **Last updated:** [December 9, 2021, 3:02pm UTC](https://discuss.elastic.co/t/i-want-to-set-up-the-initial-password-without-any-environment-variables-after-xpack-security-enabled-true/291209 "2021-12-09T15:02:00Z")
Hi there. I'm trying to use T-Pot with "xpack.security.enabled: true". I want to set up passwords for Elasticsearch's built-in users (elastic, kibana\_system, and so on) as following conditions. Use as few interactive… --- ## [SSO and Subscription](https://discuss.elastic.co/t/sso-and-subscription/291275)
**Author:** [@suren\_es](https://discuss.elastic.co/u/suren_es)\ **Replies:** 7\ **Last updated:** [December 9, 2021, 2:13pm UTC](https://discuss.elastic.co/t/sso-and-subscription/291275 "2021-12-09T14:13:36Z")
I did download ELK stack tarballs, installed, configured. All is working fine. But when I started enabling SSO (via parameter xpack.security.authc.realms.oidc.oidc1 in config file), it says that this is not possible wit… --- ## [Is there a reason you cannot use \[runtime\_mappings\] in a \_count request?](https://discuss.elastic.co/t/is-there-a-reason-you-cannot-use-runtime-mappings-in-a-count-request/291334)
**Author:** [@sz\_bb](https://discuss.elastic.co/u/sz_bb)\ **Replies:** 0\ **Last updated:** [December 9, 2021, 1:54pm UTC](https://discuss.elastic.co/t/is-there-a-reason-you-cannot-use-runtime-mappings-in-a-count-request/291334 "2021-12-09T13:54:55Z")
I'd like to take advantage of \[runtime\_mappings\] in a search request, specifically leveraging that runtime\_field as a filter. In that scenario, while I may only have a size of 50 to return the top 50 docs, I also request… --- ## [Nested mapping or comparing fields from different indices \[SQL\]](https://discuss.elastic.co/t/nested-mapping-or-comparing-fields-from-different-indices-sql/291327)
**Author:** [@Katgust](https://discuss.elastic.co/u/Katgust)\ **Replies:** 0\ **Last updated:** [December 9, 2021, 12:44pm UTC](https://discuss.elastic.co/t/nested-mapping-or-comparing-fields-from-different-indices-sql/291327 "2021-12-09T12:44:36Z")
I'm aiming to replace our existing search functionality with Elasticsearch and I'm taking small steps towards that goal. Right now I'm trying to update a specific method but I would like some guidance. We have in our sy… --- ## [Bucket aggregation - calculate my own metric inside bucket](https://discuss.elastic.co/t/bucket-aggregation-calculate-my-own-metric-inside-bucket/291326)
**Author:** [@Detlic\_Detlicko](https://discuss.elastic.co/u/Detlic_Detlicko)\ **Replies:** 0\ **Last updated:** [December 9, 2021, 12:42pm UTC](https://discuss.elastic.co/t/bucket-aggregation-calculate-my-own-metric-inside-bucket/291326 "2021-12-09T12:42:42Z")
Hello, i collect data from my router and store into Elasticsearch. My data mappings are in this form : \< {"interface\_name":"keyword", "value":"double", "timestamp":"integer"}. \> I need to measure flow for each inter… --- ## [Ignore\_throttled parameter is deprecated](https://discuss.elastic.co/t/ignore-throttled-parameter-is-deprecated/291224)
**Author:** [@SirConius](https://discuss.elastic.co/u/SirConius)\ **Replies:** 2\ **Last updated:** [December 9, 2021, 9:00am UTC](https://discuss.elastic.co/t/ignore-throttled-parameter-is-deprecated/291224 "2021-12-09T09:00:43Z")
I've updated my Elasticsearch installation from 7.15.2 to 7.16, but when I open a dashboard I'm greeted with the following Warning: I looked in the deprecated log file before updating and this wasn't in there and I ha… --- ## [Elasticsearch nlpcn sql client getting eroor](https://discuss.elastic.co/t/elasticsearch-nlpcn-sql-client-getting-eroor/291299)
**Author:** [@SRIGURU\_VEL](https://discuss.elastic.co/u/SRIGURU_VEL)\ **Replies:** 0\ **Last updated:** [December 9, 2021, 7:56am UTC](https://discuss.elastic.co/t/elasticsearch-nlpcn-sql-client-getting-eroor/291299 "2021-12-09T07:56:09Z")
Hi Experts, I am Using ES 7.9.3 with NLPCN Sql client. SO I am getting Error when i try to GET index. {"error":{"root\_cause":\[{"type":"x\_content\_parse\_exception","reason":"\[1:1\] \[sql/query\] failed to parse object"}\],"… --- ## [How to fix cluster is red, reason: no\_valid\_shard\_copy after stopping data nodes?](https://discuss.elastic.co/t/how-to-fix-cluster-is-red-reason-no-valid-shard-copy-after-stopping-data-nodes/291206)
**Author:** [@bugb](https://discuss.elastic.co/u/bugb)\ **Replies:** 6\ **Last updated:** [December 9, 2021, 7:39am UTC](https://discuss.elastic.co/t/how-to-fix-cluster-is-red-reason-no-valid-shard-copy-after-stopping-data-nodes/291206 "2021-12-09T07:39:33Z")
After stopping some data nodes, my cluster becomes red and the reason is: no\_valid\_shard\_copy after stopping data nodes? How can I fix this? Update: should I use reroute? POST \_cluster/reroute?retry\_failed --- ## [Hyphenation token filter seems to ignore minimum subword size](https://discuss.elastic.co/t/hyphenation-token-filter-seems-to-ignore-minimum-subword-size/291295)
**Author:** [@mariskaas](https://discuss.elastic.co/u/mariskaas)\ **Replies:** 0\ **Last updated:** [December 9, 2021, 7:28am UTC](https://discuss.elastic.co/t/hyphenation-token-filter-seems-to-ignore-minimum-subword-size/291295 "2021-12-09T07:28:45Z")
Hello, I was looking to use the hyphenation decompounder filter. However during testing I found out it seems to be ignoring the minimum subword size option. Here is an example analyze I tried (normally I use a text fil… --- ## [How to list out all \*\*hidden\*\* indices in a cluster](https://discuss.elastic.co/t/how-to-list-out-all-hidden-indices-in-a-cluster/291291)
**Author:** [@Bruce\_Liu](https://discuss.elastic.co/u/Bruce_Liu)\ **Replies:** 1\ **Last updated:** [December 9, 2021, 5:59am UTC](https://discuss.elastic.co/t/how-to-list-out-all-hidden-indices-in-a-cluster/291291 "2021-12-09T05:59:15Z")
I'm trying to list out all hidden indices in a cluster. I tried GET \_cat/indices?expand\_wildcards=hidden,open&v, but even the status of the hidden index(index.hidden=true) is open, and the request lists out all open indi… --- ## [How to increase writing speed to an index using Spark ES](https://discuss.elastic.co/t/how-to-increase-writing-speed-to-an-index-using-spark-es/291165)
**Author:** [@ElasticQuestion\_1234](https://discuss.elastic.co/u/ElasticQuestion_1234)\ **Replies:** 11\ **Last updated:** [December 9, 2021, 5:43am UTC](https://discuss.elastic.co/t/how-to-increase-writing-speed-to-an-index-using-spark-es/291165 "2021-12-09T05:43:16Z")
Hi, I am trying to write a DataFrame with 10k rows and 31 columns into an Elasticsearch index using spark. I am using "JavaEsSparkSQL.saveToEs" function. There are 12 concurrent tasks. I used the default "es.batch.size.b… --- ## [Shards management in ElasticSearch](https://discuss.elastic.co/t/shards-management-in-elasticsearch/291252)
**Author:** [@Akanksha\_Pandey](https://discuss.elastic.co/u/Akanksha_Pandey)\ **Replies:** 5\ **Last updated:** [December 9, 2021, 4:49am UTC](https://discuss.elastic.co/t/shards-management-in-elasticsearch/291252 "2021-12-09T04:49:41Z")
I'm using Elasticsearch version 7.9.2 as a single node docker cluster for my production as well as development environment. Every once in a while I'm getting these two errors frequently. Error1: ElasticsearchError erro… --- ## [Elastic stack can monitor app metric](https://discuss.elastic.co/t/elastic-stack-can-monitor-app-metric/291283)
**Author:** [@wanfenjia](https://discuss.elastic.co/u/wanfenjia)\ **Replies:** 1\ **Last updated:** [December 9, 2021, 4:37am UTC](https://discuss.elastic.co/t/elastic-stack-can-monitor-app-metric/291283 "2021-12-09T04:37:14Z")
Hi Elastic stack team, I need monitor server app(for example, http access number, http reposend time,etc), Could u pls help me to fix it, and have u some doucment to share for me ? many thanks! --- ## [Clone index failing](https://discuss.elastic.co/t/clone-index-failing/291249)
**Author:** [@madshov](https://discuss.elastic.co/u/madshov)\ **Replies:** 1\ **Last updated:** [December 8, 2021, 10:33pm UTC](https://discuss.elastic.co/t/clone-index-failing/291249 "2021-12-08T22:33:51Z")
Hi all, I'm getting an error when I try to clone my index called myindex into myindex\_clone. I have set up a 3 node cluster on my machine, and I'm using the (Go) SDK client to call the CloneIndex() function. However I'm… --- ## [How to do join field/routing in bigquery to elasticsearch dataflow template?](https://discuss.elastic.co/t/how-to-do-join-field-routing-in-bigquery-to-elasticsearch-dataflow-template/291269)
**Author:** [@rickbenchsci](https://discuss.elastic.co/u/rickbenchsci)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 10:27pm UTC](https://discuss.elastic.co/t/how-to-do-join-field-routing-in-bigquery-to-elasticsearch-dataflow-template/291269 "2021-12-08T22:27:29Z")
Hi, I am using the dataflow template provided by google for importing data from BigQuery to Elasticsearch. see here We managed to get the thing working (sans API key. adding basic auth overwrites the need for it), and … --- ## [Elasticsearch-PHP requires cURL, or a custom HTTP handler](https://discuss.elastic.co/t/elasticsearch-php-requires-curl-or-a-custom-http-handler/291250)
**Author:** [@tranha31](https://discuss.elastic.co/u/tranha31)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 6:16pm UTC](https://discuss.elastic.co/t/elasticsearch-php-requires-curl-or-a-custom-http-handler/291250 "2021-12-08T18:16:58Z")
I try to connect Elasticsearch from php with Elasticsearch-php Library. But it notices 127.0.0.1:43202 \[500\]: GET /main.php?q=a - Uncaught RuntimeException: Elasticsearch-PHP requires cURL, or a custom HTTP handler. in… --- ## [SORTING, SUM, PAGINATION, and AGGREGATION all in one!](https://discuss.elastic.co/t/sorting-sum-pagination-and-aggregation-all-in-one/291248)
**Author:** [@michael\_jaskiewicz](https://discuss.elastic.co/u/michael_jaskiewicz)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 6:04pm UTC](https://discuss.elastic.co/t/sorting-sum-pagination-and-aggregation-all-in-one/291248 "2021-12-08T18:04:42Z")
'm using Elasticsearch and I see questions now and then about doing some aggregations with sorting or aggregations with paging, but I never see anything GROUP\_BY, SUM, SORT, and PAGINATION together. If I were to write wh… --- ## [PySpark - How to read timestamp date\_nanos from ElasticSearch to Spark?](https://discuss.elastic.co/t/pyspark-how-to-read-timestamp-date-nanos-from-elasticsearch-to-spark/260020)
**Author:** [@ngocdangrby](https://discuss.elastic.co/u/ngocdangrby)\ **Replies:** 1\ **Last updated:** [December 8, 2021, 4:42pm UTC](https://discuss.elastic.co/t/pyspark-how-to-read-timestamp-date-nanos-from-elasticsearch-to-spark/260020 "2021-12-08T16:42:17Z")
Hi all, I need to read the timestamp from ElasticSearch to Spark RDD or DataFrame. I can read other fields (text, interger..) but timestamp i can not. Spark v2.4.7 Python: v3.6.11 ElasticSearch: v7.10.0 Here is my c… --- ## [Setup hot-delete policy for Elastichsearch index without using templates](https://discuss.elastic.co/t/setup-hot-delete-policy-for-elastichsearch-index-without-using-templates/291242)
**Author:** [@obadakhalili](https://discuss.elastic.co/u/obadakhalili)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 4:24pm UTC](https://discuss.elastic.co/t/setup-hot-delete-policy-for-elastichsearch-index-without-using-templates/291242 "2021-12-08T16:24:16Z")
I'm trying to setup an ILM policy that deletes my index every 3 days, and creates a new one. Here's my attempt for achieving that: const policyName = "hot\_delete\_twitter\_stream\_indexing\_policy" await ESClient.ilm.p… --- ## [Create an error log with an ingest pipeline when there is an indexing error](https://discuss.elastic.co/t/create-an-error-log-with-an-ingest-pipeline-when-there-is-an-indexing-error/291069)
**Author:** [@GinkoLucas](https://discuss.elastic.co/u/GinkoLucas)\ **Replies:** 6\ **Last updated:** [December 8, 2021, 3:12pm UTC](https://discuss.elastic.co/t/create-an-error-log-with-an-ingest-pipeline-when-there-is-an-indexing-error/291069 "2021-12-08T15:12:48Z")
Hi, I would like to create an error log when a document is not suitable for my mapping. For example if I send a document containing an unknown field in my mapping (dynamic:false), I would like to be able to create an e… --- ## [ES Hadoop push down aggregations or not?](https://discuss.elastic.co/t/es-hadoop-push-down-aggregations-or-not/277514)
**Author:** [@priamai](https://discuss.elastic.co/u/priamai)\ **Replies:** 1\ **Last updated:** [December 8, 2021, 2:09pm UTC](https://discuss.elastic.co/t/es-hadoop-push-down-aggregations-or-not/277514 "2021-12-08T14:09:10Z")
I am on ELK 7.13.0 and I have a super very slow query: events\_df = es\_reader.load("priam\_unified\_host-{0}/unified-host".format(date)) counts\_df = events\_df.where('EventID == 4688').groupBy(col("LogHost"),col("ProcessNa… --- ## [How to removes master-eligible nodes from elasticsearch cluster when master is not discovered](https://discuss.elastic.co/t/how-to-removes-master-eligible-nodes-from-elasticsearch-cluster-when-master-is-not-discovered/291210)
**Author:** [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Replies:** 1\ **Last updated:** [December 8, 2021, 11:34am UTC](https://discuss.elastic.co/t/how-to-removes-master-eligible-nodes-from-elasticsearch-cluster-when-master-is-not-discovered/291210 "2021-12-08T11:34:46Z")
I have an Elasticsearch cluster with three master nodes. Two of my master nodes VM's got corrupted so I am trying to remove them so that I can add newly created master node but I am unable to do so using the Voting confi… --- ## [Elasticsearch Watcher scheduler](https://discuss.elastic.co/t/elasticsearch-watcher-scheduler/291135)
**Author:** [@Bigbad](https://discuss.elastic.co/u/Bigbad)\ **Replies:** 2\ **Last updated:** [December 8, 2021, 11:13am UTC](https://discuss.elastic.co/t/elasticsearch-watcher-scheduler/291135 "2021-12-08T11:13:17Z")
Hello, Is it possible to configure watcher for business days and hours I have have watcher and I want it to run every 30mn from monday to friday and from 7h to 19h. The next expression don't work "cron":"\*/30 7-19 \* … --- ## [User session duration calculation](https://discuss.elastic.co/t/user-session-duration-calculation/290993)
**Author:** [@ramous-tun](https://discuss.elastic.co/u/ramous-tun)\ **Replies:** 2\ **Last updated:** [December 8, 2021, 10:20am UTC](https://discuss.elastic.co/t/user-session-duration-calculation/290993 "2021-12-08T10:20:46Z")
Hi, I'm trying to get user session duration by day for the last 30 days from windows logs, so I used Transform but something wrong in my script, I always get duration for all period recorded: from the first login to the… --- ## [Defying synonyms for roman numbers doesn't seem to work](https://discuss.elastic.co/t/defying-synonyms-for-roman-numbers-doesnt-seem-to-work/291126)
**Author:** [@eliezra236](https://discuss.elastic.co/u/eliezra236)\ **Replies:** 2\ **Last updated:** [December 8, 2021, 10:18am UTC](https://discuss.elastic.co/t/defying-synonyms-for-roman-numbers-doesnt-seem-to-work/291126 "2021-12-08T10:18:09Z")
Hi, I've been trying to set up roman numbers synonyms the following way: PUT testing\_index { "settings": { "analysis": { "analyzer": { "my\_custome\_analyzer": { "type": "custom", … --- ## [Problem with Elastic Docker login](https://discuss.elastic.co/t/problem-with-elastic-docker-login/291208)
**Author:** [@Pascal\_Paulis](https://discuss.elastic.co/u/Pascal_Paulis)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 10:12am UTC](https://discuss.elastic.co/t/problem-with-elastic-docker-login/291208 "2021-12-08T10:12:36Z")
Hi, I'm trying to upgrade from 7.12 to 7.15. It seems that 7.14 was the last release available on hub.docker.com and that all newer images are only available on the Elastic Docker repository. Can anyboby tell me how t… --- ## [Need to include an external privacy notice in OIDC](https://discuss.elastic.co/t/need-to-include-an-external-privacy-notice-in-oidc/291204)
**Author:** [@Ali\_Muhammad](https://discuss.elastic.co/u/Ali_Muhammad)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 9:43am UTC](https://discuss.elastic.co/t/need-to-include-an-external-privacy-notice-in-oidc/291204 "2021-12-08T09:43:45Z")
Hello community. Is there a way to include an external redirect to privacy notice. This strategy will follow the usual oidc authentication flow and will act as a consent form after the user has authenticated. Does Elast… --- ## [How to get positions of the matched string?](https://discuss.elastic.co/t/how-to-get-positions-of-the-matched-string/291199)
**Author:** [@wildansupernova](https://discuss.elastic.co/u/wildansupernova)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 9:34am UTC](https://discuss.elastic.co/t/how-to-get-positions-of-the-matched-string/291199 "2021-12-08T09:34:20Z")
Sorry I am new to Elasticsearch I have already searched through internet, some technique I found useful is around highlighting. But I want to ask is there any way to fulfill this scenario other than using highlighting. … --- ## [EQL Query for User account creation and deletion](https://discuss.elastic.co/t/eql-query-for-user-account-creation-and-deletion/291200)
**Author:** [@anaghadeoreofficial](https://discuss.elastic.co/u/anaghadeoreofficial)\ **Replies:** 0\ **Last updated:** [December 8, 2021, 9:35am UTC](https://discuss.elastic.co/t/eql-query-for-user-account-creation-and-deletion/291200 "2021-12-08T09:35:40Z")
I am currently working on one use case for winlogbeat data. Usecase is as follows: Use Case: Trigger alert if the user account is deleted within one hour after creation. Trying to understand the EQL syntax for the abov… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=527) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=529)