# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1034 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1035 --- ## [Issue in Syncing the data between nodes](https://discuss.elastic.co/t/issue-in-syncing-the-data-between-nodes/214059)
**Author:** [@baijugoradia](https://discuss.elastic.co/u/baijugoradia)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 12:19pm UTC](https://discuss.elastic.co/t/issue-in-syncing-the-data-between-nodes/214059 "2020-01-28T12:19:28Z")
We have 2 nodes node\_a and node\_b they both are on different servers where node\_a is the main node here all the data needs to be where as node\_b is a temp node where the data is being dump which needs to be synced to nod… --- ## [Composite with max query in source](https://discuss.elastic.co/t/composite-with-max-query-in-source/214117)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 12:01pm UTC](https://discuss.elastic.co/t/composite-with-max-query-in-source/214117 "2020-01-28T12:01:32Z")
Hi all, I am trying to get a max timestamp from a field inside an composite aggregation but I am unable to successfully do it. I have tried this so far: https://gist.github.com/pjanzen/ce752ac4f4fcdea4db9d9946237ba098 … --- ## [What happens under the hood for a bulk index call in Elasticsearch?](https://discuss.elastic.co/t/what-happens-under-the-hood-for-a-bulk-index-call-in-elasticsearch/216713)
**Author:** [@tlandelin](https://discuss.elastic.co/u/tlandelin)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 12:01pm UTC](https://discuss.elastic.co/t/what-happens-under-the-hood-for-a-bulk-index-call-in-elasticsearch/216713 "2020-01-28T12:01:27Z")
I'm tuning my ingestion pipeline to maximize throughput. I've read through the recommendations in the reference guide about how to tune for index speed (https://www.elastic.co/guide/en/elasticsearch/reference/master/tune… --- ## [To denormalize or not denormalize](https://discuss.elastic.co/t/to-denormalize-or-not-denormalize/213999)
**Author:** [@henrikjon](https://discuss.elastic.co/u/henrikjon)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 11:57am UTC](https://discuss.elastic.co/t/to-denormalize-or-not-denormalize/213999 "2020-01-28T11:57:57Z")
Greetings! :slight\_smile: First, thanks to Elastic for the great software they create. I am working with a product catalog search function that needs some changes to accomodate customer needs. The product in our setup … --- ## [Cluster development in elasticsearch 7.4.2](https://discuss.elastic.co/t/cluster-development-in-elasticsearch-7-4-2/216795)
**Author:** [@ELK\_CL](https://discuss.elastic.co/u/ELK_CL)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 11:16am UTC](https://discuss.elastic.co/t/cluster-development-in-elasticsearch-7-4-2/216795 "2020-01-28T11:16:02Z")
Hi, I have 3 servers of same specification(32GB RAM, 8 CORE, 200 GB Disk). I want to setup a cluster that can handle heavy aggregations and search operations. The data per day to be indexed is 500 MB , but there are ma… --- ## [Elasticsearch Cluster setup using 3 nodes(version 7.4.2)](https://discuss.elastic.co/t/elasticsearch-cluster-setup-using-3-nodes-version-7-4-2/216760)
**Author:** [@ELK\_CL](https://discuss.elastic.co/u/ELK_CL)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 6:13am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-setup-using-3-nodes-version-7-4-2/216760 "2020-01-28T06:13:57Z")
Hi, I have 3 servers of same specification(32GB RAM, 8 CORE, 200 GB Disk). I want to setup a cluster that can handle heavy aggregations and search operations. The data per day to be indexed is 500 MB , but there are ma… --- ## [Two nodes belongs to different different server not able to communicate each other within a Cluster](https://discuss.elastic.co/t/two-nodes-belongs-to-different-different-server-not-able-to-communicate-each-other-within-a-cluster/216815)
**Author:** [@GuptaN](https://discuss.elastic.co/u/GuptaN)\ **Replies:** 2\ **Last updated:** [January 28, 2020, 11:02am UTC](https://discuss.elastic.co/t/two-nodes-belongs-to-different-different-server-not-able-to-communicate-each-other-within-a-cluster/216815 "2020-01-28T11:02:38Z")
Hi, I just want to make two nodes cluster which are running on different different server. One node is for only master eligible node and other one will work as both master and data node. When i run the master eligible… --- ## [Snapshot running very slowly after upgrade from 5.x to 6.x](https://discuss.elastic.co/t/snapshot-running-very-slowly-after-upgrade-from-5-x-to-6-x/216782)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 5\ **Last updated:** [January 28, 2020, 10:32am UTC](https://discuss.elastic.co/t/snapshot-running-very-slowly-after-upgrade-from-5-x-to-6-x/216782 "2020-01-28T10:32:19Z")
Hi, I have upgraded my cluster from old version 5.x to 6.8.5 and every night I have a Python script that, using elasticsearch library, creates snapshot of several indices. the amount should be more or less 200 GB. Befo… --- ## [Elastic search query for getting array of values](https://discuss.elastic.co/t/elastic-search-query-for-getting-array-of-values/216808)
**Author:** [@Harmeet\_Kaur](https://discuss.elastic.co/u/Harmeet_Kaur)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 10:22am UTC](https://discuss.elastic.co/t/elastic-search-query-for-getting-array-of-values/216808 "2020-01-28T10:22:01Z")
Hi have written a query for getting the avg of values at a position in Elastic search elastic search payload : "userData": \[ { "sub":1234, "value":678,"condition" :"A" },{ "sub":1234, "value":678,"condition" :"B" }\] Qu… --- ## [Main ERROR ESJsonLayout contains an invalid element or a ttribute "esmessagefields"](https://discuss.elastic.co/t/main-error-esjsonlayout-contains-an-invalid-element-or-a-ttribute-esmessagefields/216752)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 5\ **Last updated:** [January 28, 2020, 9:00am UTC](https://discuss.elastic.co/t/main-error-esjsonlayout-contains-an-invalid-element-or-a-ttribute-esmessagefields/216752 "2020-01-28T09:00:36Z")
Hello, I have ES cluster with version 7.4.2. and i have version 7.3.1 also. ES clustering is working fine. But when i am trying to start ES services for version 7.3.1 . I am getting an error: Java HotSpot(TM) 64-Bit… --- ## [Elasticsearch not responding for CAT and other apis](https://discuss.elastic.co/t/elasticsearch-not-responding-for-cat-and-other-apis/216659)
**Author:** [@shivani\_aggarwal](https://discuss.elastic.co/u/shivani_aggarwal)\ **Replies:** 5\ **Last updated:** [January 28, 2020, 8:17am UTC](https://discuss.elastic.co/t/elasticsearch-not-responding-for-cat-and-other-apis/216659 "2020-01-28T08:17:26Z")
Hi, Env details: I am using ELK 7.0.1 in kubernetes environment. I have a 9 node cluster which has 3 master pods, 3 data pods and 3 client pods. Memory & cpu limits are configured as: Master pods: RAM: no limit, CP… --- ## [Publishing the elasticsearch Search APIs to a API gateway](https://discuss.elastic.co/t/publishing-the-elasticsearch-search-apis-to-a-api-gateway/214836)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 4\ **Last updated:** [January 28, 2020, 7:57am UTC](https://discuss.elastic.co/t/publishing-the-elasticsearch-search-apis-to-a-api-gateway/214836 "2020-01-28T07:57:09Z")
Is it possible to publish the Elasticsearch Search APIs to a API gateway like axway. --- ## [How to match search results with some conditions.?](https://discuss.elastic.co/t/how-to-match-search-results-with-some-conditions/216667)
**Author:** [@sany](https://discuss.elastic.co/u/sany)\ **Replies:** 0\ **Last updated:** [January 27, 2020, 1:52pm UTC](https://discuss.elastic.co/t/how-to-match-search-results-with-some-conditions/216667 "2020-01-27T13:52:59Z")
Hello..! I am going through the Function score, bool query, and many other things... but I want to do the following thing on this platform. I want to match one searching result with some condition (Like: some text )… --- ## [Import index patterns using requests.py](https://discuss.elastic.co/t/import-index-patterns-using-requests-py/216767)
**Author:** [@LimorA](https://discuss.elastic.co/u/LimorA)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 7:17am UTC](https://discuss.elastic.co/t/import-index-patterns-using-requests-py/216767 "2020-01-28T07:17:38Z")
Hi, I'm trying to import index patterns to Kibana. I tried the following command from the documentation: curl -X POST "my\_username:my\_password@10.2.25.209:5601/api/saved\_objects/\_import" -H "kbn-xsrf: true" --form fil… --- ## [Need help to correctly perform wildcard search on a field](https://discuss.elastic.co/t/need-help-to-correctly-perform-wildcard-search-on-a-field/216746)
**Author:** [@meallhour](https://discuss.elastic.co/u/meallhour)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 6:50am UTC](https://discuss.elastic.co/t/need-help-to-correctly-perform-wildcard-search-on-a-field/216746 "2020-01-28T06:50:19Z")
My mapping looks as below: { "abc\_history": { "mappings": { "abc-data-type": { "sData.Name": { "full\_name": "sData.Name", "mapping": { … --- ## [Few buckets are missing in terms aggregation result](https://discuss.elastic.co/t/few-buckets-are-missing-in-terms-aggregation-result/216706)
**Author:** [@darkwarrior](https://discuss.elastic.co/u/darkwarrior)\ **Replies:** 0\ **Last updated:** [January 27, 2020, 5:09pm UTC](https://discuss.elastic.co/t/few-buckets-are-missing-in-terms-aggregation-result/216706 "2020-01-27T17:09:10Z")
we have day wise elasticsearch indices they do contain millions of documents and we are maintaining one rollup index of them as well, so the problem is when I used aggregation on one field on rollup index, I got some b… --- ## [Elasticsearch restore API to validate restore](https://discuss.elastic.co/t/elasticsearch-restore-api-to-validate-restore/216438)
**Author:** [@akshatsharma](https://discuss.elastic.co/u/akshatsharma)\ **Replies:** 6\ **Last updated:** [January 28, 2020, 6:02am UTC](https://discuss.elastic.co/t/elasticsearch-restore-api-to-validate-restore/216438 "2020-01-28T06:02:19Z")
Hi, This is just a query about elasticsearch data restore. I am trying to handle any kinds of failure while restoring elasticsearch data. Is there any API/curl command that can give me a response of a failed restore al… --- ## [Query works perfectly in Kibana - but doesn't work at all in Elastalert](https://discuss.elastic.co/t/query-works-perfectly-in-kibana-but-doesnt-work-at-all-in-elastalert/216758)
**Author:** [@SecOpsWarrior](https://discuss.elastic.co/u/SecOpsWarrior)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 5:44am UTC](https://discuss.elastic.co/t/query-works-perfectly-in-kibana-but-doesnt-work-at-all-in-elastalert/216758 "2020-01-28T05:44:18Z")
So I have a little query im using to detect powershell being run. To keep it from being noisy, I have some regex to ignore certain powershell files and some other things. It works PERFECTLY in Kibana when i test it, but … --- ## [SAML OKTA on KIBANA not working](https://discuss.elastic.co/t/saml-okta-on-kibana-not-working/216551)
**Author:** [@hatakora](https://discuss.elastic.co/u/hatakora)\ **Replies:** 4\ **Last updated:** [January 28, 2020, 4:15am UTC](https://discuss.elastic.co/t/saml-okta-on-kibana-not-working/216551 "2020-01-28T04:15:28Z")
Hello , Please assist and thank you in advance for the help. Below are my configuration for everything . What am i doing wrong? Elasticsearch Config cluster.name: elasticsearch node.name: node-1 bootstrap.memory\_lock:… --- ## [Elasticsearch-dsl Python Library Using Scan](https://discuss.elastic.co/t/elasticsearch-dsl-python-library-using-scan/216743)
**Author:** [@ellje](https://discuss.elastic.co/u/ellje)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 1:46am UTC](https://discuss.elastic.co/t/elasticsearch-dsl-python-library-using-scan/216743 "2020-01-28T01:46:28Z")
My code here essentially is for hit in search.scan(): do something I believe it will use pagination/scrolling on this, and it seems that after a while, it will have this error message: 17:23:34 for hit in search.… --- ## [Indexing speed and SSD hardware](https://discuss.elastic.co/t/indexing-speed-and-ssd-hardware/216720)
**Author:** [@tlandelin](https://discuss.elastic.co/u/tlandelin)\ **Replies:** 4\ **Last updated:** [January 28, 2020, 12:37am UTC](https://discuss.elastic.co/t/indexing-speed-and-ssd-hardware/216720 "2020-01-28T00:37:32Z")
In Tune for indexing speed | Elasticsearch Guide \[master\] | Elastic it says If indexing is I/O bound, you should investigate giving more memory to the filesystem cache (see above) or buying faster drives. In particular… --- ## [Shard has exceeded the maximum number of retries](https://discuss.elastic.co/t/shard-has-exceeded-the-maximum-number-of-retries/215954)
**Author:** [@Jonathan\_Mendenhall](https://discuss.elastic.co/u/Jonathan_Mendenhall)\ **Replies:** 12\ **Last updated:** [January 27, 2020, 11:54pm UTC](https://discuss.elastic.co/t/shard-has-exceeded-the-maximum-number-of-retries/215954 "2020-01-27T23:54:53Z")
After moving one of our clusters from ES 6.4 to 7.5,we have been seeing frequent instances of shards failing to allocate because they hit the max of 5 retries. "explanation" : "shard has exceeded the maximum number of r… --- ## [Elasticsearch X-Opaque-Id Not in application.log Exception](https://discuss.elastic.co/t/elasticsearch-x-opaque-id-not-in-application-log-exception/216739)
**Author:** [@y\_xi](https://discuss.elastic.co/u/y_xi)\ **Replies:** 0\ **Last updated:** [January 27, 2020, 11:53pm UTC](https://discuss.elastic.co/t/elasticsearch-x-opaque-id-not-in-application-log-exception/216739 "2020-01-27T23:53:10Z")
In https://github.com/elastic/elasticsearch/issues/31521 Elasticsearch added X-Opaque-Id to slow log and access log. One more use case for me is to link X-Opaque-Id to exception trace whenever ES throws exception, which … --- ## [Indexing happens every 10 minutes. Elastic is idle in between](https://discuss.elastic.co/t/indexing-happens-every-10-minutes-elastic-is-idle-in-between/216728)
**Author:** [@arunp](https://discuss.elastic.co/u/arunp)\ **Replies:** 0\ **Last updated:** [January 27, 2020, 9:32pm UTC](https://discuss.elastic.co/t/indexing-happens-every-10-minutes-elastic-is-idle-in-between/216728 "2020-01-27T21:32:40Z")
I am processing past x days of data. Data ingestion is slow. I started looking at CPU and memory and they are at 40-50%. But indexing pattern is weird As seen in the pic , peak indexing of around 4k-5k happens once e… --- ## [Open, Search, Close Old Indexes](https://discuss.elastic.co/t/open-search-close-old-indexes/216715)
**Author:** [@Timothy\_Pangburn](https://discuss.elastic.co/u/Timothy_Pangburn)\ **Replies:** 2\ **Last updated:** [January 27, 2020, 7:26pm UTC](https://discuss.elastic.co/t/open-search-close-old-indexes/216715 "2020-01-27T19:26:43Z")
I have a request to search through 200 closed indexes for some data. Our current cluster is running well but at the edge of what it can handle. I don't want to open all the necessary indexes at one time to avoid overlo… --- ## [Elastic Features for Matching](https://discuss.elastic.co/t/elastic-features-for-matching/216696)
**Author:** [@rahwan1977](https://discuss.elastic.co/u/rahwan1977)\ **Replies:** 1\ **Last updated:** [January 27, 2020, 6:34pm UTC](https://discuss.elastic.co/t/elastic-features-for-matching/216696 "2020-01-27T18:34:24Z")
I am not used to the open source community, and I have some questions about the matching capabilities of ElasticSearch. Who shall I ask or contact and how? Best regards, Amr --- ## [Elasticsearch and overload](https://discuss.elastic.co/t/elasticsearch-and-overload/216700)
**Author:** [@kernelpanic](https://discuss.elastic.co/u/kernelpanic)\ **Replies:** 2\ **Last updated:** [January 27, 2020, 6:13pm UTC](https://discuss.elastic.co/t/elasticsearch-and-overload/216700 "2020-01-27T18:13:43Z")
Hello, our Elasticsearch cluster is used to store a variety of logs including Windows domain controller logs; I noticed that since adding Cisco netflow logs (which was over a million events ever 15 minutes) Elasticsearch… --- ## [Problem to Disable Xpack.Security.Enabled ECK K8S](https://discuss.elastic.co/t/problem-to-disable-xpack-security-enabled-eck-k8s/216705)
**Author:** [@Marinho\_DevOPs](https://discuss.elastic.co/u/Marinho_DevOPs)\ **Replies:** 0\ **Last updated:** [January 27, 2020, 5:05pm UTC](https://discuss.elastic.co/t/problem-to-disable-xpack-security-enabled-eck-k8s/216705 "2020-01-27T17:05:29Z")
Hi, I'm problem to disable authentication in ECK K8S. i set in config xpack.security.enabled to false. But my cluster not work Up. config: node.master: true node.data: true node.ingest: true xpack.security.enabled… --- ## [Rank\_feature takes a lot of space](https://discuss.elastic.co/t/rank-feature-takes-a-lot-of-space/216337)
**Author:** [@snakeztc](https://discuss.elastic.co/u/snakeztc)\ **Replies:** 1\ **Last updated:** [January 27, 2020, 11:16am UTC](https://discuss.elastic.co/t/rank-feature-takes-a-lot-of-space/216337 "2020-01-27T11:16:05Z")
I am using rank features for a customized ranking function. I used rank\_features as the data type, and a document usually contains about 5000 features out of 30K possible keys. It's taking way too much space though. For… --- ## [Can i use a custom field to update a elasticsearch doc?](https://discuss.elastic.co/t/can-i-use-a-custom-field-to-update-a-elasticsearch-doc/216652)
**Author:** [@noelzubinvictor](https://discuss.elastic.co/u/noelzubinvictor)\ **Replies:** 1\ **Last updated:** [January 27, 2020, 12:47pm UTC](https://discuss.elastic.co/t/can-i-use-a-custom-field-to-update-a-elasticsearch-doc/216652 "2020-01-27T12:47:21Z")
Say my doc looks something like this. { ... a: {...}, b: {...} } and i need to partially update a and b asychronously and maintain the order of updates. Since the exteranal version in elasticsearch is for the who… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1033) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1035)