# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1078 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1079 --- ## [Why is ElasticSearch SQL excruciatingly slow?](https://discuss.elastic.co/t/why-is-elasticsearch-sql-excruciatingly-slow/209276)
**Author:** [@Nick\_Walton](https://discuss.elastic.co/u/Nick_Walton)\ **Replies:** 14\ **Last updated:** [December 2, 2019, 9:14pm UTC](https://discuss.elastic.co/t/why-is-elasticsearch-sql-excruciatingly-slow/209276 "2019-12-02T21:14:19Z")
I have a very simple SQL query select hashstring from "flink-index-deduplicated" where bearing = 60 over about 118 million documents of which around 1/19th will match bearing = 60. The bearing values are defined as Lon… --- ## [Cosine similarity not working in elastic search](https://discuss.elastic.co/t/cosine-similarity-not-working-in-elastic-search/209702)
**Author:** [@Divyanshu\_Pandey](https://discuss.elastic.co/u/Divyanshu_Pandey)\ **Replies:** 4\ **Last updated:** [December 2, 2019, 8:52pm UTC](https://discuss.elastic.co/t/cosine-similarity-not-working-in-elastic-search/209702 "2019-12-02T20:52:21Z")
This is my my query : script\_query = { "function\_score": { "query" : { "match\_all": {} }, "script\_score": {"script": { "inline": "cosineSimilarity(params.query\_vector, doc\['title\_vector'\]) + 1.0", "params": { "q… --- ## [Custom function for Text Similarity Search](https://discuss.elastic.co/t/custom-function-for-text-similarity-search/210158)
**Author:** [@Loreto\_Parisi](https://discuss.elastic.co/u/Loreto_Parisi)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 8:49pm UTC](https://discuss.elastic.co/t/custom-function-for-text-similarity-search/210158 "2019-12-02T20:49:10Z")
I'm using a dense\_vector and cosineSimilarity to get documents similarity following the good tutorial here script\_query = { "script\_score": { "query": {"match\_all": {}}, "script": { … --- ## [Update timestamp only if partial update changes other fields](https://discuss.elastic.co/t/update-timestamp-only-if-partial-update-changes-other-fields/210243)
**Author:** [@Anver\_S](https://discuss.elastic.co/u/Anver_S)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 8:41pm UTC](https://discuss.elastic.co/t/update-timestamp-only-if-partial-update-changes-other-fields/210243 "2019-12-02T20:41:02Z")
Hi, I am making a partial update on a document and I'd want to update a corresponding timestamp field only if any of the other fields defined in partial update have changed. If I explicitly set timestamp field to "now" … --- ## [Help with Advanced Watcher](https://discuss.elastic.co/t/help-with-advanced-watcher/210201)
**Author:** [@Cho1409](https://discuss.elastic.co/u/Cho1409)\ **Replies:** 2\ **Last updated:** [December 2, 2019, 7:40pm UTC](https://discuss.elastic.co/t/help-with-advanced-watcher/210201 "2019-12-02T19:40:30Z")
Hi guys, I'm trying to create a watcher for a simple query that goes like this: field: "service\_X" and event.level: "ERROR" The problem is, when creating the watcher itself, it doesn't allow me to use multiple fields: … --- ## [Where can i find a list of available options for elasticssearch.yml?](https://discuss.elastic.co/t/where-can-i-find-a-list-of-available-options-for-elasticssearch-yml/210232)
**Author:** [@the-vampiire](https://discuss.elastic.co/u/the-vampiire)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 6:51pm UTC](https://discuss.elastic.co/t/where-can-i-find-a-list-of-available-options-for-elasticssearch-yml/210232 "2019-12-02T18:51:04Z")
going through the configuration reference there are a few properties listed. i did find this generic config file on github there was another property discovery.type listed on the docker elasticsearch image example too. … --- ## [Alias not created based on index name - beats output](https://discuss.elastic.co/t/alias-not-created-based-on-index-name-beats-output/210230)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 6:48pm UTC](https://discuss.elastic.co/t/alias-not-created-based-on-index-name-beats-output/210230 "2019-12-02T18:48:09Z")
Hi there, I'm trying to make a custom ILM and Index Template for my incoming filebeat and metricbeat shippers. I've got it now so that the beats send to a custom index in ElasticSearch called "filebeat" and "metricbeat… --- ## [Unable to parse data when I try to POST to Elasticsearch](https://discuss.elastic.co/t/unable-to-parse-data-when-i-try-to-post-to-elasticsearch/210227)
**Author:** [@chari](https://discuss.elastic.co/u/chari)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 6:14pm UTC](https://discuss.elastic.co/t/unable-to-parse-data-when-i-try-to-post-to-elasticsearch/210227 "2019-12-02T18:14:10Z")
page='ABCD' header='{"content":"' footer='", "my-join-field":{"name": "my\_page","parent": "1"}}' !curl -X POST "localhost:9200/docpage/\_doc/?refresh&pretty" -H 'Content-Type: application/json' -d"$header$page$footer" … --- ## [Elastic node crashing with JVM dump file error generated after a while](https://discuss.elastic.co/t/elastic-node-crashing-with-jvm-dump-file-error-generated-after-a-while/209898)
**Author:** [@viniciof](https://discuss.elastic.co/u/viniciof)\ **Replies:** 7\ **Last updated:** [December 2, 2019, 6:10pm UTC](https://discuss.elastic.co/t/elastic-node-crashing-with-jvm-dump-file-error-generated-after-a-while/209898 "2019-12-02T18:10:02Z")
Hello everyone, I've started to notice very recently that some of our ElasticSearch services crash all of a sudden and they generate the attached dumpfile. Gist here. Please help as our nodes are currently restarting v… --- ## [Loading pySpark DataFrame into Elasticsearch - keep getting illegal argument error](https://discuss.elastic.co/t/loading-pyspark-dataframe-into-elasticsearch-keep-getting-illegal-argument-error/208348)
**Author:** [@dkajtoch](https://discuss.elastic.co/u/dkajtoch)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 6:04pm UTC](https://discuss.elastic.co/t/loading-pyspark-dataframe-into-elasticsearch-keep-getting-illegal-argument-error/208348 "2019-12-02T18:04:52Z")
I am trying to send data stored in pySpark Dataframe directly into Elasticsearch. I found some code snippets online on how to do this and basically it looks like this: # df - DataFrame df.write.format("org.elasticsearch… --- ## [Error when loading Google Cloud Storage credentials file](https://discuss.elastic.co/t/error-when-loading-google-cloud-storage-credentials-file/94370)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 11\ **Last updated:** [December 2, 2019, 4:14pm UTC](https://discuss.elastic.co/t/error-when-loading-google-cloud-storage-credentials-file/94370 "2019-12-02T16:14:22Z")
I’m trying to get Google Cloud Storage Repository Plugin going: I installed repository-gcs plugin: # /opt/elasticsearch/bin/elasticsearch-plugin install repository-gcs -\> Downloading repository-gcs from elastic \[======… --- ## [Elastic Search - All Parents and Matching Nested](https://discuss.elastic.co/t/elastic-search-all-parents-and-matching-nested/209088)
**Author:** [@ci\_ms](https://discuss.elastic.co/u/ci_ms)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 3:47pm UTC](https://discuss.elastic.co/t/elastic-search-all-parents-and-matching-nested/209088 "2019-12-02T15:47:52Z")
Our use case is having a list of products and in a nested field have the last purchase date by customer number. When a customer searches for products we want to return all items that match their search term. The inner\_hi… --- ## [Creating ILM in a automated way](https://discuss.elastic.co/t/creating-ilm-in-a-automated-way/209463)
**Author:** [@ganesh2](https://discuss.elastic.co/u/ganesh2)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 3:39pm UTC](https://discuss.elastic.co/t/creating-ilm-in-a-automated-way/209463 "2019-12-02T15:39:48Z")
Hi Team, I'm trying to setup a Elasticstack in a automated way wherein i would like to automate configuring the ILM policy as well. I am referring to the link below for policy creation. https://www.elastic.co/guide/en/e… --- ## [How does script caching work when the script is updated?](https://discuss.elastic.co/t/how-does-script-caching-work-when-the-script-is-updated/210199)
**Author:** [@brabster](https://discuss.elastic.co/u/brabster)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 2:46pm UTC](https://discuss.elastic.co/t/how-does-script-caching-work-when-the-script-is-updated/210199 "2019-12-02T14:46:48Z")
Does Elasticsearch recognise that a previously cached script has been updated, and replace the cache entry with the new script compilation? I've seen code that manages script caching client-side, explicitly uploading ne… --- ## [Keeping latest document only](https://discuss.elastic.co/t/keeping-latest-document-only/207721)
**Author:** [@shankaran](https://discuss.elastic.co/u/shankaran)\ **Replies:** 2\ **Last updated:** [December 2, 2019, 2:40pm UTC](https://discuss.elastic.co/t/keeping-latest-document-only/207721 "2019-12-02T14:40:23Z")
in My application Every day 6 times (4hrs per run) index will be updated by latest exchange rates documents. Any one could guide me on best approach how to delete my old documents (having SourceCurrency=USD) from logsta… --- ## [Wrong Timezone in Painless Function](https://discuss.elastic.co/t/wrong-timezone-in-painless-function/209905)
**Author:** [@TomTom](https://discuss.elastic.co/u/TomTom)\ **Replies:** 4\ **Last updated:** [December 2, 2019, 2:33pm UTC](https://discuss.elastic.co/t/wrong-timezone-in-painless-function/209905 "2019-12-02T14:33:55Z")
The government of our country recently canceled daylight saving time, and now functions using timezone return to America / Sao\_Paulo the value -02 instead of -03, but if I test with America / Buenos\_Aires, the value retu… --- ## [BulkProcessor retry on connection timeout](https://discuss.elastic.co/t/bulkprocessor-retry-on-connection-timeout/210188)
**Author:** [@Shivesh\_Kumar](https://discuss.elastic.co/u/Shivesh_Kumar)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 1:27pm UTC](https://discuss.elastic.co/t/bulkprocessor-retry-on-connection-timeout/210188 "2019-12-02T13:27:35Z")
Is there any way to retry in case of a socket or connection timeout? --- ## [What is the best approach for streaming large number of docs from ES 7.1v scrolling vs slicing?](https://discuss.elastic.co/t/what-is-the-best-approach-for-streaming-large-number-of-docs-from-es-7-1v-scrolling-vs-slicing/210115)
**Author:** [@yash.tandon](https://discuss.elastic.co/u/yash.tandon)\ **Replies:** 8\ **Last updated:** [December 2, 2019, 9:40am UTC](https://discuss.elastic.co/t/what-is-the-best-approach-for-streaming-large-number-of-docs-from-es-7-1v-scrolling-vs-slicing/210115 "2019-12-02T09:40:24Z")
--- ## [Simple queries takes lots of time and uses 100% cpu](https://discuss.elastic.co/t/simple-queries-takes-lots-of-time-and-uses-100-cpu/206243)
**Author:** [@aimarjg](https://discuss.elastic.co/u/aimarjg)\ **Replies:** 33\ **Last updated:** [December 2, 2019, 9:38am UTC](https://discuss.elastic.co/t/simple-queries-takes-lots-of-time-and-uses-100-cpu/206243 "2019-12-02T09:38:20Z")
Hi! I'm having an issue with filtering data gathered using winlogbeat to ES - whenever I want to filter for single host.name or winlog.computer\_name (kinda the same) only for last 24 hours - it takes more than 60 000ms … --- ## [Performance difference in 'must' and 'must\_not'](https://discuss.elastic.co/t/performance-difference-in-must-and-must-not/210147)
**Author:** [@Prasiddha](https://discuss.elastic.co/u/Prasiddha)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 9:33am UTC](https://discuss.elastic.co/t/performance-difference-in-must-and-must-not/210147 "2019-12-02T09:33:57Z")
I wanted to know the performance difference between using 'must' clause and 'must\_not'. I am getting different timings trying from both of them. Suppose I have 10 groups, and I want to make 5 groups accessible to a use… --- ## [Search multiple words within the same paragraph in the target document?](https://discuss.elastic.co/t/search-multiple-words-within-the-same-paragraph-in-the-target-document/210093)
**Author:** [@ddream777](https://discuss.elastic.co/u/ddream777)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 9:32am UTC](https://discuss.elastic.co/t/search-multiple-words-within-the-same-paragraph-in-the-target-document/210093 "2019-12-02T09:32:28Z")
I was wondering whether it's possible to set a condition so as to limit the range of the search to within a paragraph, the query being a multiple-word one. --- ## [Count in other index based on Current Index field](https://discuss.elastic.co/t/count-in-other-index-based-on-current-index-field/209971)
**Author:** [@Vinothkumar\_Ganeshan](https://discuss.elastic.co/u/Vinothkumar_Ganeshan)\ **Replies:** 5\ **Last updated:** [December 2, 2019, 9:31am UTC](https://discuss.elastic.co/t/count-in-other-index-based-on-current-index-field/209971 "2019-12-02T09:31:00Z")
Scenario: Index - 2019-05\*, 2019-06\*, 2019-07\*, 2019-08\*, 2019-09\* I have ID field in all the index which Contains Millions of documents. I want to analyse, how many id appeared in 2019-05\* is repeated in other … --- ## [Only create an alert when the query gets three hits from the same device](https://discuss.elastic.co/t/only-create-an-alert-when-the-query-gets-three-hits-from-the-same-device/209711)
**Author:** [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\ **Replies:** 3\ **Last updated:** [December 2, 2019, 9:17am UTC](https://discuss.elastic.co/t/only-create-an-alert-when-the-query-gets-three-hits-from-the-same-device/209711 "2019-12-02T09:17:58Z")
Hi, We want to create an alert when there has been over three failed logon events. Our query looks like this: POST /logstash-test-2019-11/\_search { "query": { "bool": { "must": \[ { … --- ## [Accessing array of object in painless script , search query \[ES 6.4\]](https://discuss.elastic.co/t/accessing-array-of-object-in-painless-script-search-query-es-6-4/209559)
**Author:** [@kushal](https://discuss.elastic.co/u/kushal)\ **Replies:** 3\ **Last updated:** [December 2, 2019, 6:35am UTC](https://discuss.elastic.co/t/accessing-array-of-object-in-painless-script-search-query-es-6-4/209559 "2019-12-02T06:35:56Z")
Documents have an array of objects, which i need to iterate over in order to compute the final score. Basic mapping is following \< { "settings": { "number\_of\_shards": 1, "numbe… --- ## [Sum aggregation on field by distinct value of another field with high cardinality](https://discuss.elastic.co/t/sum-aggregation-on-field-by-distinct-value-of-another-field-with-high-cardinality/210111)
**Author:** [@artur](https://discuss.elastic.co/u/artur)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 5:51am UTC](https://discuss.elastic.co/t/sum-aggregation-on-field-by-distinct-value-of-another-field-with-high-cardinality/210111 "2019-12-02T05:51:40Z")
I have a document like this: { "\_id": 1, "factor": 100, "field\_with\_high\_cardinality": 1000 } I know that for each unique field\_with\_high\_cardinality the factor is always the same. I need to calculate… --- ## [Elasticsearch not starting when i deleted few indices](https://discuss.elastic.co/t/elasticsearch-not-starting-when-i-deleted-few-indices/210110)
**Author:** [@pankaj90](https://discuss.elastic.co/u/pankaj90)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 5:51am UTC](https://discuss.elastic.co/t/elasticsearch-not-starting-when-i-deleted-few-indices/210110 "2019-12-02T05:51:18Z")
I accidentally deleted few indices from elastic-search data but not it's not starting the service. i'm getting error while starting the service which was " attempt to resurrect connection to dead ES instances" when i che… --- ## [POST \_analyze error](https://discuss.elastic.co/t/post-analyze-error/210054)
**Author:** [@xyx](https://discuss.elastic.co/u/xyx)\ **Replies:** 5\ **Last updated:** [December 2, 2019, 5:34am UTC](https://discuss.elastic.co/t/post-analyze-error/210054 "2019-12-02T05:34:58Z")
The analyzer does not work，So who can help me.Thanks!Here is my error and example: (1)error: { "error": { "root\_cause": \[ { "type": "x\_content\_parse\_exception", "reason": "\[3:12\] \[analyze\_request\] failed to parse … --- ## [Elasticsearch takes long time to update simple index](https://discuss.elastic.co/t/elasticsearch-takes-long-time-to-update-simple-index/210104)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 5:03am UTC](https://discuss.elastic.co/t/elasticsearch-takes-long-time-to-update-simple-index/210104 "2019-12-02T05:03:49Z")
I have installed ES and Logstash 7.2.0 in 2 servers and I tried pushing a simple text file from Logstash to ES. The Logstash seems to send the data quickly, however, the ES takes about 5 mins (everytime) to update this d… --- ## [\[QUESTION\] What is the default sorting between ES 1.7.x and 7.x?](https://discuss.elastic.co/t/question-what-is-the-default-sorting-between-es-1-7-x-and-7-x/210100)
**Author:** [@joneslee85](https://discuss.elastic.co/u/joneslee85)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 4:14am UTC](https://discuss.elastic.co/t/question-what-is-the-default-sorting-between-es-1-7-x-and-7-x/210100 "2019-12-02T04:14:37Z")
I am working on migrating from ES 1.7 to 7.4. There is one issue I picked up that is related to sorting. In ES 1.7, if returned documents share same \_score, the results are seems to be ordered randomly. Whilst ES 7.4 wo… --- ## [Updating slow query log in production environment](https://discuss.elastic.co/t/updating-slow-query-log-in-production-environment/210099)
**Author:** [@Manikant](https://discuss.elastic.co/u/Manikant)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 3:16am UTC](https://discuss.elastic.co/t/updating-slow-query-log-in-production-environment/210099 "2019-12-02T03:16:29Z")
Hi All, we are using Elastic Search 5.6 and want to know what elastic queries is running for search API in production environment. we run below command for slow query logs. curl -XPUT http://localhost:9200/\_cluster/se… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1077) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1079)