# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1033 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1034 --- ## [How to use pagination per batch](https://discuss.elastic.co/t/how-to-use-pagination-per-batch/216940)
**Author:** [@Toshif](https://discuss.elastic.co/u/Toshif)\ **Replies:** 4\ **Last updated:** [January 29, 2020, 8:43am UTC](https://discuss.elastic.co/t/how-to-use-pagination-per-batch/216940 "2020-01-29T08:43:56Z")
Hello people, My query is retrieving 10000 records from elasticSearch in 1 hit. I used pagination (from & size) but its giving me only specified records. I want 10000 records but in a batch size of 50. (I am not able… --- ## [Using analyzer defined in index leads not to the same result when using \_analyze API](https://discuss.elastic.co/t/using-analyzer-defined-in-index-leads-not-to-the-same-result-when-using-analyze-api/216834)
**Author:** [@itarch](https://discuss.elastic.co/u/itarch)\ **Replies:** 2\ **Last updated:** [January 29, 2020, 7:19am UTC](https://discuss.elastic.co/t/using-analyzer-defined-in-index-leads-not-to-the-same-result-when-using-analyze-api/216834 "2020-01-29T07:19:30Z")
Hi, need some help please! I expect that A) and B) leads to the same result. However, in A), using the \_analayze API leads to what I expect. In B), defining "my\_analyzer" like i do in A) in "mytest01" index do not lea… --- ## [Hint: Readable SQL Queries to elasticsearch via curl](https://discuss.elastic.co/t/hint-readable-sql-queries-to-elasticsearch-via-curl/216611)
**Author:** [@Skeeve](https://discuss.elastic.co/u/Skeeve)\ **Replies:** 2\ **Last updated:** [January 29, 2020, 6:33am UTC](https://discuss.elastic.co/t/hint-readable-sql-queries-to-elasticsearch-via-curl/216611 "2020-01-29T06:33:43Z")
I wanted to do some tests using SQL queries to elasticsearch from commandline. I ended up with a really messy looking curl command: curl -XPOST http://ACCOUNT:APIKEY@elasticsearch:9200/\_sql?format=tsv -H 'Content-Type: … --- ## [Append nested object to exisiting nested field in elasticsearch](https://discuss.elastic.co/t/append-nested-object-to-exisiting-nested-field-in-elasticsearch/215696)
**Author:** [@Ram\_29](https://discuss.elastic.co/u/Ram_29)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 9:41am UTC](https://discuss.elastic.co/t/append-nested-object-to-exisiting-nested-field-in-elasticsearch/215696 "2020-01-28T09:41:34Z")
I'm having nested object like below "appdata": { "type":"nested", "include\_in\_parent":true, "properties": { "accessType": { "type": "text", "norms": false, "fields": { "keyword": { "type": "keyword", "ignore\_a… --- ## [Total Storage for Data Nodes](https://discuss.elastic.co/t/total-storage-for-data-nodes/216925)
**Author:** [@Jesbourne](https://discuss.elastic.co/u/Jesbourne)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 10:33pm UTC](https://discuss.elastic.co/t/total-storage-for-data-nodes/216925 "2020-01-28T22:33:09Z")
I was watching the webinars on sizing, and they give this math equation to calculate Total Storage needed - Total Storage (GB) = Total Data (GB) \* (1 + 0.15 default Disk watermark threshold + 0.05 Margin of error) And i… --- ## [Spark 2.4 to Elasticsearch : prevent data loss during dataproc nodes decommissioning?](https://discuss.elastic.co/t/spark-2-4-to-elasticsearch-prevent-data-loss-during-dataproc-nodes-decommissioning/216076)
**Author:** [@fredrouvier](https://discuss.elastic.co/u/fredrouvier)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 9:51pm UTC](https://discuss.elastic.co/t/spark-2-4-to-elasticsearch-prevent-data-loss-during-dataproc-nodes-decommissioning/216076 "2020-01-28T21:51:49Z")
My technical task is to synchronize data from GCS (Google Cloud Storage) to our Elasticsearch cluster. We use Apache Spark 2.4 with the Elastic Hadoop connector on a Google dataproc cluster (autoscaling enabled). During… --- ## [Spark Structured Streaming JSON Serialization](https://discuss.elastic.co/t/spark-structured-streaming-json-serialization/214661)
**Author:** [@execdd17](https://discuss.elastic.co/u/execdd17)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 9:48pm UTC](https://discuss.elastic.co/t/spark-structured-streaming-json-serialization/214661 "2020-01-28T21:48:59Z")
Hey Everyone, I hope this question hasn't been asked before, I looked around quite a bit before posting. If I missed it, sorry! I'm trying to use the org.elasticsearch.spark.sql structured streaming sink for ES, and am… --- ## [Compare three different fields from two different indices in a watcher](https://discuss.elastic.co/t/compare-three-different-fields-from-two-different-indices-in-a-watcher/216528)
**Author:** [@pierrejutard](https://discuss.elastic.co/u/pierrejutard)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 9:47pm UTC](https://discuss.elastic.co/t/compare-three-different-fields-from-two-different-indices-in-a-watcher/216528 "2020-01-28T21:47:40Z")
Hello, I have two indice on elastic search and i would like to compare it on a watcher. I have from one index the spot field : "hits" : \[ { "\_index" : "mktdata-2020.01.25", "\_type" : "\_doc", "\_id" : "Ue883W8BBSmq… --- ## [Watcher source script problem /Index\_out\_of\_bounds\_exception](https://discuss.elastic.co/t/watcher-source-script-problem-index-out-of-bounds-exception/216909)
**Author:** [@pierrejutard](https://discuss.elastic.co/u/pierrejutard)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 9:46pm UTC](https://discuss.elastic.co/t/watcher-source-script-problem-index-out-of-bounds-exception/216909 "2020-01-28T21:46:52Z")
Hello, I try to make a watcher where one field ("spot") from one indice ("mktdata-2020.01.28") will be compared from two fields ("lbound" and "ubound") from another indice ("price-request-2020.01.28") but i am stuck sin… --- ## [Watcher / Deprecation: \[types removal\] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/\_doc/{id}, /{index}/\_doc, or /{index}/\_create/{id})](https://discuss.elastic.co/t/watcher-deprecation-types-removal-specifying-types-in-document-index-requests-is-deprecated-use-the-typeless-endpoints-instead-index-doc-id-index-doc-or-index-create-id/216800)
**Author:** [@pierrejutard](https://discuss.elastic.co/u/pierrejutard)\ **Replies:** 8\ **Last updated:** [January 28, 2020, 9:45pm UTC](https://discuss.elastic.co/t/watcher-deprecation-types-removal-specifying-types-in-document-index-requests-is-deprecated-use-the-typeless-endpoints-instead-index-doc-id-index-doc-or-index-create-id/216800 "2020-01-28T21:45:17Z")
Hello, When trying to insert a watcher in ElasticSearch: PUT /watcher/watch/pricealert { "watch": { "trigger": { "schedule": { "interval": "1m" } }, "input": { "chain": { "inputs": \[ { "first": { "search"… --- ## [Flatten nested dictionaries in index](https://discuss.elastic.co/t/flatten-nested-dictionaries-in-index/216711)
**Author:** [@ellisvalentiner](https://discuss.elastic.co/u/ellisvalentiner)\ **Replies:** 3\ **Last updated:** [January 28, 2020, 7:54pm UTC](https://discuss.elastic.co/t/flatten-nested-dictionaries-in-index/216711 "2020-01-28T19:54:21Z")
I'm new to these forums so please let me know if I've posted this in the wrong place. I have an index that contains a key-value tags consisting of an array of values, which are dictionaries. The dictionaries always have… --- ## [Which file/script of ES Rally is responsible for pulling real time data from elastic.co site?](https://discuss.elastic.co/t/which-file-script-of-es-rally-is-responsible-for-pulling-real-time-data-from-elastic-co-site/216755)
**Author:** [@kumarvikash1](https://discuss.elastic.co/u/kumarvikash1)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 5:43pm UTC](https://discuss.elastic.co/t/which-file-script-of-es-rally-is-responsible-for-pulling-real-time-data-from-elastic-co-site/216755 "2020-01-28T17:43:57Z")
Hi All, Need one help !! I am exploring ES Rally scripts to understand data flow from elastic.co-\>ES Rally-\> ES. Does any body know, Which file/script of ES Rally is responsible for pulling real time data from elastic… --- ## [How can I sort results based by a custom field in a multi-field property?](https://discuss.elastic.co/t/how-can-i-sort-results-based-by-a-custom-field-in-a-multi-field-property/216889)
**Author:** [@jamesfaix](https://discuss.elastic.co/u/jamesfaix)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 5:17pm UTC](https://discuss.elastic.co/t/how-can-i-sort-results-based-by-a-custom-field-in-a-multi-field-property/216889 "2020-01-28T17:17:56Z")
I am working on upgrading a system at work from using ES1 to ES7. Part of the ES1 implementation included a custom plugin to add an analyzer for custom sorting. The custom sorting behavior we have is similar to "natural… --- ## [ELK stack as a private source](https://discuss.elastic.co/t/elk-stack-as-a-private-source/216862)
**Author:** [@Amna\_sayed\_ali](https://discuss.elastic.co/u/Amna_sayed_ali)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 5:13pm UTC](https://discuss.elastic.co/t/elk-stack-as-a-private-source/216862 "2020-01-28T17:13:40Z")
Is Elk still available for free or will go into a paid tier only? --- ## [Sampler aggregration overhead](https://discuss.elastic.co/t/sampler-aggregration-overhead/216324)
**Author:** [@Bruno\_Lage\_Aguiar](https://discuss.elastic.co/u/Bruno_Lage_Aguiar)\ **Replies:** 10\ **Last updated:** [January 28, 2020, 5:05pm UTC](https://discuss.elastic.co/t/sampler-aggregration-overhead/216324 "2020-01-28T17:05:35Z")
Hi, I have a search use case with a catalog over 20M products. If the search being made matches a lot of products the aggregations takes quite a lot of time, since they have to analyze millions of products. Being so we… --- ## [New to ES Rally not creating metrics](https://discuss.elastic.co/t/new-to-es-rally-not-creating-metrics/216848)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 4\ **Last updated:** [January 28, 2020, 4:15pm UTC](https://discuss.elastic.co/t/new-to-es-rally-not-creating-metrics/216848 "2020-01-28T16:15:04Z")
I set up my first Trackand Race but just hangs saying "Preparing for Race". INdex in reporting DB does create but no docs... Any Ideas? health status index uuid pri rep docs.count doc… --- ## [Update nested object using java client 6.5](https://discuss.elastic.co/t/update-nested-object-using-java-client-6-5/216874)
**Author:** [@Mohamed](https://discuss.elastic.co/u/Mohamed)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 4:09pm UTC](https://discuss.elastic.co/t/update-nested-object-using-java-client-6-5/216874 "2020-01-28T16:09:52Z")
Hello, I want to update a nested object using java highlevel client. My mapping looks like the following: "reservations":{ "type":"nested", "properties":{ "customerFul… --- ## [Getting error while installing graph-aided-search plugin](https://discuss.elastic.co/t/getting-error-while-installing-graph-aided-search-plugin/216863)
**Author:** [@Bianca\_Moldovan](https://discuss.elastic.co/u/Bianca_Moldovan)\ **Replies:** 2\ **Last updated:** [January 28, 2020, 4:06pm UTC](https://discuss.elastic.co/t/getting-error-while-installing-graph-aided-search-plugin/216863 "2020-01-28T16:06:21Z")
Please find below the exception that I'm getting when trying to install the plugin: D:\\elasticsearch-7.5.2\\bin\>elasticsearch-plugin install file:////D:/graph-aided-search-2.4.1.3.zip -\> Downloading file:////D:/graph-ai… --- ## [Skipping rollover option](https://discuss.elastic.co/t/skipping-rollover-option/209750)
**Author:** [@ark516](https://discuss.elastic.co/u/ark516)\ **Replies:** 4\ **Last updated:** [January 28, 2020, 3:34pm UTC](https://discuss.elastic.co/t/skipping-rollover-option/209750 "2020-01-28T15:34:12Z")
I have enabled skipping rollover option while using index lifyce cycle policy . But ILM still tries to perform rollover. https://www.elastic.co/guide/en/elasticsearch/reference/current/skipping-rollover.html below is t… --- ## [ES shards optimisation and improvement to cluster health](https://discuss.elastic.co/t/es-shards-optimisation-and-improvement-to-cluster-health/216102)
**Author:** [@Norman\_Khine](https://discuss.elastic.co/u/Norman_Khine)\ **Replies:** 9\ **Last updated:** [January 28, 2020, 3:33pm UTC](https://discuss.elastic.co/t/es-shards-optimisation-and-improvement-to-cluster-health/216102 "2020-01-28T15:33:11Z")
Hello, On my cluster,(version 7.1), I keep getting a Yellow state from time to time and an increase in x400 errors, currently, the cluster has 3 master nodes and 4 data nodes with 640Gb of disk space per data node. cur… --- ## [Snapshot Retention Task not deleting expired snapshots](https://discuss.elastic.co/t/snapshot-retention-task-not-deleting-expired-snapshots/215952)
**Author:** [@Micah\_Hunsberger](https://discuss.elastic.co/u/Micah_Hunsberger)\ **Replies:** 3\ **Last updated:** [January 28, 2020, 3:27pm UTC](https://discuss.elastic.co/t/snapshot-retention-task-not-deleting-expired-snapshots/215952 "2020-01-28T15:27:56Z")
For some reason, I cannot get the snapshot retention policy to delete old snapshots in one of my clusters. I successfully set up SLM on a test, standalone node ingesting a regular but small amount of data. The retention… --- ## [Elasticsearch search slowlog Timestamp](https://discuss.elastic.co/t/elasticsearch-search-slowlog-timestamp/216291)
**Author:** [@sah](https://discuss.elastic.co/u/sah)\ **Replies:** 1\ **Last updated:** [January 28, 2020, 3:24pm UTC](https://discuss.elastic.co/t/elasticsearch-search-slowlog-timestamp/216291 "2020-01-28T15:24:34Z")
Quick question (I couldnt figure it out in the docs) but search slow logs have a timestamp and a duration, for example: \[2020-01-23 12:04:38,000\]\[WARN \]\[index.search.slowlog.query\] \[p17us101esn01\] \[indexample\]\[3\] took\[1… --- ## [Sql query, histogram interval one month problem](https://discuss.elastic.co/t/sql-query-histogram-interval-one-month-problem/216442)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 5\ **Last updated:** [January 28, 2020, 2:58pm UTC](https://discuss.elastic.co/t/sql-query-histogram-interval-one-month-problem/216442 "2020-01-28T14:58:04Z")
I am trying to do histogram in sql query for month and it does not work daily or hourly works fine here is query GET \_sql?format=txt { "query": """ select histogram(achieved, INTERVAL 1 month) as m, count(\*) … --- ## [Heartbeat monitors stopping working!](https://discuss.elastic.co/t/heartbeat-monitors-stopping-working/216663)
**Author:** [@joao\_vitor](https://discuss.elastic.co/u/joao_vitor)\ **Replies:** 8\ **Last updated:** [January 28, 2020, 2:21pm UTC](https://discuss.elastic.co/t/heartbeat-monitors-stopping-working/216663 "2020-01-28T14:21:31Z")
Hi guys! I'm use heartbeat to monitoring my applications... I am configuring on three identical machines, on two it is working correctly, but on the third it stops, it works if I restart the windows service, but again i… --- ## [How to prepare a proper SLA query in Elasticsearch](https://discuss.elastic.co/t/how-to-prepare-a-proper-sla-query-in-elasticsearch/216857)
**Author:** [@An\_Eye](https://discuss.elastic.co/u/An_Eye)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 2:17pm UTC](https://discuss.elastic.co/t/how-to-prepare-a-proper-sla-query-in-elasticsearch/216857 "2020-01-28T14:17:14Z")
Hello! Currently I am using ELK + Grafana stack and I am supplying it with data from JMeter. Jmeter is responsible for testing web service and asserting the responses - which later on are being pushed to my ELKG stack. … --- ## [Creating Watcher and Trigger alert from outlook 365](https://discuss.elastic.co/t/creating-watcher-and-trigger-alert-from-outlook-365/216510)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 11\ **Last updated:** [January 28, 2020, 2:14pm UTC](https://discuss.elastic.co/t/creating-watcher-and-trigger-alert-from-outlook-365/216510 "2020-01-28T14:14:41Z")
Hello, I am trying to trigger an alert but I am getting an error \[2020-01-25T06:26:06,821\]\[ERROR\]\[o.e.x.w.a.e.ExecutableEmailAction\] \[CGAUH-PDLXS03\] failed to execute action \[inlined/email\_1\] org.elasticsearch.common.… --- ## [Elasticsearch is in logs retrieve only first time to restart filebeat](https://discuss.elastic.co/t/elasticsearch-is-in-logs-retrieve-only-first-time-to-restart-filebeat/216853)
**Author:** [@Ritesh\_Pradhan](https://discuss.elastic.co/u/Ritesh_Pradhan)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 2:09pm UTC](https://discuss.elastic.co/t/elasticsearch-is-in-logs-retrieve-only-first-time-to-restart-filebeat/216853 "2020-01-28T14:09:28Z")
Dear Team, We are use kibana dashboard and configure Elasticseach, filebeat, metricbeat and kibana. Current we are facing issue: I restart filebeat to our server, So retrieve one time to all logs and after that filebea… --- ## [Are there limits for the 14 day trial?](https://discuss.elastic.co/t/are-there-limits-for-the-14-day-trial/216730)
**Author:** [@fallenreaper](https://discuss.elastic.co/u/fallenreaper)\ **Replies:** 2\ **Last updated:** [January 28, 2020, 1:31pm UTC](https://discuss.elastic.co/t/are-there-limits-for-the-14-day-trial/216730 "2020-01-28T13:31:26Z")
I was wanting to try out the 14 day trial, but i was curious about a few things. It says free, but is it really, completely free? I was dabbling the opensource versions, but i wanted to try some of the most interesting… --- ## [Production-cluster monitoring shows monitoring-cluster monitoring](https://discuss.elastic.co/t/production-cluster-monitoring-shows-monitoring-cluster-monitoring/216839)
**Author:** [@alexolivan](https://discuss.elastic.co/u/alexolivan)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 1:07pm UTC](https://discuss.elastic.co/t/production-cluster-monitoring-shows-monitoring-cluster-monitoring/216839 "2020-01-28T13:07:05Z")
Hi forum. I've managed (or that seems) to setup the production-cluster and monitoring-cluster combo somehow up... these are the steps I did: Setup 2 mockup clusters, x-pack, authentication and TLS/SSL HTTPS is setup. … --- ## [How to sort by a date/time column on the Dashboard in Kibana?](https://discuss.elastic.co/t/how-to-sort-by-a-date-time-column-on-the-dashboard-in-kibana/216837)
**Author:** [@franzisk](https://discuss.elastic.co/u/franzisk)\ **Replies:** 0\ **Last updated:** [January 28, 2020, 1:01pm UTC](https://discuss.elastic.co/t/how-to-sort-by-a-date-time-column-on-the-dashboard-in-kibana/216837 "2020-01-28T13:01:24Z")
Hi guys, Sorry for asking again such a newbie question, but... I need to know how I show this date/time column in desc order in the Dashboard in Kibana. My data is being inserted via API call from a Spring Boot API us… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1032) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1034)