# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=580 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 581 --- ## [MIgrate ELK setup from cloud to on-prem](https://discuss.elastic.co/t/migrate-elk-setup-from-cloud-to-on-prem/283503)
**Author:** [@Bryce\_Fernandes](https://discuss.elastic.co/u/Bryce_Fernandes)\ **Replies:** 1\ **Last updated:** [September 7, 2021, 8:09am UTC](https://discuss.elastic.co/t/migrate-elk-setup-from-cloud-to-on-prem/283503 "2021-09-07T08:09:33Z")
Hi, I currently have elk setup on cloud and want to replicate it on onprem and remove cloud setup. I want to migrate development part like dashboards, index-patterns, users created etc. Will vm snapshot backup restore… --- ## [Search performance difference between from&size and search&after](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400)
**Author:** [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Replies:** 6\ **Last updated:** [September 7, 2021, 7:16am UTC](https://discuss.elastic.co/t/search-performance-difference-between-from-size-and-search-after/283400 "2021-09-07T07:16:43Z")
There are hundreds of millions of documents in my index. When I search, I find that search&after is much slower than from&size。 Use from&size，search is quick，took several ms return，but use search after，it took 20 seconds… --- ## [Misplaced codec footer keystore](https://discuss.elastic.co/t/misplaced-codec-footer-keystore/283493)
**Author:** [@itslogesh](https://discuss.elastic.co/u/itslogesh)\ **Replies:** 1\ **Last updated:** [September 7, 2021, 6:59am UTC](https://discuss.elastic.co/t/misplaced-codec-footer-keystore/283493 "2021-09-07T06:59:12Z")
org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file extended?): remaining=17, expected=16, fp=183 (resource=BufferedChecksumIndexInput(SimpleFSIndexInput(path="/etc/elasticsearch/elasticsearch.key… --- ## [Add large attachments from S3 to Elastic Search](https://discuss.elastic.co/t/add-large-attachments-from-s3-to-elastic-search/283452)
**Author:** [@Reghu\_Shanker](https://discuss.elastic.co/u/Reghu_Shanker)\ **Replies:** 5\ **Last updated:** [September 7, 2021, 3:37am UTC](https://discuss.elastic.co/t/add-large-attachments-from-s3-to-elastic-search/283452 "2021-09-07T03:37:42Z")
Hi, I have large pdf files which are located in AWS S3 which needs to be indexed and searchable. I want to upload these files to elastic search. The files need to be uploaded as single documents. How can I do this from… --- ## [Mock RestHighLevelClient for elasticsearch](https://discuss.elastic.co/t/mock-resthighlevelclient-for-elasticsearch/283465)
**Author:** [@romeoopk](https://discuss.elastic.co/u/romeoopk)\ **Replies:** 1\ **Last updated:** [September 7, 2021, 3:35am UTC](https://discuss.elastic.co/t/mock-resthighlevelclient-for-elasticsearch/283465 "2021-09-07T03:35:06Z")
HI I would like to test my rest api in quarkus which internally calls elasticsearch using RestHighLevelClient. My problem is that the RestHighLevelClient is not mockable and hence I cannot execute my unit test to test … --- ## [Boost score on token order over proximity](https://discuss.elastic.co/t/boost-score-on-token-order-over-proximity/283126)
**Author:** [@Dahea\_Moon](https://discuss.elastic.co/u/Dahea_Moon)\ **Replies:** 2\ **Last updated:** [September 7, 2021, 1:21am UTC](https://discuss.elastic.co/t/boost-score-on-token-order-over-proximity/283126 "2021-09-07T01:21:55Z")
Hello, elastic team. Is there way to adjust score between proximity and order? I want to search by "any word in order with some slop". I found out that most of DSLs that matters with proximity and in order option cann… --- ## [Force shards to single node when changing phases](https://discuss.elastic.co/t/force-shards-to-single-node-when-changing-phases/283476)
**Author:** [@rschwa2](https://discuss.elastic.co/u/rschwa2)\ **Replies:** 5\ **Last updated:** [September 6, 2021, 11:52pm UTC](https://discuss.elastic.co/t/force-shards-to-single-node-when-changing-phases/283476 "2021-09-06T23:52:45Z")
We have 2 warm data nodes in our cluster, and 6 shards configured for each index. When an index migrates from hot to warm, the shards are evenly distributed across both warm nodes. However we also have a shrink action … --- ## [ILM - Warm phase index is showing up in Hot phase](https://discuss.elastic.co/t/ilm-warm-phase-index-is-showing-up-in-hot-phase/282788)
**Author:** [@Music91](https://discuss.elastic.co/u/Music91)\ **Replies:** 11\ **Last updated:** [September 6, 2021, 11:11pm UTC](https://discuss.elastic.co/t/ilm-warm-phase-index-is-showing-up-in-hot-phase/282788 "2021-09-06T23:11:48Z")
Team, One of the index i have created with routing policy set to warm but under the Lifecycle status the index is showing up under Hot. Any idea why it is showing in Hot when it is expected to show only in Warm. Please a… --- ## [Is it possible?](https://discuss.elastic.co/t/is-it-possible/281197)
**Author:** [@ailengcon](https://discuss.elastic.co/u/ailengcon)\ **Replies:** 9\ **Last updated:** [September 6, 2021, 11:07pm UTC](https://discuss.elastic.co/t/is-it-possible/281197 "2021-09-06T23:07:33Z")
Can someone answer me this. I have a database with several configurations, and each configuration can be linked to the same serial\_number. To be able to analyze the data correctly, I need to filter så that each serial\_n… --- ## [To retrieve complete metricbeat data from the elasticsearch](https://discuss.elastic.co/t/to-retrieve-complete-metricbeat-data-from-the-elasticsearch/283112)
**Author:** [@nehasinha1807](https://discuss.elastic.co/u/nehasinha1807)\ **Replies:** 4\ **Last updated:** [September 6, 2021, 11:01pm UTC](https://discuss.elastic.co/t/to-retrieve-complete-metricbeat-data-from-the-elasticsearch/283112 "2021-09-06T23:01:46Z")
I want to get full metricbeat data from elasticsearch in json through API. I tried with - /metricbeat\*/\_search but it is not giving complete data. Please suggest. --- ## [Joining aggregation results to the original documents](https://discuss.elastic.co/t/joining-aggregation-results-to-the-original-documents/282547)
**Author:** [@lyubimtsev](https://discuss.elastic.co/u/lyubimtsev)\ **Replies:** 4\ **Last updated:** [September 6, 2021, 5:20pm UTC](https://discuss.elastic.co/t/joining-aggregation-results-to-the-original-documents/282547 "2021-09-06T17:20:21Z")
Hi I need help to create a query that creates metrics aggregations, merge created metrics to original documents and then filters documents using these metrics. Let's consider following dataset with managers' sales: … --- ## [Index rollover with date and time](https://discuss.elastic.co/t/index-rollover-with-date-and-time/283458)
**Author:** [@Felipe\_Fuller](https://discuss.elastic.co/u/Felipe_Fuller)\ **Replies:** 4\ **Last updated:** [September 6, 2021, 3:33pm UTC](https://discuss.elastic.co/t/index-rollover-with-date-and-time/283458 "2021-09-06T15:33:40Z")
Hi Community! I'm trying to create an index rollover with the date and hour when the new index is created because I set up the rollover every hour during the day. Having this in consideration, the format of the new inde… --- ## [Convert Elasticsearch Date/Time field from UTC To Local Time?](https://discuss.elastic.co/t/convert-elasticsearch-date-time-field-from-utc-to-local-time/283441)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 9\ **Last updated:** [September 6, 2021, 1:11pm UTC](https://discuss.elastic.co/t/convert-elasticsearch-date-time-field-from-utc-to-local-time/283441 "2021-09-06T13:11:52Z")
I am getting 12 Hours difference in Date. I want to set that date format to local time that is GMT+5. And I have set kibana default time period to Browser. Any one can help me how to set is as local time. --- ## [How to get all Field names with filter(or query) criteria on ElasticSearch?](https://discuss.elastic.co/t/how-to-get-all-field-names-with-filter-or-query-criteria-on-elasticsearch/283435)
**Author:** [@jinssssun](https://discuss.elastic.co/u/jinssssun)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 12:40pm UTC](https://discuss.elastic.co/t/how-to-get-all-field-names-with-filter-or-query-criteria-on-elasticsearch/283435 "2021-09-06T12:40:09Z")
PUT /test-product { "mappings": { "properties": { "profile": { "type": "nested" }, "product": { "type": "nested" } } } } i have a index that name's 'test-product' POS… --- ## [Sort data so that my exact match will be on top and then semi exact match with most words snd so on](https://discuss.elastic.co/t/sort-data-so-that-my-exact-match-will-be-on-top-and-then-semi-exact-match-with-most-words-snd-so-on/283338)
**Author:** [@Jaymin](https://discuss.elastic.co/u/Jaymin)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 12:30pm UTC](https://discuss.elastic.co/t/sort-data-so-that-my-exact-match-will-be-on-top-and-then-semi-exact-match-with-most-words-snd-so-on/283338 "2021-09-06T12:30:48Z")
I am using query\_string for searching my data. and wanted to sort my data such that exact match should be on top and less matched words would follow exact match. For Instance, { "size": 100, //"explain":true, "qu… --- ## [Database file changed error](https://discuss.elastic.co/t/database-file-changed-error/281903)
**Author:** [@acezeus](https://discuss.elastic.co/u/acezeus)\ **Replies:** 13\ **Last updated:** [September 6, 2021, 11:38am UTC](https://discuss.elastic.co/t/database-file-changed-error/281903 "2021-09-06T11:38:42Z")
hello, i installed the elastic search for the first time along with kibana. everything was working fine until i switched off my pc for the day and started it the next morning. now when i try to start elasticsearch by ty… --- ## [Installation seems to hang](https://discuss.elastic.co/t/installation-seems-to-hang/281116)
**Author:** [@D-Groenewegen](https://discuss.elastic.co/u/D-Groenewegen)\ **Replies:** 4\ **Last updated:** [September 6, 2021, 11:12am UTC](https://discuss.elastic.co/t/installation-seems-to-hang/281116 "2021-09-06T11:12:31Z")
Dear all, I hope you'll forgive me for my utter noobness. With the help of Install Elasticsearch from archive on Linux or MacOS | Elasticsearch Guide \[7.14\] | Elastic, I tried installing ES for Linux. When I got to step… --- ## [Distance feature relevance score behaviour with boost \> 1](https://discuss.elastic.co/t/distance-feature-relevance-score-behaviour-with-boost-1/283386)
**Author:** [@Courtan](https://discuss.elastic.co/u/Courtan)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 9:00pm UTC](https://discuss.elastic.co/t/distance-feature-relevance-score-behaviour-with-boost-1/283386 "2021-09-05T21:00:30Z")
Hello, I am confused about the calculation of relevance scores with distance\_feature. In the documentation it says that: The distance\_feature query calculates a document’s relevance score as follows: relevance score … --- ## [How to manage high level client connection to Elasticsearch](https://discuss.elastic.co/t/how-to-manage-high-level-client-connection-to-elasticsearch/283410)
**Author:** [@111505](https://discuss.elastic.co/u/111505)\ **Replies:** 3\ **Last updated:** [September 6, 2021, 9:31am UTC](https://discuss.elastic.co/t/how-to-manage-high-level-client-connection-to-elasticsearch/283410 "2021-09-06T09:31:17Z")
I wonder how to manage connection when using RestHighlevelClient in Java. The old connection is automatically disconnected. So I know that i need to connect and close when i need it. Is this right? So what is I to be ma… --- ## [Find all indices belonging to box type hot](https://discuss.elastic.co/t/find-all-indices-belonging-to-box-type-hot/283415)
**Author:** [@smlbiobot](https://discuss.elastic.co/u/smlbiobot)\ **Replies:** 1\ **Last updated:** [September 6, 2021, 9:00am UTC](https://discuss.elastic.co/t/find-all-indices-belonging-to-box-type-hot/283415 "2021-09-06T09:00:55Z")
Is there an easy way to find all indices belonging to box type hot? I know that I can run GET /\_nodes/box\_type:hot to get all the nodes, and then maybe run a second step to find which node an index belong to, but I’m tr… --- ## [Min aggregation issue](https://discuss.elastic.co/t/min-aggregation-issue/283171)
**Author:** [@str8y](https://discuss.elastic.co/u/str8y)\ **Replies:** 3\ **Last updated:** [September 6, 2021, 8:46am UTC](https://discuss.elastic.co/t/min-aggregation-issue/283171 "2021-09-06T08:46:24Z")
If a min aggregation is used inside a terms aggregation, the result doesn't contain any buckets for the terms aggregation, as long as no mapping exists for the min aggregation field (e.g. there is no value for that field… --- ## [Query and sort inside nested object](https://discuss.elastic.co/t/query-and-sort-inside-nested-object/283404)
**Author:** [@Are](https://discuss.elastic.co/u/Are)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 8:29am UTC](https://discuss.elastic.co/t/query-and-sort-inside-nested-object/283404 "2021-09-06T08:29:51Z")
Edit: see step-by-step and deeper explanation in post below. I am trying to sort my search results based on a field inside a nested object. Running ES 7.9. The mapping of the nested object looks like this: { "metaDate… --- ## [How can I set X-Opaque-Id while using restHighLevelClient?](https://discuss.elastic.co/t/how-can-i-set-x-opaque-id-while-using-resthighlevelclient/283390)
**Author:** [@yeziblo](https://discuss.elastic.co/u/yeziblo)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 7:58am UTC](https://discuss.elastic.co/t/how-can-i-set-x-opaque-id-while-using-resthighlevelclient/283390 "2021-09-06T07:58:06Z")
Hi everyone. Can I set X-Opaque-Id while using restHighLevelClient so that I can trace it in the slow log. And if so, how to set? Any advice is helpful. best regards. --- ## [Is there a way to just post a command using the low level or nest .net clients?](https://discuss.elastic.co/t/is-there-a-way-to-just-post-a-command-using-the-low-level-or-nest-net-clients/283392)
**Author:** [@astanton1978](https://discuss.elastic.co/u/astanton1978)\ **Replies:** 1\ **Last updated:** [September 6, 2021, 7:07am UTC](https://discuss.elastic.co/t/is-there-a-way-to-just-post-a-command-using-the-low-level-or-nest-net-clients/283392 "2021-09-06T07:07:03Z")
I dont see an appropriate category or tags for either of those .net clients, and the GitHub repo for them says to ask questions here, so sorry if this is the wrong category. I need to create an index mapping with some a… --- ## [Using Nest in AWS Lambda](https://discuss.elastic.co/t/using-nest-in-aws-lambda/283340)
**Author:** [@imranazad](https://discuss.elastic.co/u/imranazad)\ **Replies:** 1\ **Last updated:** [September 6, 2021, 6:59am UTC](https://discuss.elastic.co/t/using-nest-in-aws-lambda/283340 "2021-09-06T06:59:38Z")
Has anyone used Nest with AWS Lambda? How did you deal with cold starts? The size of dlls affects performance and Nests library is large, the version we're using is 4 megabytes. Is there a light weight version of the NE… --- ## [Using Scroll Api causes Out of Memory errors](https://discuss.elastic.co/t/using-scroll-api-causes-out-of-memory-errors/283388)
**Author:** [@lakshmiteja\_salapu](https://discuss.elastic.co/u/lakshmiteja_salapu)\ **Replies:** 8\ **Last updated:** [September 6, 2021, 6:54am UTC](https://discuss.elastic.co/t/using-scroll-api-causes-out-of-memory-errors/283388 "2021-09-06T06:54:06Z")
I am trying to use scroll for getting over 41 million documents (156 GB) but it fails with out of memory error after getting 410000 documents Using batch size of 5000 documents and 30s - scroll time (also tried changing… --- ## [Nested aggregation referencing outer aggregation](https://discuss.elastic.co/t/nested-aggregation-referencing-outer-aggregation/283164)
**Author:** [@Jakob](https://discuss.elastic.co/u/Jakob)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 6:29am UTC](https://discuss.elastic.co/t/nested-aggregation-referencing-outer-aggregation/283164 "2021-09-06T06:29:14Z")
I am learning about aggregations by looking at the "kibana\_sample\_data\_flights" index and I'm trying to find how many flights have the same weather on departure and arrival. Here is what I have so far: GET /kibana\_samp… --- ## [Cannot start transform job - Cannot set after key in the composite aggregation \[\_transform\]](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013)
**Author:** [@ciment](https://discuss.elastic.co/u/ciment)\ **Replies:** 4\ **Last updated:** [September 6, 2021, 6:02am UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013 "2021-09-06T06:02:40Z")
Hi, We would like to setup transform job to create statistics - how many devices are connected per hour. In our code we have added for every request, deviceId and clientId to labels data \[labels.deviceId, labels.clientI… --- ## [Why does es upgrade index data synchronously？](https://discuss.elastic.co/t/why-does-es-upgrade-index-data-synchronously/283226)
**Author:** [@jiankunking](https://discuss.elastic.co/u/jiankunking)\ **Replies:** 12\ **Last updated:** [September 6, 2021, 5:50am UTC](https://discuss.elastic.co/t/why-does-es-upgrade-index-data-synchronously/283226 "2021-09-06T05:50:32Z")
When I upgrade from 5.x to 7.x, I find that I cannot upgrade directly and need to reindex all index data. Why not update the index data automatically during the upgrade? --- ## [Index management](https://discuss.elastic.co/t/index-management/283393)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 5:31am UTC](https://discuss.elastic.co/t/index-management/283393 "2021-09-06T05:31:53Z")
what does indices mean? and why do i have so many indices? and what is the difference between xxxx-000004 and xxxx-000005? here's my indices for example: \* winlogbeat-7.10.2-2021.08.22-000005 \* auditbeat-7.11.2-2021.0… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=579) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=581)