# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1191 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1192 --- ## [Adding a field with a script to all documents for de-duplicating records](https://discuss.elastic.co/t/adding-a-field-with-a-script-to-all-documents-for-de-duplicating-records/194270)
**Author:** [@Jeremy\_Foran](https://discuss.elastic.co/u/Jeremy_Foran)\ **Replies:** 2\ **Last updated:** [August 14, 2019, 3:56pm UTC](https://discuss.elastic.co/t/adding-a-field-with-a-script-to-all-documents-for-de-duplicating-records/194270 "2019-08-14T15:56:16Z")
I wrote a query to find duplicate documents using aggregated functions: GET log/\_search { "profile": false, "size": 0, "query": { "range": { "@timestamp": { "gte": "now-2d/d", "lt": "n… --- ## [Connecting to remote ES index from behind a proxy](https://discuss.elastic.co/t/connecting-to-remote-es-index-from-behind-a-proxy/195233)
**Author:** [@pmadley](https://discuss.elastic.co/u/pmadley)\ **Replies:** 0\ **Last updated:** [August 14, 2019, 3:49pm UTC](https://discuss.elastic.co/t/connecting-to-remote-es-index-from-behind-a-proxy/195233 "2019-08-14T15:49:23Z")
Issue: How to connect to a remote IP-restricted ES server from behind a company proxy Background: We need to connect to a 3rd party IP-restricted Elasticsearch server. Our local proxy IP is whitelisted at their end, i… --- ## [Investigating reindexing error (1.7 -\> 5.6)](https://discuss.elastic.co/t/investigating-reindexing-error-1-7-5-6/195232)
**Author:** [@maniard](https://discuss.elastic.co/u/maniard)\ **Replies:** 0\ **Last updated:** [August 14, 2019, 3:44pm UTC](https://discuss.elastic.co/t/investigating-reindexing-error-1-7-5-6/195232 "2019-08-14T15:44:33Z")
Hi, I am trying to reindex my 1.7.5 ES cluster into a 5.6 one using Reindex API. While most indices are migrated successfully, for one of them I am getting the following error: org.elasticsearch.ElasticsearchException:… --- ## [\_cat/indices data](https://discuss.elastic.co/t/cat-indices-data/195070)
**Author:** [@rorixrebel](https://discuss.elastic.co/u/rorixrebel)\ **Replies:** 6\ **Last updated:** [August 14, 2019, 2:24pm UTC](https://discuss.elastic.co/t/cat-indices-data/195070 "2019-08-14T14:24:56Z")
so i'd like to start visualize some of the data from the \_cat apis like indices, volume and replicas, etc. tried to see if i could scrape the info from .monitoring-es but no dice, is the data from the api somewhere we c… --- ## [Multiple \`sort\` using with only one \`search\_after\`](https://discuss.elastic.co/t/multiple-sort-using-with-only-one-search-after/195213)
**Author:** [@coExp](https://discuss.elastic.co/u/coExp)\ **Replies:** 0\ **Last updated:** [August 14, 2019, 1:24pm UTC](https://discuss.elastic.co/t/multiple-sort-using-with-only-one-search-after/195213 "2019-08-14T13:24:16Z")
Hi everyone, I sort my results on field name witch is not unique, and the field uuid (unique id): "sort": \[ { "name": {"order": "desc"} }, { "uuid": {"order": "asc"} } \] But I would like to search after the field uuid … --- ## [Time series of various processing numbers](https://discuss.elastic.co/t/time-series-of-various-processing-numbers/195069)
**Author:** [@JGT](https://discuss.elastic.co/u/JGT)\ **Replies:** 2\ **Last updated:** [August 14, 2019, 12:40pm UTC](https://discuss.elastic.co/t/time-series-of-various-processing-numbers/195069 "2019-08-14T12:40:46Z")
I have several logs that have periodic messages with performance data. I want to graph the metrics over time. Things like number of users logged in, virtual memory used, pages in and out, etc. Each record/message has … --- ## [Elasticsearch cluster in two different hosts with one node as Docker container and another as Service](https://discuss.elastic.co/t/elasticsearch-cluster-in-two-different-hosts-with-one-node-as-docker-container-and-another-as-service/195197)
**Author:** [@Namrata\_Khobragade](https://discuss.elastic.co/u/Namrata_Khobragade)\ **Replies:** 0\ **Last updated:** [August 14, 2019, 12:13pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-in-two-different-hosts-with-one-node-as-docker-container-and-another-as-service/195197 "2019-08-14T12:13:34Z")
I have an existing Elasticsearch server in Ubuntu, for the purpose of better performance I want to add one more node in the Cluster. Can I do it with docker by not changing the existing system? I have tried it with dock… --- ## [ES 6.4 s3 feature with endpoint as hostname](https://discuss.elastic.co/t/es-6-4-s3-feature-with-endpoint-as-hostname/194372)
**Author:** [@sai\_bharanisrikanth](https://discuss.elastic.co/u/sai_bharanisrikanth)\ **Replies:** 6\ **Last updated:** [August 14, 2019, 11:58am UTC](https://discuss.elastic.co/t/es-6-4-s3-feature-with-endpoint-as-hostname/194372 "2019-08-14T11:58:20Z")
Hello Team, I am using ES 6.4 version along with S3 plugin v 6.4.3 installed in the cluster, and have configured end point to be minio service host name which is internal to the cluster. Configuration: s3.client.defa… --- ## [Unable to push nested json from filebeat to elasticsearch](https://discuss.elastic.co/t/unable-to-push-nested-json-from-filebeat-to-elasticsearch/194920)
**Author:** [@Kumar\_Pratyush](https://discuss.elastic.co/u/Kumar_Pratyush)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 10:47am UTC](https://discuss.elastic.co/t/unable-to-push-nested-json-from-filebeat-to-elasticsearch/194920 "2019-08-14T10:47:06Z")
So my requirement is fairly simple. I have logs in the form of json where each line corresponds to one document. The json is in nested format. When I use a simple json, i.e. without any nestings, it works fine and the sa… --- ## [Highlight stopwords in hits](https://discuss.elastic.co/t/highlight-stopwords-in-hits/194636)
**Author:** [@fedoroko](https://discuss.elastic.co/u/fedoroko)\ **Replies:** 3\ **Last updated:** [August 14, 2019, 10:48am UTC](https://discuss.elastic.co/t/highlight-stopwords-in-hits/194636 "2019-08-14T10:48:49Z")
Hey, what is the best way to skip stopwords in query but highlight them in hits? Current settings: 'settings': { 'analysis': { 'analyzer':{ my\_analyzer:{ 'tokenizer':'standard', … --- ## [Query returns non matching records](https://discuss.elastic.co/t/query-returns-non-matching-records/195184)
**Author:** [@ManuMP](https://discuss.elastic.co/u/ManuMP)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 10:27am UTC](https://discuss.elastic.co/t/query-returns-non-matching-records/195184 "2019-08-14T10:27:17Z")
Hi , I am using the following query to identify log entries having the message "DeviceStatus changed to Idle" within a period. But the query also returns the following messages "Module State is changed to idle Module… --- ## [Elastic Search can't setup password for Kibana](https://discuss.elastic.co/t/elastic-search-cant-setup-password-for-kibana/195051)
**Author:** [@Alexis\_Prat](https://discuss.elastic.co/u/Alexis_Prat)\ **Replies:** 13\ **Last updated:** [August 14, 2019, 10:16am UTC](https://discuss.elastic.co/t/elastic-search-cant-setup-password-for-kibana/195051 "2019-08-14T10:16:47Z")
Hi ! I'm a new user of the ELK suite. I would like to collect my log files onseveral server and put them on a central server. To do so, I use rsyslog. I succeed having them on my server but know I would like to impr… --- ## [How to get selected log from filebeat index by python](https://discuss.elastic.co/t/how-to-get-selected-log-from-filebeat-index-by-python/194466)
**Author:** [@Fosiul\_Alam](https://discuss.elastic.co/u/Fosiul_Alam)\ **Replies:** 9\ **Last updated:** [August 14, 2019, 10:06am UTC](https://discuss.elastic.co/t/how-to-get-selected-log-from-filebeat-index-by-python/194466 "2019-08-14T10:06:23Z")
i am using python elasticsearch module to search filebeat log by using using keyword. same as i do in "Kibana Discover" but my search does not return anything. # make sure ES is up and running import requests res = requ… --- ## [Index blocks write operation when it moved to warm phase by ILM](https://discuss.elastic.co/t/index-blocks-write-operation-when-it-moved-to-warm-phase-by-ilm/194552)
**Author:** [@worapojc](https://discuss.elastic.co/u/worapojc)\ **Replies:** 6\ **Last updated:** [August 14, 2019, 9:45am UTC](https://discuss.elastic.co/t/index-blocks-write-operation-when-it-moved-to-warm-phase-by-ilm/194552 "2019-08-14T09:45:32Z")
Hi Elasticsearch team, Index blocks write operation when it moved to the warm phase by ILM. { "indices" : { "nxlog-2019.08.06" : { "index" : "nxlog-2019.08.06", "managed" : true, "policy" : "com… --- ## [Is it possible to set up snapshot repo without restarting anything?](https://discuss.elastic.co/t/is-it-possible-to-set-up-snapshot-repo-without-restarting-anything/195140)
**Author:** [@iwinux](https://discuss.elastic.co/u/iwinux)\ **Replies:** 2\ **Last updated:** [August 14, 2019, 8:55am UTC](https://discuss.elastic.co/t/is-it-possible-to-set-up-snapshot-repo-without-restarting-anything/195140 "2019-08-14T08:55:09Z")
Hi, I'm trying to export data to our internal Ceph KV store (S3 protocol) from an old cluster (6.3.0) with 21 nodes. It's really frustrating that all these steps require restarting the cluster over and over. What I've … --- ## [Index exist using wildcard](https://discuss.elastic.co/t/index-exist-using-wildcard/195067)
**Author:** [@mamadly](https://discuss.elastic.co/u/mamadly)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 8:26am UTC](https://discuss.elastic.co/t/index-exist-using-wildcard/195067 "2019-08-14T08:26:21Z")
I am getting weird behavior when I use a wildcard, when checking if index exist or not using \ pattern. Regardless whether an index with this prefix exist or not a match is … --- ## [Compound sub-files must have a valid codec header and footer: file is too small (0 bytes)](https://discuss.elastic.co/t/compound-sub-files-must-have-a-valid-codec-header-and-footer-file-is-too-small-0-bytes/194936)
**Author:** [@liulinisgood](https://discuss.elastic.co/u/liulinisgood)\ **Replies:** 2\ **Last updated:** [August 14, 2019, 8:13am UTC](https://discuss.elastic.co/t/compound-sub-files-must-have-a-valid-codec-header-and-footer-file-is-too-small-0-bytes/194936 "2019-08-14T08:13:38Z")
I met a error for a long time, it occurs at 23:10 every day , I am very puzzled. I don't find any period run task . ES version: 5.5.3 os: centos7 3.10.0-327.el7.x86\_64 disk: ssd jdk8, g1 \[2019-08-11T23:08:20,178\]\[WA… --- ## [Retry reindexing starting from a certain point](https://discuss.elastic.co/t/retry-reindexing-starting-from-a-certain-point/195160)
**Author:** [@nico127](https://discuss.elastic.co/u/nico127)\ **Replies:** 0\ **Last updated:** [August 14, 2019, 8:09am UTC](https://discuss.elastic.co/t/retry-reindexing-starting-from-a-certain-point/195160 "2019-08-14T08:09:09Z")
Hello. It sometimes happen that reindexing fails, for one reason or another (mostly elastic thread pool's task queue full). I'm working on improving performance in some ways, to run into the issue less often; but I don't… --- ## [How to repeat watcher action for each payload data as per search input string?](https://discuss.elastic.co/t/how-to-repeat-watcher-action-for-each-payload-data-as-per-search-input-string/195076)
**Author:** [@saurabh06](https://discuss.elastic.co/u/saurabh06)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 7:52am UTC](https://discuss.elastic.co/t/how-to-repeat-watcher-action-for-each-payload-data-as-per-search-input-string/195076 "2019-08-14T07:52:34Z")
I want to send multiple email as per the list of payload coming for searched input string? how its possible with watcher API of ES?? --- ## [Is their any way to get unique result from the payload list data with search input string in watcher?](https://discuss.elastic.co/t/is-their-any-way-to-get-unique-result-from-the-payload-list-data-with-search-input-string-in-watcher/195077)
**Author:** [@saurabh06](https://discuss.elastic.co/u/saurabh06)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 7:51am UTC](https://discuss.elastic.co/t/is-their-any-way-to-get-unique-result-from-the-payload-list-data-with-search-input-string-in-watcher/195077 "2019-08-14T07:51:28Z")
I have passed more than 2 input string in watcher and revived the listed payload data from Elastic Search but now i want to send mail only on latest result for each input string data in payload instead of all payload hav… --- ## [How to use multiple statement inside the watcher transform?](https://discuss.elastic.co/t/how-to-use-multiple-statement-inside-the-watcher-transform/195078)
**Author:** [@saurabh06](https://discuss.elastic.co/u/saurabh06)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 7:50am UTC](https://discuss.elastic.co/t/how-to-use-multiple-statement-inside-the-watcher-transform/195078 "2019-08-14T07:50:26Z")
I have to write script inside the watcher transform i.e adding the loop and condition under watcher transform using groovy then how to add multiple statement under transform. while creating it throwing me error \[script\] … --- ## [Char\_filter and lowercase not working for search\_analyzer](https://discuss.elastic.co/t/char-filter-and-lowercase-not-working-for-search-analyzer/195074)
**Author:** [@KrisMeister](https://discuss.elastic.co/u/KrisMeister)\ **Replies:** 2\ **Last updated:** [August 14, 2019, 7:49am UTC](https://discuss.elastic.co/t/char-filter-and-lowercase-not-working-for-search-analyzer/195074 "2019-08-14T07:49:36Z")
I've been experimenting with many different iterations of the below but haven't been able to match all the cases I'd like to. My objective is to match the following cases: "7-eleven" or "7eleven" to the text "7-Eleven… --- ## [Long range not working](https://discuss.elastic.co/t/long-range-not-working/195096)
**Author:** [@Lucas\_Rezende](https://discuss.elastic.co/u/Lucas_Rezende)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 7:41am UTC](https://discuss.elastic.co/t/long-range-not-working/195096 "2019-08-14T07:41:58Z")
Hi there. I am trying range filter based on a long attribute but it seems not to be working. This is the que query I am using: GET my\_index/\_search { "query": { "bool": { "must\_not": \[ { "… --- ## [What is my error?](https://discuss.elastic.co/t/what-is-my-error/193502)
**Author:** [@JANGGILYONG](https://discuss.elastic.co/u/JANGGILYONG)\ **Replies:** 5\ **Last updated:** [August 14, 2019, 7:41am UTC](https://discuss.elastic.co/t/what-is-my-error/193502 "2019-08-14T07:41:48Z")
Sorry, It is my second notebook for testing the server. So I couldn't copy the scripts. --- ## [Getting all documents without duplicate values in Field](https://discuss.elastic.co/t/getting-all-documents-without-duplicate-values-in-field/195110)
**Author:** [@misiakj](https://discuss.elastic.co/u/misiakj)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 7:37am UTC](https://discuss.elastic.co/t/getting-all-documents-without-duplicate-values-in-field/195110 "2019-08-14T07:37:28Z")
I have an index that looks similar to the below (it really has a number of additional fields that aren't relevant for this post) case\_id: int group: string unique\_id: case\_id + group value: int for any one case\_id v… --- ## [How to set the time in rally so that it can perform (time \* events/s) events to benchmark the cpu usage and memory usage](https://discuss.elastic.co/t/how-to-set-the-time-in-rally-so-that-it-can-perform-time-events-s-events-to-benchmark-the-cpu-usage-and-memory-usage/195141)
**Author:** [@shantanu\_bhardwaj](https://discuss.elastic.co/u/shantanu_bhardwaj)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 7:21am UTC](https://discuss.elastic.co/t/how-to-set-the-time-in-rally-so-that-it-can-perform-time-events-s-events-to-benchmark-the-cpu-usage-and-memory-usage/195141 "2019-08-14T07:21:04Z")
I want to benchmark the cpu usage and memory usage for specific time with events/s that is if we have 500 events/s and set time for 900s so processing should be for 900 \* 500 events So at that moment what would be the … --- ## [Elastic Search SSL installation issue](https://discuss.elastic.co/t/elastic-search-ssl-installation-issue/195030)
**Author:** [@Prakash\_Choudhary](https://discuss.elastic.co/u/Prakash_Choudhary)\ **Replies:** 16\ **Last updated:** [August 14, 2019, 5:34am UTC](https://discuss.elastic.co/t/elastic-search-ssl-installation-issue/195030 "2019-08-14T05:34:30Z")
I tried to increase the security of elasticsearch but facing the issue below. i am unable to find the solution of it. request to please help me. below are the logs. Caused by: java.security.cert.CertificateParsingExcept… --- ## [Elastic exits with network.host set](https://discuss.elastic.co/t/elastic-exits-with-network-host-set/195118)
**Author:** [@Mezoloth](https://discuss.elastic.co/u/Mezoloth)\ **Replies:** 1\ **Last updated:** [August 14, 2019, 2:11am UTC](https://discuss.elastic.co/t/elastic-exits-with-network-host-set/195118 "2019-08-14T02:11:43Z")
I have installed Elastic on the latest ubuntu server. I cannot connect to Elastic from other hosts. I can connect to the server using SSH and I can see in netstat that 9200 is bound to 127.0.0.1. I can curl to Elastic … --- ## [Can't register snapshot repository to S3](https://discuss.elastic.co/t/cant-register-snapshot-repository-to-s3/195115)
**Author:** [@Bursade](https://discuss.elastic.co/u/Bursade)\ **Replies:** 0\ **Last updated:** [August 13, 2019, 10:14pm UTC](https://discuss.elastic.co/t/cant-register-snapshot-repository-to-s3/195115 "2019-08-13T22:14:41Z")
Hi all! I'm trying to register an S3 bucket as my snapshot repository. I'm using Elasticsearch Service, from AWS (which uses version 5.5 of Elasticsearch) When I run this script import boto3 import requests from req… --- ## [Does number of fields increases if we index some fields with offsets?](https://discuss.elastic.co/t/does-number-of-fields-increases-if-we-index-some-fields-with-offsets/193326)
**Author:** [@elk11](https://discuss.elastic.co/u/elk11)\ **Replies:** 1\ **Last updated:** [August 13, 2019, 8:47pm UTC](https://discuss.elastic.co/t/does-number-of-fields-increases-if-we-index-some-fields-with-offsets/193326 "2019-08-13T20:47:55Z")
Hi, To solve the issue of 'lenght of the field x exceeding the limit set for highlighting', we decided to index large fields with 'index\_options: offsets' as suggested by error message. But post this fix we are getting… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1190) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1192)