# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1961 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1962 --- ## [How to fix client ip for cross-cluster search (fw in between)](https://discuss.elastic.co/t/how-to-fix-client-ip-for-cross-cluster-search-fw-in-between/97567)
**Author:** [@Alberto\_Yoldi](https://discuss.elastic.co/u/Alberto_Yoldi)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 2:16pm UTC](https://discuss.elastic.co/t/how-to-fix-client-ip-for-cross-cluster-search-fw-in-between/97567 "2017-09-01T14:16:43Z")
The situation is we have a ES cluster (A) on a security zone A and another cluster (B) on a different security zone B. Both zones are separated using a firewall. We would like to use the cross-cluster search from the ES … --- ## [Find missing documents in a time series](https://discuss.elastic.co/t/find-missing-documents-in-a-time-series/99135)
**Author:** [@redX](https://discuss.elastic.co/u/redX)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 1:45pm UTC](https://discuss.elastic.co/t/find-missing-documents-in-a-time-series/99135 "2017-09-01T13:45:34Z")
I run a daily import of hotels with their offer count. The goal is to see when the offer count of a hotel “provider” changes significantly. If this happens, I need to know, what hotels have gone missing. My mapping: "… --- ## [Calculation on ES query?](https://discuss.elastic.co/t/calculation-on-es-query/98813)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 1:36pm UTC](https://discuss.elastic.co/t/calculation-on-es-query/98813 "2017-09-01T13:36:32Z")
Hi, until now I was mostly just using kibana and i borrowed some queries from there, to fire some of them via curl. So I am quite a Newbie about quering ES directly. So I need your help. I have an index which has docu… --- ## [Kibana to python dsl](https://discuss.elastic.co/t/kibana-to-python-dsl/98420)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 1:33pm UTC](https://discuss.elastic.co/t/kibana-to-python-dsl/98420 "2017-09-01T13:33:51Z")
I have this agg in kibana: “aggs”: { “2”: { “terms”: { “field”: “event\_data.TargetUserName”, “size”: 50, “order”: { "\_count": “desc” } } } } I can get this far in python dsl s.aggs.bucket(‘per\_user’, ‘terms’… --- ## [Only last 10 minutes relevant](https://discuss.elastic.co/t/only-last-10-minutes-relevant/98750)
**Author:** [@irom77](https://discuss.elastic.co/u/irom77)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 1:07pm UTC](https://discuss.elastic.co/t/only-last-10-minutes-relevant/98750 "2017-09-01T13:07:28Z")
How to index documents in Elastic and Logstash 5.5 in case I am interested in analyzing documents created only in last 10 minutes , should I reindex and delete index every 10 minutes ? What’s the best approach. --- ## [Error creating machine learning job "autodetect"](https://discuss.elastic.co/t/error-creating-machine-learning-job-autodetect/98703)
**Author:** [@Richard\_Neely](https://discuss.elastic.co/u/Richard_Neely)\ **Replies:** 13\ **Last updated:** [September 1, 2017, 12:48pm UTC](https://discuss.elastic.co/t/error-creating-machine-learning-job-autodetect/98703 "2017-09-01T12:48:25Z")
I get an error creating the most basic machine learning job. In kibana it says “could not open job an internal server error occured.” In the elastic logs this is what it says. I can launch autodetect plugin from the cli … --- ## [Prefix vs Wildcard vs Regexp in terms of performance and usability](https://discuss.elastic.co/t/prefix-vs-wildcard-vs-regexp-in-terms-of-performance-and-usability/98460)
**Author:** [@BentCoder](https://discuss.elastic.co/u/BentCoder)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 12:38pm UTC](https://discuss.elastic.co/t/prefix-vs-wildcard-vs-regexp-in-terms-of-performance-and-usability/98460 "2017-09-01T12:38:54Z")
Hi, I have queries below which work like SELECT \* FROM table WHERE raw.name LIKE 'Product1\*'. I just want to find out which one would be the best in terms of performance and usability? Thanks {"query":{"prefix":{"name… --- ## [Why is there an Index lag in elasticsearch?](https://discuss.elastic.co/t/why-is-there-an-index-lag-in-elasticsearch/98278)
**Author:** [@rajesh](https://discuss.elastic.co/u/rajesh)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 12:37pm UTC](https://discuss.elastic.co/t/why-is-there-an-index-lag-in-elasticsearch/98278 "2017-09-01T12:37:00Z")
Hi, Elasticsearch is indexing the documents with a lag(of 10 minutes). If the actual event is at 11:00am, Elasticsearch shows the index time as 11:10am. That means I am not seeing the logs real time. Can someone please … --- ## [Make aggregation response shorter](https://discuss.elastic.co/t/make-aggregation-response-shorter/99009)
**Author:** [@Wullschi](https://discuss.elastic.co/u/Wullschi)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 12:27pm UTC](https://discuss.elastic.co/t/make-aggregation-response-shorter/99009 "2017-09-01T12:27:49Z")
Hi! Is it possible to make an aggregation response shorter by leaving away fields like “doc\_count\_error\_upper\_bound”, “sum\_other\_doc\_count” or “doc\_count”? I could not find any information on how to disable those fiel… --- ## [Socket hangs up while running Timelion](https://discuss.elastic.co/t/socket-hangs-up-while-running-timelion/97396)
**Author:** [@sreejithps303](https://discuss.elastic.co/u/sreejithps303)\ **Replies:** 7\ **Last updated:** [September 1, 2017, 9:56am UTC](https://discuss.elastic.co/t/socket-hangs-up-while-running-timelion/97396 "2017-09-01T09:56:40Z")
Hi , I am getting socket hangs up error in kibana while running timelion visualisations.Elasticsearch load also increasing drastically. I have tried the below things. set thread\_pool.search.size: 20 and thread\_pool.se… --- ## [Auto summarization of elasticsearch text content](https://discuss.elastic.co/t/auto-summarization-of-elasticsearch-text-content/98791)
**Author:** [@Narmatha](https://discuss.elastic.co/u/Narmatha)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 9:16am UTC](https://discuss.elastic.co/t/auto-summarization-of-elasticsearch-text-content/98791 "2017-09-01T09:16:27Z")
Need help in summarizing the media content in elasticsearch and store in different field during data ingestion. Is there an option in elasticsearch directly to perform summarizing larger contents into readable text and … --- ## [Machine Learning & Elastic Cloud](https://discuss.elastic.co/t/machine-learning-elastic-cloud/97916)
**Author:** [@Druidmk1](https://discuss.elastic.co/u/Druidmk1)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 9:15am UTC](https://discuss.elastic.co/t/machine-learning-elastic-cloud/97916 "2017-09-01T09:15:59Z")
Hello, Is subject available on Cloud? I see that existing version now 5.5.2 but can't find machine learning button in Kibana. It will be great to have it. Max --- ## [Making text field aggregatable in kibana using elasticsearch](https://discuss.elastic.co/t/making-text-field-aggregatable-in-kibana-using-elasticsearch/98257)
**Author:** [@Singa\_Forever](https://discuss.elastic.co/u/Singa_Forever)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 9:09am UTC](https://discuss.elastic.co/t/making-text-field-aggregatable-in-kibana-using-elasticsearch/98257 "2017-09-01T09:09:13Z")
Hello, This is my first time using elk 5.5 for gathering tweets from twitter, everything is ok and works fine. In order to handle my application, I need to make the filed text (tweets ) as an aggregatable field, i tri… --- ## [A query to create flat table from different data](https://discuss.elastic.co/t/a-query-to-create-flat-table-from-different-data/98824)
**Author:** [@ronlik26](https://discuss.elastic.co/u/ronlik26)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 8:45am UTC](https://discuss.elastic.co/t/a-query-to-create-flat-table-from-different-data/98824 "2017-09-01T08:45:54Z")
Hi, I’m trying to come up with a query that will help me take my data and aggregate it into one table. I have many records such as the following: { "\_index": "doc", "\_type": "info", "\_id": "4", "\_routing": "123", "\_pa… --- ## [Increase nproc limit in ES Docker image](https://discuss.elastic.co/t/increase-nproc-limit-in-es-docker-image/99101)
**Author:** [@nickytd](https://discuss.elastic.co/u/nickytd)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 8:42am UTC](https://discuss.elastic.co/t/increase-nproc-limit-in-es-docker-image/99101 "2017-09-01T08:42:32Z")
Hi in some K8S environments container users with ID 1000 are quite common. Current limit set in /etc/security/limits.d/20-nproc.conf is 4096 and that is easily hit on node running multiple pods. As a result es container… --- ## [Duplicate Entries of Log data](https://discuss.elastic.co/t/duplicate-entries-of-log-data/98894)
**Author:** [@nktvys](https://discuss.elastic.co/u/nktvys)\ **Replies:** 5\ **Last updated:** [September 1, 2017, 8:38am UTC](https://discuss.elastic.co/t/duplicate-entries-of-log-data/98894 "2017-09-01T08:38:36Z")
Hi, I have multiple copies for a single entry of a log file in Elasticsearch. For eg. In my original log file, for a timestamp 12:55:03:234 there is a log entry X. In Elasticsearch for that timestamp, I have duplicates … --- ## [Suitability of kafka in current ELK pipeline](https://discuss.elastic.co/t/suitability-of-kafka-in-current-elk-pipeline/99085)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 3\ **Last updated:** [September 1, 2017, 8:35am UTC](https://discuss.elastic.co/t/suitability-of-kafka-in-current-elk-pipeline/99085 "2017-09-01T08:35:29Z")
Hi, I am using ELK stack for log monitoring. Its in production now. The pipeline I use is as follow. FB–\>Logstash–\>Redis–\>Logstash–\>Nginx–\>ES I am now reading about kafka. Just for quick direction, please suggest ho… --- ## [Query to calculate successrate](https://discuss.elastic.co/t/query-to-calculate-successrate/98886)
**Author:** [@mathias](https://discuss.elastic.co/u/mathias)\ **Replies:** 9\ **Last updated:** [September 1, 2017, 8:09am UTC](https://discuss.elastic.co/t/query-to-calculate-successrate/98886 "2017-09-01T08:09:09Z")
Hi, I am struggling with elasticsearch aggregation queries. I would like to calculate successrate per hour between two different events. I have one event with a field “name” with the value “attempt” each time an attem… --- ## [IP address ranges for elastic nodes](https://discuss.elastic.co/t/ip-address-ranges-for-elastic-nodes/98991)
**Author:** [@wmjb](https://discuss.elastic.co/u/wmjb)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 8:00am UTC](https://discuss.elastic.co/t/ip-address-ranges-for-elastic-nodes/98991 "2017-09-01T08:00:24Z")
Hi, My hosting service requesting me to provide IP address ranges for elastic search nodes in order for them to white-list the out going ports. May I know where I can find those? Thank you --- ## [Metricbeat Index turns red every hours](https://discuss.elastic.co/t/metricbeat-index-turns-red-every-hours/99004)
**Author:** [@marcelwolf](https://discuss.elastic.co/u/marcelwolf)\ **Replies:** 5\ **Last updated:** [September 1, 2017, 7:40am UTC](https://discuss.elastic.co/t/metricbeat-index-turns-red-every-hours/99004 "2017-09-01T07:40:03Z")
I run a 2 Node elasticsearch 5.5.1 cluster that holds a metricbeat index. Every about three hour the index from metricbeat turns red. The cluster status: { "cluster\_name": "elasticsearch", "status": "red", "timed… --- ## [Long GC even after mlockall set to true](https://discuss.elastic.co/t/long-gc-even-after-mlockall-set-to-true/99095)
**Author:** [@SKumarMN](https://discuss.elastic.co/u/SKumarMN)\ **Replies:** 1\ **Last updated:** [September 1, 2017, 7:28am UTC](https://discuss.elastic.co/t/long-gc-even-after-mlockall-set-to-true/99095 "2017-09-01T07:28:27Z")
Hi, I am having a 3 node ES 2.3.2 cluster running on Linux(2.6.32-400.37.1.el6uek.x86\_64). For every few hours nodes each nodes goes into Long GC’s taking time from 2 minutes to 5 minutes and hence node gets disconnecte… --- ## [Version conflict when using update\_by\_query](https://discuss.elastic.co/t/version-conflict-when-using-update-by-query/98706)
**Author:** [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 7:07am UTC](https://discuss.elastic.co/t/version-conflict-when-using-update-by-query/98706 "2017-09-01T07:07:47Z")
Hi all, I'm trying to use update\_by\_query to update a field in each document but I keep failing due to a version conflict. I don't understand why I am getting version conflicts as I know nothing else is running against … --- ## [Logs appear out of order when burst logging is done](https://discuss.elastic.co/t/logs-appear-out-of-order-when-burst-logging-is-done/98843)
**Author:** [@sbsatter](https://discuss.elastic.co/u/sbsatter)\ **Replies:** 0\ **Last updated:** [August 30, 2017, 11:08am UTC](https://discuss.elastic.co/t/logs-appear-out-of-order-when-burst-logging-is-done/98843 "2017-08-30T11:08:33Z")
When the logs appear to have the same timestamps, some of it appears out of order. Reading through some of the topics, I couldn’t find a suitable solution. But I believe there must be something done about it since this i… --- ## [Delete documents with version 0 with query](https://discuss.elastic.co/t/delete-documents-with-version-0-with-query/99089)
**Author:** [@elk\_user1](https://discuss.elastic.co/u/elk_user1)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 6:12am UTC](https://discuss.elastic.co/t/delete-documents-with-version-0-with-query/99089 "2017-09-01T06:12:20Z")
Hello There, I am trying to use delete\_by\_query to delete some documents for a particular index. But it fails with illegal version value and I read that delete\_by\_query cannot be used to delete documents with index ver… --- ## [Secured setting for azure repository in ES 5.4](https://discuss.elastic.co/t/secured-setting-for-azure-repository-in-es-5-4/97341)
**Author:** [@elk\_user1](https://discuss.elastic.co/u/elk_user1)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 6:08am UTC](https://discuss.elastic.co/t/secured-setting-for-azure-repository-in-es-5-4/97341 "2017-09-01T06:08:25Z")
Hi, We are using Elasticsearch 5.4 in production and planning to use snapshot API and store it in Azure. I see the secured settings only in master branch. So isn’t it available in 5.4 version? Any idea how to take care … --- ## [Read Latency for elastic search](https://discuss.elastic.co/t/read-latency-for-elastic-search/98728)
**Author:** [@Neeraj\_Gupta](https://discuss.elastic.co/u/Neeraj_Gupta)\ **Replies:** 4\ **Last updated:** [September 1, 2017, 5:46am UTC](https://discuss.elastic.co/t/read-latency-for-elastic-search/98728 "2017-09-01T05:46:37Z")
We have a use case where we are inserting data in Elastic search cluster at 19-20K QPS. We are not getting any errors while inserting data using bulk processor. However there are times where we observe read latency of a… --- ## [How to bump elasticsearch 'processors' setting in order to increase thread\_pool.bulk.size?](https://discuss.elastic.co/t/how-to-bump-elasticsearch-processors-setting-in-order-to-increase-thread-pool-bulk-size/98781)
**Author:** [@Chris\_Bedford](https://discuss.elastic.co/u/Chris_Bedford)\ **Replies:** 11\ **Last updated:** [September 1, 2017, 5:25am UTC](https://discuss.elastic.co/t/how-to-bump-elasticsearch-processors-setting-in-order-to-increase-thread-pool-bulk-size/98781 "2017-09-01T05:25:35Z")
We've done pretty extensive performance analysis and found that our write heavy-Elasticsearch 5.5.1-based application is under-utilizing CPU resources (load average is about same as number of cores, and we are 25% idle… --- ## [Is there a query similar to joins in Elasticsearch?](https://discuss.elastic.co/t/is-there-a-query-similar-to-joins-in-elasticsearch/99084)
**Author:** [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 5:09am UTC](https://discuss.elastic.co/t/is-there-a-query-similar-to-joins-in-elasticsearch/99084 "2017-09-01T05:09:58Z")
Hello. I know its not efficient to use “JOIN” in Elasticsearch, but I need to use it. I have to extract values by find same field of index A and index B. there is an example below. A/type1/1 { “serial”:“abc”, “mem… --- ## [How to extract quoted values in ingest mode kv processor](https://discuss.elastic.co/t/how-to-extract-quoted-values-in-ingest-mode-kv-processor/99082)
**Author:** [@ivofrolov](https://discuss.elastic.co/u/ivofrolov)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 4:51am UTC](https://discuss.elastic.co/t/how-to-extract-quoted-values-in-ingest-mode-kv-processor/99082 "2017-09-01T04:51:46Z")
Hello. I have messages like this key1=value1,key2='foo value3=foo,bar,baz' So there are quoted values that contains patterns similar to what I have to write in kv processor settings. When I use kv processor it splits al… --- ## [How can I sort String with has\_parent in ElasticSearch](https://discuss.elastic.co/t/how-can-i-sort-string-with-has-parent-in-elasticsearch/99080)
**Author:** [@nutanek](https://discuss.elastic.co/u/nutanek)\ **Replies:** 0\ **Last updated:** [September 1, 2017, 4:43am UTC](https://discuss.elastic.co/t/how-can-i-sort-string-with-has-parent-in-elasticsearch/99080 "2017-09-01T04:43:30Z")
Hello, I created parent-child in ElasticSearch. I use has\_parent for query data, but I want to sort this data follow some parent field which is a String. I tried to find solutions, I found how to sort in ElasticSearch… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1960) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1962)