# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1915 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1916 --- ## [Store the data more than 1TB per day](https://discuss.elastic.co/t/store-the-data-more-than-1tb-per-day/104511)
**Author:** [@Sharon\_wsf](https://discuss.elastic.co/u/Sharon_wsf)\ **Replies:** 7\ **Last updated:** [October 20, 2017, 6:16am UTC](https://discuss.elastic.co/t/store-the-data-more-than-1tb-per-day/104511 "2017-10-20T06:16:16Z")
Hi Team, Can elasticsearch store big data more than 1TB per day? When the data corrupt, this corrupted data have backup?? Thanks and best regards Sharon --- ## [Connecting to Elasticsearch cluster in cloud from Django using connections.create\_connection API](https://discuss.elastic.co/t/connecting-to-elasticsearch-cluster-in-cloud-from-django-using-connections-create-connection-api/104650)
**Author:** [@shethsaam](https://discuss.elastic.co/u/shethsaam)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 5:14am UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-cluster-in-cloud-from-django-using-connections-create-connection-api/104650 "2017-10-20T05:14:51Z")
I am working on connecting Django to Elastic Search cluster held in cloud. To do so I am using the connections.create\_connection API. I am able to connect to my local ElasticSearch cluster using: connections.create\_conne… --- ## [Dangling - can not be imported as a dangling index, as index with same name already exists in cluster metadata](https://discuss.elastic.co/t/dangling-can-not-be-imported-as-a-dangling-index-as-index-with-same-name-already-exists-in-cluster-metadata/104649)
**Author:** [@iemem15](https://discuss.elastic.co/u/iemem15)\ **Replies:** 0\ **Last updated:** [October 20, 2017, 5:03am UTC](https://discuss.elastic.co/t/dangling-can-not-be-imported-as-a-dangling-index-as-index-with-same-name-already-exists-in-cluster-metadata/104649 "2017-10-20T05:03:42Z")
I Had two nodes independant not part of the cluster. I used logstash to index on each node. After i figured to configure the cluster, I added 2 nodes and one as client for kibana. Now when I start the cluster it repeadea… --- ## [Search Operators](https://discuss.elastic.co/t/search-operators/104012)
**Author:** [@Illidan](https://discuss.elastic.co/u/Illidan)\ **Replies:** 4\ **Last updated:** [October 20, 2017, 4:11am UTC](https://discuss.elastic.co/t/search-operators/104012 "2017-10-20T04:11:56Z")
I trying to achieve behavior similar to using Google search operators. For instance, Google supports "AND", and "OR" operators as part of query string, in order to produce more refined searches. For example: "Junior AND… --- ## [Min RAM during dev for large dataset](https://discuss.elastic.co/t/min-ram-during-dev-for-large-dataset/104479)
**Author:** [@duderson](https://discuss.elastic.co/u/duderson)\ **Replies:** 2\ **Last updated:** [October 19, 2017, 11:17pm UTC](https://discuss.elastic.co/t/min-ram-during-dev-for-large-dataset/104479 "2017-10-19T23:17:57Z")
So I've really got two questions here. If I have about 100GB of documents that I want to make searchable with elasticsearch, is it bad to just stick it in a single node / shard? ( I can figure out replicas later when … --- ## [ElasticSearch not working on IP](https://discuss.elastic.co/t/elasticsearch-not-working-on-ip/104572)
**Author:** [@adwaitjoshi](https://discuss.elastic.co/u/adwaitjoshi)\ **Replies:** 4\ **Last updated:** [October 19, 2017, 10:38pm UTC](https://discuss.elastic.co/t/elasticsearch-not-working-on-ip/104572 "2017-10-19T22:38:02Z")
I tried to modify the elasticsearch.yml and set the network.host: 192.168.1.226 which is my local IP and the port to 9200 however when I restart elastic and try to do a GREP I see that its not listening on 9200. When I c… --- ## [Connection refused when trying to run on aws ec2](https://discuss.elastic.co/t/connection-refused-when-trying-to-run-on-aws-ec2/104595)
**Author:** [@lemongrab](https://discuss.elastic.co/u/lemongrab)\ **Replies:** 3\ **Last updated:** [October 19, 2017, 9:52pm UTC](https://discuss.elastic.co/t/connection-refused-when-trying-to-run-on-aws-ec2/104595 "2017-10-19T21:52:29Z")
Hello. I tried curling to my ec2 instance's instance of elasticsearch but got a connection refused error. I tried doing this while remoting into it via ssh as follows: curl -X GET http://127.0.0.1:9200 and variations… --- ## [Master Node Heap Percent Usage vs. Refresh Rate](https://discuss.elastic.co/t/master-node-heap-percent-usage-vs-refresh-rate/104627)
**Author:** [@corona](https://discuss.elastic.co/u/corona)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 9:41pm UTC](https://discuss.elastic.co/t/master-node-heap-percent-usage-vs-refresh-rate/104627 "2017-10-19T21:41:16Z")
Hello, We have been experiencing issues with the master node (also a data node) running out of memory. I've been gathering metrics and correlating the heap used percent against our collected metrics. I found that in the… --- ## [Mapping dilemma](https://discuss.elastic.co/t/mapping-dilemma/104588)
**Author:** [@SebScoFr](https://discuss.elastic.co/u/SebScoFr)\ **Replies:** 2\ **Last updated:** [October 19, 2017, 6:16pm UTC](https://discuss.elastic.co/t/mapping-dilemma/104588 "2017-10-19T18:16:29Z")
I need to come up with a proper data model for the following scenario: We are building an application on which users can search for categories, stores and products. 1 Store can have multiple Products, 1 Product belongs… --- ## [Cannot find grok debugger after installing xpack](https://discuss.elastic.co/t/cannot-find-grok-debugger-after-installing-xpack/104619)
**Author:** [@pdna](https://discuss.elastic.co/u/pdna)\ **Replies:** 1\ **Last updated:** [October 19, 2017, 9:18pm UTC](https://discuss.elastic.co/t/cannot-find-grok-debugger-after-installing-xpack/104619 "2017-10-19T21:18:40Z")
I have installed elasticsearch5-x-pack-5.3.0 and kibana5-x-pack-5.3.0 packages and I cannot find grok debugger on the Kibana console. I only see Search Profile and Console. ES, Kibana version: 5.3 OS: FreeBSD 11 --- ## [Unable to save documents by timestamp - only one document is saved and updated everytime](https://discuss.elastic.co/t/unable-to-save-documents-by-timestamp-only-one-document-is-saved-and-updated-everytime/104597)
**Author:** [@nitinware](https://discuss.elastic.co/u/nitinware)\ **Replies:** 1\ **Last updated:** [October 19, 2017, 6:22pm UTC](https://discuss.elastic.co/t/unable-to-save-documents-by-timestamp-only-one-document-is-saved-and-updated-everytime/104597 "2017-10-19T18:22:44Z")
I want to save documents in an Index by timestamp. I am saving documents every second, so I want to see a document being created every second. But when I save the document by passing new timestamp value, a new document… --- ## [Large Drops in IOPS on AWS Elasticsearch Service & 504 Transport Error](https://discuss.elastic.co/t/large-drops-in-iops-on-aws-elasticsearch-service-504-transport-error/102716)
**Author:** [@blahblahdev](https://discuss.elastic.co/u/blahblahdev)\ **Replies:** 3\ **Last updated:** [October 19, 2017, 5:05pm UTC](https://discuss.elastic.co/t/large-drops-in-iops-on-aws-elasticsearch-service-504-transport-error/102716 "2017-10-19T17:05:19Z")
We have an AWS Elasticsearch Service cluster that we (are trying to) continually push data to. To me, it isn't a crazy amount of data, maybe 12M documents a day. We use the bulk endpoint to push through ~500 documents at… --- ## [Update by query performance](https://discuss.elastic.co/t/update-by-query-performance/104599)
**Author:** [@morphers82](https://discuss.elastic.co/u/morphers82)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 4:49pm UTC](https://discuss.elastic.co/t/update-by-query-performance/104599 "2017-10-19T16:49:12Z")
I have about 100million "parent" documents with about 20 "random" products in a field with an array of these 20 products. These documents includes field values that average and sum values from the 20 products, and these… --- ## [Aggregation on top\_hits aggregation results?](https://discuss.elastic.co/t/aggregation-on-top-hits-aggregation-results/104598)
**Author:** [@fpamba](https://discuss.elastic.co/u/fpamba)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 4:42pm UTC](https://discuss.elastic.co/t/aggregation-on-top-hits-aggregation-results/104598 "2017-10-19T16:42:25Z")
Hi, I've an offer data model containing a productId, a price and a rating. I want to: Only retain the cheapest offer for each productId Compute the average rating for remaining offers Here is what I've done below. … --- ## [Mapping issue: unsupported parameters](https://discuss.elastic.co/t/mapping-issue-unsupported-parameters/104593)
**Author:** [@Di\_Zou](https://discuss.elastic.co/u/Di_Zou)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 4:21pm UTC](https://discuss.elastic.co/t/mapping-issue-unsupported-parameters/104593 "2017-10-19T16:21:53Z")
Need some help here. I am not sure what is problem with my mapping. I am using elasticsearch 5.3.1. I got an error from my logstash: unsupported parameters: \[properties : {arn={fields={keyword={ignore\_above=256, type=k… --- ## [Bulk update successful, but documents aren't updated](https://discuss.elastic.co/t/bulk-update-successful-but-documents-arent-updated/104587)
**Author:** [@Natalia\_Kuleniuk](https://discuss.elastic.co/u/Natalia_Kuleniuk)\ **Replies:** 1\ **Last updated:** [October 19, 2017, 4:25pm UTC](https://discuss.elastic.co/t/bulk-update-successful-but-documents-arent-updated/104587 "2017-10-19T16:25:22Z")
I added new field to mapping, and then try to update all documents with new values for this field. curl -XPOST 'https://my\_end\_point/myindex/mytype/\_bulk' -d ' {"update":{"\_id":"1"}} {"doc": { "test\_field": "new field… --- ## [Mapping Explosion on Non-Indexed Fields?](https://discuss.elastic.co/t/mapping-explosion-on-non-indexed-fields/104580)
**Author:** [@HJewkes](https://discuss.elastic.co/u/HJewkes)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 3:41pm UTC](https://discuss.elastic.co/t/mapping-explosion-on-non-indexed-fields/104580 "2017-10-19T15:41:50Z")
I am familiar with the Mapping Explosion problem, but am unclear if it applies to fields that are not indexed in Lucene. My intention is to store a properties object that has a high cardinality of keys (on the order of … --- ## [How to reindex data in Elasticsearch without actually changing index names?](https://discuss.elastic.co/t/how-to-reindex-data-in-elasticsearch-without-actually-changing-index-names/104034)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 12\ **Last updated:** [October 19, 2017, 2:12pm UTC](https://discuss.elastic.co/t/how-to-reindex-data-in-elasticsearch-without-actually-changing-index-names/104034 "2017-10-19T14:12:58Z")
Hi, I am trying to reindex data in Elasticsearch without actually changing index names. I am ingesting data to indices based on monthly index names(index-%{+YYYY.MM}). I've seen concept of using aliases, but I am not su… --- ## [Possible to manually delete S3 backup repository?](https://discuss.elastic.co/t/possible-to-manually-delete-s3-backup-repository/101402)
**Author:** [@Patrick\_Hannigan](https://discuss.elastic.co/u/Patrick_Hannigan)\ **Replies:** 8\ **Last updated:** [October 19, 2017, 1:29pm UTC](https://discuss.elastic.co/t/possible-to-manually-delete-s3-backup-repository/101402 "2017-10-19T13:29:40Z")
Hi all - we've started backing up our elasticsearch cluster into a new S3 bucket/repo and would like to delete the previous bucket/repo that we were using. However, when we try to delete the old repo, we're getting the e… --- ## [ES plugin to expand search results](https://discuss.elastic.co/t/es-plugin-to-expand-search-results/104429)
**Author:** [@thampton](https://discuss.elastic.co/u/thampton)\ **Replies:** 2\ **Last updated:** [October 19, 2017, 1:29pm UTC](https://discuss.elastic.co/t/es-plugin-to-expand-search-results/104429 "2017-10-19T13:29:01Z")
Is it possible to build a plugin that can add additional records to a query result before the aggregation of the data happens? I have data external to the index that identifies additional documents from the index that I… --- ## [Configure an index using a low level rest Client](https://discuss.elastic.co/t/configure-an-index-using-a-low-level-rest-client/104389)
**Author:** [@gringogordo](https://discuss.elastic.co/u/gringogordo)\ **Replies:** 5\ **Last updated:** [October 19, 2017, 12:26pm UTC](https://discuss.elastic.co/t/configure-an-index-using-a-low-level-rest-client/104389 "2017-10-19T12:26:05Z")
I am (re)writing an index creation app in java and struggling a little bit. The existing app uses the TransportClient and I want to move to the High Level Rest Client (using the low level Rest Client to set it up). I w… --- ## [Discovery.zen.ping.unicast.hosts and custom ports](https://discuss.elastic.co/t/discovery-zen-ping-unicast-hosts-and-custom-ports/104530)
**Author:** [@toper](https://discuss.elastic.co/u/toper)\ **Replies:** 2\ **Last updated:** [October 19, 2017, 11:52am UTC](https://discuss.elastic.co/t/discovery-zen-ping-unicast-hosts-and-custom-ports/104530 "2017-10-19T11:52:31Z")
Hi, could somebody explain why i need to point ALL nodes (with ports) in discovery.zen.ping.unicast.hosts setting (my data nodes use custom http/transport ports)? My understanding was, that i need to point master nodes… --- ## [Failed to process cluster event Exception](https://discuss.elastic.co/t/failed-to-process-cluster-event-exception/103451)
**Author:** [@Pradeep\_Gowda](https://discuss.elastic.co/u/Pradeep_Gowda)\ **Replies:** 31\ **Last updated:** [October 19, 2017, 11:17am UTC](https://discuss.elastic.co/t/failed-to-process-cluster-event-exception/103451 "2017-10-19T11:17:56Z")
Currently, I am running Elasticsearch in cluster mode and there are 3 nodes. Approximately I have around 15000 indices. Each node has 64GB memory and 16 core CPU. My logs are flooded with below message whenever I try to… --- ## [Query, aggregation (date\_histogram) and timezone performance ES 5.6.2](https://discuss.elastic.co/t/query-aggregation-date-histogram-and-timezone-performance-es-5-6-2/104545)
**Author:** [@JohanRask](https://discuss.elastic.co/u/JohanRask)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 11:13am UTC](https://discuss.elastic.co/t/query-aggregation-date-histogram-and-timezone-performance-es-5-6-2/104545 "2017-10-19T11:13:29Z")
Hi! We are running ES 5.6.2 on an 8 core, 64GB (30 GB heap). We are using a SAN (yes, I know this is not recommended but let me continue). During the tests that we perform at the moment we use a single data node + sepa… --- ## [Sort by nested-nested where parent has type](https://discuss.elastic.co/t/sort-by-nested-nested-where-parent-has-type/104540)
**Author:** [@Bolandish](https://discuss.elastic.co/u/Bolandish)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 10:49am UTC](https://discuss.elastic.co/t/sort-by-nested-nested-where-parent-has-type/104540 "2017-10-19T10:49:07Z")
How do i sort these people by attributes.values.value where attribute.type is "score"?? I really can't figure it out. My data structure : \[ { name: "Thomas Bolander", attributes: \[ { type: "scor… --- ## [Watcher query returns nothing, works on manual search?](https://discuss.elastic.co/t/watcher-query-returns-nothing-works-on-manual-search/104449)
**Author:** [@ryanlutgen](https://discuss.elastic.co/u/ryanlutgen)\ **Replies:** 1\ **Last updated:** [October 19, 2017, 10:24am UTC](https://discuss.elastic.co/t/watcher-query-returns-nothing-works-on-manual-search/104449 "2017-10-19T10:24:40Z")
Hello, I am attempting to create a watcher with ES/Xpack 5.6.2 that will send a notification if an Xpack Security user count threshold is reached. The watcher looks like this, note I have the interval set to 10s for te… --- ## [Missing documents in new replica shards following Elasticsearch upgrade](https://discuss.elastic.co/t/missing-documents-in-new-replica-shards-following-elasticsearch-upgrade/104535)
**Author:** [@rdavies](https://discuss.elastic.co/u/rdavies)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 10:09am UTC](https://discuss.elastic.co/t/missing-documents-in-new-replica-shards-following-elasticsearch-upgrade/104535 "2017-10-19T10:09:53Z")
We have been running Elasticsearch clusters for a number of years, and they have provided sterling service. These clusters are used for time series data; for each cluster around 30 new indices are created daily at the w… --- ## [Go-elasticsearch index operation](https://discuss.elastic.co/t/go-elasticsearch-index-operation/104534)
**Author:** [@danielcuervo](https://discuss.elastic.co/u/danielcuervo)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 10:00am UTC](https://discuss.elastic.co/t/go-elasticsearch-index-operation/104534 "2017-10-19T10:00:08Z")
Hello there, I am trying to use the go-elasticsearch library in order to index documents from my application. I have been trying the Index operation but it seems not to work (probably I am missusing it). I went to loo… --- ## [How to handle same url with different parameters](https://discuss.elastic.co/t/how-to-handle-same-url-with-different-parameters/104526)
**Author:** [@VidyVideni](https://discuss.elastic.co/u/VidyVideni)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 9:13am UTC](https://discuss.elastic.co/t/how-to-handle-same-url-with-different-parameters/104526 "2017-10-19T09:13:36Z")
I wanna calculate response time for each url in my website, \[webwxgetmsgimg\] the first two rows ,.the third , the fourth actually the same url with different parameters, any way to merge them as one in elastic searc… --- ## [I tested elastic search but I get this error in logs](https://discuss.elastic.co/t/i-tested-elastic-search-but-i-get-this-error-in-logs/104524)
**Author:** [@Ahmed\_Soltan](https://discuss.elastic.co/u/Ahmed_Soltan)\ **Replies:** 0\ **Last updated:** [October 19, 2017, 9:08am UTC](https://discuss.elastic.co/t/i-tested-elastic-search-but-i-get-this-error-in-logs/104524 "2017-10-19T09:08:48Z")
Elasticsearch\\Common\\Exceptions\\ServerErrorResponseException: {"error":{"root\_cause":\[\],"type":"search\_phase\_execution\_exception","reason":"all shards failed","phase":"query","grouped":true,"failed\_shards":\[\]},"status":5… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1914) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1916)