# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1801 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1802 --- ## [Date Histogram -- Aggregate By Hour?](https://discuss.elastic.co/t/date-histogram-aggregate-by-hour/118519)
**Author:** [@corona](https://discuss.elastic.co/u/corona)\ **Replies:** 1\ **Last updated:** [February 5, 2018, 10:17pm UTC](https://discuss.elastic.co/t/date-histogram-aggregate-by-hour/118519 "2018-02-05T22:17:40Z")
Hello, I'm running ES version 2.4.0. I would like to aggregate search results by hour. I'm running the same query over two time different time ranges. One time range has hourly indexes and the other has daily indexes. I… --- ## [ES6.1 Heap memory used by the index writer](https://discuss.elastic.co/t/es6-1-heap-memory-used-by-the-index-writer/116535)
**Author:** [@loren](https://discuss.elastic.co/u/loren)\ **Replies:** 8\ **Last updated:** [February 5, 2018, 10:15pm UTC](https://discuss.elastic.co/t/es6-1-heap-memory-used-by-the-index-writer/116535 "2018-02-05T22:15:02Z")
QQ: what would cause IndexWriter memory to constantly spike? I'm indexing data to a new 3 node ES6.1 cluster running on i3.xlarge instances. Each node is showing roughly equal numbers for index request rate, disk usage,… --- ## [Unable to assign shards - "structure needs cleaning"](https://discuss.elastic.co/t/unable-to-assign-shards-structure-needs-cleaning/118499)
**Author:** [@aleej](https://discuss.elastic.co/u/aleej)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 8:38pm UTC](https://discuss.elastic.co/t/unable-to-assign-shards-structure-needs-cleaning/118499 "2018-02-05T20:38:58Z")
Hi All! Wondering if anyone has suggestions. We've got a cluster that current keeps running into problems with shards being unassigned. Manually allocating the shard fails with the error "structure needs cleaning" (see… --- ## [ES full cluster restart requirement for major upgrade](https://discuss.elastic.co/t/es-full-cluster-restart-requirement-for-major-upgrade/118424)
**Author:** [@laurentiusoica](https://discuss.elastic.co/u/laurentiusoica)\ **Replies:** 4\ **Last updated:** [February 5, 2018, 8:27pm UTC](https://discuss.elastic.co/t/es-full-cluster-restart-requirement-for-major-upgrade/118424 "2018-02-05T20:27:42Z")
Hello, I plan an ES cluster that should not have downtime. I read that for major version upgrades there is a requirement to do a full cluster restart https://www.elastic.co/blog/multiple-elasticsearch-clusters. I see t… --- ## [Using painless with aggregation results](https://discuss.elastic.co/t/using-painless-with-aggregation-results/115368)
**Author:** [@elkcurious](https://discuss.elastic.co/u/elkcurious)\ **Replies:** 13\ **Last updated:** [February 5, 2018, 7:35pm UTC](https://discuss.elastic.co/t/using-painless-with-aggregation-results/115368 "2018-02-05T19:35:47Z")
Hi, I am a little new to using painless scripting. I was able to use it in a kibana visualization to convert the units of a field. Also in a query\_and\_update scenario. Now I have a situation/aggregation that does not… --- ## [Reindex + growing storage size](https://discuss.elastic.co/t/reindex-growing-storage-size/118502)
**Author:** [@mkovrigovich](https://discuss.elastic.co/u/mkovrigovich)\ **Replies:** 0\ **Last updated:** [February 5, 2018, 5:25pm UTC](https://discuss.elastic.co/t/reindex-growing-storage-size/118502 "2018-02-05T17:25:07Z")
Hi, in our system we do backups using reindex command with wait\_for\_completion=true like POST \_reindex/?wait\_for\_completion=true { "source": { "index": "master\_data" }, "dest": { … --- ## [Is it difficult to convert grok patterns with logstash to ElasticSearch's ingest pipelines?](https://discuss.elastic.co/t/is-it-difficult-to-convert-grok-patterns-with-logstash-to-elasticsearchs-ingest-pipelines/118496)
**Author:** [@ujjain](https://discuss.elastic.co/u/ujjain)\ **Replies:** 0\ **Last updated:** [February 5, 2018, 5:14pm UTC](https://discuss.elastic.co/t/is-it-difficult-to-convert-grok-patterns-with-logstash-to-elasticsearchs-ingest-pipelines/118496 "2018-02-05T17:14:07Z")
Do I have to do just this? Configure all files to use a specific pipeline in filebeat Set up 1 or multiple grok patterns per pipeline by curls to ElasticSearch --- ## [Bulk API doesn't do anything but passes command](https://discuss.elastic.co/t/bulk-api-doesnt-do-anything-but-passes-command/118487)
**Author:** [@tomak](https://discuss.elastic.co/u/tomak)\ **Replies:** 1\ **Last updated:** [February 5, 2018, 4:13pm UTC](https://discuss.elastic.co/t/bulk-api-doesnt-do-anything-but-passes-command/118487 "2018-02-05T16:13:14Z")
So I would like to add a JSON document using the bulk API. In terminal I use the command: curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/some\_index/keyword/\_bulk?pretty' --data-binary @ComparisonRe… --- ## [Types for JSON file using bulk API](https://discuss.elastic.co/t/types-for-json-file-using-bulk-api/118474)
**Author:** [@tomak](https://discuss.elastic.co/u/tomak)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 3:40pm UTC](https://discuss.elastic.co/t/types-for-json-file-using-bulk-api/118474 "2018-02-05T15:40:11Z")
I would like to add a JSON file with a lot of lines like this. {"index":{"\_index": "fake", "\_type": "keyword", "\_id":"2"}} {"info":"2", "bob":3} Is there a way of defining the type and index once and for all for a JSON… --- ## [Suddenly can't search data](https://discuss.elastic.co/t/suddenly-cant-search-data/118381)
**Author:** [@Casiram](https://discuss.elastic.co/u/Casiram)\ **Replies:** 3\ **Last updated:** [February 5, 2018, 12:42pm UTC](https://discuss.elastic.co/t/suddenly-cant-search-data/118381 "2018-02-05T12:42:49Z")
Hello, I have been using Elastic Search for a month and everything works fine. However since February 1 at midnight I can not recover the data that I insert via POST request like "url: 9200/xxx/xxx / \_search" (with para… --- ## [Sorting comblex Array elements with Painless script](https://discuss.elastic.co/t/sorting-comblex-array-elements-with-painless-script/118472)
**Author:** [@DBFBlackbull](https://discuss.elastic.co/u/DBFBlackbull)\ **Replies:** 1\ **Last updated:** [February 5, 2018, 2:46pm UTC](https://discuss.elastic.co/t/sorting-comblex-array-elements-with-painless-script/118472 "2018-02-05T14:46:00Z")
Hello! I am looking for a way to use Update API with painless script to update the following record. I am using C# and NEST but writing the script as inline so the functionality should be the same. { "name": "Boubl… --- ## [Validation failed because of missing type with bulk API](https://discuss.elastic.co/t/validation-failed-because-of-missing-type-with-bulk-api/118465)
**Author:** [@tomak](https://discuss.elastic.co/u/tomak)\ **Replies:** 5\ **Last updated:** [February 5, 2018, 2:30pm UTC](https://discuss.elastic.co/t/validation-failed-because-of-missing-type-with-bulk-api/118465 "2018-02-05T14:30:04Z")
Hello, I would like to add the following JSON file using the bulk API "index":{"\_id":"1"} "info":"1" however I get the following error "type" : "action\_request\_validation\_exception", "reason" : "Validation Failed: 1:… --- ## [Creating multi metric job can only use distinct count on IP](https://discuss.elastic.co/t/creating-multi-metric-job-can-only-use-distinct-count-on-ip/118218)
**Author:** [@macnibblet](https://discuss.elastic.co/u/macnibblet)\ **Replies:** 7\ **Last updated:** [February 5, 2018, 2:14pm UTC](https://discuss.elastic.co/t/creating-multi-metric-job-can-only-use-distinct-count-on-ip/118218 "2018-02-05T14:14:15Z")
Heya, Maybe I have misunderstood something when trying to setup my anomaly detection but I want to track an unusual amount of messages being sent from a sender IP and then split that by service. But for some reason in K… --- ## [Datatype Derive](https://discuss.elastic.co/t/datatype-derive/117819)
**Author:** [@Marcelo\_Nunes\_Alves](https://discuss.elastic.co/u/Marcelo_Nunes_Alves)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 1:25pm UTC](https://discuss.elastic.co/t/datatype-derive/117819 "2018-02-05T13:25:05Z")
Hello, Is there datatype Derive in the ElasticSearch? Similar to what there is in the RRDTool. --- ## [java.lang.NoSuchMethodError:](https://discuss.elastic.co/t/java-lang-nosuchmethoderror/118445)
**Author:** [@aravind\_apalya](https://discuss.elastic.co/u/aravind_apalya)\ **Replies:** 1\ **Last updated:** [February 5, 2018, 1:04pm UTC](https://discuss.elastic.co/t/java-lang-nosuchmethoderror/118445 "2018-02-05T13:04:47Z")
Hi Team, I have upgraded my elastic search java API's to v6.1.1 from v0.90.5 Everything compiled fine and have deployed in tomcat server While starting the server it's throwing following error. java.lang.NoSuchMethod… --- ## [Elastic uses hardware minimally while reindexing](https://discuss.elastic.co/t/elastic-uses-hardware-minimally-while-reindexing/115793)
**Author:** [@seb\_ma](https://discuss.elastic.co/u/seb_ma)\ **Replies:** 6\ **Last updated:** [February 5, 2018, 12:54pm UTC](https://discuss.elastic.co/t/elastic-uses-hardware-minimally-while-reindexing/115793 "2018-02-05T12:54:55Z")
Hello, our Setup: 4 Server, each with 2 cpus (each 24 vCore) 256 GB Ram, 2 SSDs (each 120GB) Raid0, 10Gbit interconnect. On each Server we have 4 ES 6.1.1 Docker-Container running, created with docker compose. Each ES… --- ## [ES service fails to start when a snapshot repo is unavailable](https://discuss.elastic.co/t/es-service-fails-to-start-when-a-snapshot-repo-is-unavailable/117112)
**Author:** [@Alexander\_Iliev](https://discuss.elastic.co/u/Alexander_Iliev)\ **Replies:** 1\ **Last updated:** [February 5, 2018, 12:45pm UTC](https://discuss.elastic.co/t/es-service-fails-to-start-when-a-snapshot-repo-is-unavailable/117112 "2018-02-05T12:45:55Z")
Hi, I noticed that when the NFS snapshot repo is unavailable during the elasticsearch service startup (e.g. is not mounted) the service refuses to start. Is this the expected behavior? I'd guess that issuing a warning… --- ## [Issue Upgrading from 2.3.1 to 6.1.0](https://discuss.elastic.co/t/issue-upgrading-from-2-3-1-to-6-1-0/117063)
**Author:** [@Sam\_Schlaak](https://discuss.elastic.co/u/Sam_Schlaak)\ **Replies:** 6\ **Last updated:** [February 5, 2018, 12:23pm UTC](https://discuss.elastic.co/t/issue-upgrading-from-2-3-1-to-6-1-0/117063 "2018-02-05T12:23:50Z")
Hello. My application uses the Elasticsearch Java API to communicate with the cluster. Previously, when creating an index and assigning a mapping, it was done as follows: create: client.admin().indices().prepareCreate… --- ## [Elasticsearch NEST client \[search\] requests with source filtering are not working as expected](https://discuss.elastic.co/t/elasticsearch-nest-client-search-requests-with-source-filtering-are-not-working-as-expected/118412)
**Author:** [@manwa](https://discuss.elastic.co/u/manwa)\ **Replies:** 4\ **Last updated:** [February 5, 2018, 12:27pm UTC](https://discuss.elastic.co/t/elasticsearch-nest-client-search-requests-with-source-filtering-are-not-working-as-expected/118412 "2018-02-05T12:27:55Z")
Hello , I'm currently using elasticsearch NEST client v 6.0.0 with ES v6.1.2. When i use NEST search request with source filtering to include the specified fields only in the returned documents, the source for each doc… --- ## [Cannot see non-reserved users and roles since 6.1 upgrade](https://discuss.elastic.co/t/cannot-see-non-reserved-users-and-roles-since-6-1-upgrade/117970)
**Author:** [@stemps](https://discuss.elastic.co/u/stemps)\ **Replies:** 4\ **Last updated:** [February 5, 2018, 12:23pm UTC](https://discuss.elastic.co/t/cannot-see-non-reserved-users-and-roles-since-6-1-upgrade/117970 "2018-02-05T12:23:51Z")
We upgraded from 5.x to 6.1 and since then I cannot see any of the manually created users and roles anymore under Management -\> Users / Roles. I cannot see them on the dev console using the GET /\_xpack/security/user endp… --- ## [ERROR: All shards failed for phase: \[query\]](https://discuss.elastic.co/t/error-all-shards-failed-for-phase-query/118236)
**Author:** [@gunesg](https://discuss.elastic.co/u/gunesg)\ **Replies:** 18\ **Last updated:** [February 5, 2018, 12:11pm UTC](https://discuss.elastic.co/t/error-all-shards-failed-for-phase-query/118236 "2018-02-05T12:11:04Z")
Hello, I'm using ElasticSearch and Kibana with X-Pack. When I try to start elasticsearch, I get these error messages. And I have no logout button on Kibana. I've created a kibana\_user and a superuser, I Can log in wit… --- ## [Error in Elasticseach logs](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419)
**Author:** [@Kaouther\_Mechri](https://discuss.elastic.co/u/Kaouther_Mechri)\ **Replies:** 5\ **Last updated:** [February 5, 2018, 11:41am UTC](https://discuss.elastic.co/t/error-in-elasticseach-logs/118419 "2018-02-05T11:41:35Z")
Hello , When I access to dashbord in Kibana I have this message in elasticsearch logs . did you encouter such problems ? thanks Sc2w4wt3q-7cZg\]: Failed to execute \[SearchRequest{searchType=QUERY\_THEN\_FETCH, indices=… --- ## [Preventing a Gauss decay function from returning 0 inside a function score query](https://discuss.elastic.co/t/preventing-a-gauss-decay-function-from-returning-0-inside-a-function-score-query/117462)
**Author:** [@mortenxd](https://discuss.elastic.co/u/mortenxd)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 11:38am UTC](https://discuss.elastic.co/t/preventing-a-gauss-decay-function-from-returning-0-inside-a-function-score-query/117462 "2018-02-05T11:38:37Z")
Hello, I'm using a Gauss decay function to decrease relevance of old documents in my index (origin is current time). Is there a way to prevent a Gauss decay function from returning 0 for very old documents in this case? … --- ## [Speeding up a re-deploy](https://discuss.elastic.co/t/speeding-up-a-re-deploy/114423)
**Author:** [@Johnson\_Johnson](https://discuss.elastic.co/u/Johnson_Johnson)\ **Replies:** 3\ **Last updated:** [February 5, 2018, 11:05am UTC](https://discuss.elastic.co/t/speeding-up-a-re-deploy/114423 "2018-02-05T11:05:36Z")
I did a lot of research, and didn't find anything that was 100% clear to me as far as re-deploying a cluster. I found some good tutorials, and blog posts that touch on the subject with insightful information such as dis… --- ## [Include Fields in Watcher Email Alert](https://discuss.elastic.co/t/include-fields-in-watcher-email-alert/115700)
**Author:** [@MultiplierMultiplier](https://discuss.elastic.co/u/MultiplierMultiplier)\ **Replies:** 18\ **Last updated:** [February 5, 2018, 10:59am UTC](https://discuss.elastic.co/t/include-fields-in-watcher-email-alert/115700 "2018-02-05T10:59:35Z")
Hi there, I have set up an advanced Watch and it triggers upon 3 or more of the same TargetUserName with the EventID:4625, basically that alerts on any user failing to logon 3 times in 3 minutes. Here is my body section… --- ## [GeoPoint - how to pass accuracy?](https://discuss.elastic.co/t/geopoint-how-to-pass-accuracy/118425)
**Author:** [@Nikita\_Krasnov](https://discuss.elastic.co/u/Nikita_Krasnov)\ **Replies:** 0\ **Last updated:** [February 5, 2018, 10:41am UTC](https://discuss.elastic.co/t/geopoint-how-to-pass-accuracy/118425 "2018-02-05T10:41:18Z")
I have such information for saving to ES in my json: { "geolocation": { "latitude": 50.443278299999996, "longitude": 30.5075698, "accuracy": 3966 } } geolocation in my Java object is GeoPoint (private Ge… --- ## [Limit size of nested collection on Query Result](https://discuss.elastic.co/t/limit-size-of-nested-collection-on-query-result/118403)
**Author:** [@rupanjanbaidya](https://discuss.elastic.co/u/rupanjanbaidya)\ **Replies:** 0\ **Last updated:** [February 5, 2018, 8:26am UTC](https://discuss.elastic.co/t/limit-size-of-nested-collection-on-query-result/118403 "2018-02-05T08:26:18Z")
I am fairly a new user of Elasticsearch. Struggling a bit to build a query which limits the no of collections on a given nested field. Elasticsearch version 6.1 One product can have multiple stores as below, "hits" :… --- ## [Performance test 2.0 to 5.0 for server set up](https://discuss.elastic.co/t/performance-test-2-0-to-5-0-for-server-set-up/118373)
**Author:** [@0NE](https://discuss.elastic.co/u/0NE)\ **Replies:** 3\ **Last updated:** [February 5, 2018, 10:38am UTC](https://discuss.elastic.co/t/performance-test-2-0-to-5-0-for-server-set-up/118373 "2018-02-05T10:38:14Z")
Hello guys, We are moving away from our legacy version of elasticsearch 2.0 (with marvel and alerting) to 5.0 with xpack. I was wondering what is an easy and good way to performance test elastic? I want to see how it a… --- ## [Dedicated master-eligible nodes](https://discuss.elastic.co/t/dedicated-master-eligible-nodes/118347)
**Author:** [@laurentiusoica](https://discuss.elastic.co/u/laurentiusoica)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 10:18am UTC](https://discuss.elastic.co/t/dedicated-master-eligible-nodes/118347 "2018-02-05T10:18:40Z")
Hello, When talking about nodes, the ES documentation says: "To ensure that your master node is stable and not under pressure, it is a good idea in a bigger cluster to split the roles between dedicated master-eligible n… --- ## [Snapshots on ES 5.4 generated with id name instead of index name](https://discuss.elastic.co/t/snapshots-on-es-5-4-generated-with-id-name-instead-of-index-name/118408)
**Author:** [@andrew\_09](https://discuss.elastic.co/u/andrew_09)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 10:04am UTC](https://discuss.elastic.co/t/snapshots-on-es-5-4-generated-with-id-name-instead-of-index-name/118408 "2018-02-05T10:04:04Z")
Hi all, this is my first post here on the community forum. So, i have this kind of issue. I've set up a new 5.6 cluster (1 master/2only data). When i try to create snapshots, this is what i get: drwxr-xr-x 1 root roo… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1800) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1802)