# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1398 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1399 --- ## [Fix Unallocated Shards](https://discuss.elastic.co/t/fix-unallocated-shards/167948)
**Author:** [@keeshqs](https://discuss.elastic.co/u/keeshqs)\ **Replies:** 1\ **Last updated:** [February 12, 2019, 5:56am UTC](https://discuss.elastic.co/t/fix-unallocated-shards/167948 "2019-02-12T05:56:09Z")
Good day, I was hoping you could help us with the issue that we are are consistently encountering in Elasticsearch. We are running a single cluster with a single node in the Production environment and the indices are fa… --- ## [CAT api doesn't respond](https://discuss.elastic.co/t/cat-api-doesnt-respond/162596)
**Author:** [@sumanthkumarc](https://discuss.elastic.co/u/sumanthkumarc)\ **Replies:** 11\ **Last updated:** [February 12, 2019, 5:35am UTC](https://discuss.elastic.co/t/cat-api-doesnt-respond/162596 "2019-02-12T05:35:46Z")
Hi, I have a 4 node elasticsearch cluster on version 5.6.9, all running on containers with persistent backend and with a haproxy load balancer infront. At times there seems to be issue with only CAT api not responding. … --- ## [How to lower the burden of audit log](https://discuss.elastic.co/t/how-to-lower-the-burden-of-audit-log/167956)
**Author:** [@ivanyu](https://discuss.elastic.co/u/ivanyu)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 5:30am UTC](https://discuss.elastic.co/t/how-to-lower-the-burden-of-audit-log/167956 "2019-02-12T05:30:18Z")
Dear, My cluster broke down several times and the elasticsearch log says: failed to index audit event：\[access\_granted\]. internal queue is full. which may be caused by a high indexing rate or issue with the destination … --- ## [GCS timeout when creating snapshot](https://discuss.elastic.co/t/gcs-timeout-when-creating-snapshot/167952)
**Author:** [@cafuego](https://discuss.elastic.co/u/cafuego)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 5:01am UTC](https://discuss.elastic.co/t/gcs-timeout-when-creating-snapshot/167952 "2019-02-12T05:01:01Z")
Since upgrading my stack from 5.6.4 to 6.6.0, snapshots to GCS have started failing. The reason given in the error output is: IndexShardSnapshotFailedException\[com.google.cloud.storage.StorageException: Read timed out\];… --- ## [Fscrawler creating custome mapping](https://discuss.elastic.co/t/fscrawler-creating-custome-mapping/167950)
**Author:** [@vikas\_singhji](https://discuss.elastic.co/u/vikas_singhji)\ **Replies:** 1\ **Last updated:** [February 12, 2019, 4:35am UTC](https://discuss.elastic.co/t/fscrawler-creating-custome-mapping/167950 "2019-02-12T04:35:09Z")
I'm using FSCrawler 2.6 and it's working great for indexing the pdf document. However one issue i am facing is that it is putting all the contents of the PDF in the "content" field(i am newbie in this field). So, my ques… --- ## [How ES schema is determined while reading using hadoop](https://discuss.elastic.co/t/how-es-schema-is-determined-while-reading-using-hadoop/167694)
**Author:** [@Sonny\_Heer](https://discuss.elastic.co/u/Sonny_Heer)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 11:06pm UTC](https://discuss.elastic.co/t/how-es-schema-is-determined-while-reading-using-hadoop/167694 "2019-02-11T23:06:20Z")
Env: using this jar: elasticsearch-spark-13\_2.10-5.1.1.jar sqlContext.read.format("org.elasticsearch.spark.sql").option("es.nodes",es\_url).option("es.port", "443").option("es.nodes.wan.only", "true").option("es.net.ssl… --- ## [Cascading integration removed in 7.0.0](https://discuss.elastic.co/t/cascading-integration-removed-in-7-0-0/167934)
**Author:** [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 9:58pm UTC](https://discuss.elastic.co/t/cascading-integration-removed-in-7-0-0/167934 "2019-02-11T21:58:15Z")
Please be advised that as of ES-Hadoop version 6.6.0 the Cascading integration in ES-Hadoop has been deprecated, and has been removed from the project starting in version 7.0.0. For more information about this deprecati… --- ## [Found duplicate column(s) in the data schema, Need help on how to load such index data into Spark Dataframe](https://discuss.elastic.co/t/found-duplicate-column-s-in-the-data-schema-need-help-on-how-to-load-such-index-data-into-spark-dataframe/167517)
**Author:** [@Yasmeenc](https://discuss.elastic.co/u/Yasmeenc)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 9:50pm UTC](https://discuss.elastic.co/t/found-duplicate-column-s-in-the-data-schema-need-help-on-how-to-load-such-index-data-into-spark-dataframe/167517 "2019-02-11T21:50:54Z")
Hi Team, I am trying to read data from elasticsearch index and write into a spark dataframe, but the index has same field name with different cases(upper/lower case) below is the mapping, and the error I am getting is … --- ## [Not able to make API call on Elasticsearch 6.x for index pattern management](https://discuss.elastic.co/t/not-able-to-make-api-call-on-elasticsearch-6-x-for-index-pattern-management/167932)
**Author:** [@preddy75](https://discuss.elastic.co/u/preddy75)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 9:48pm UTC](https://discuss.elastic.co/t/not-able-to-make-api-call-on-elasticsearch-6-x-for-index-pattern-management/167932 "2019-02-11T21:48:14Z")
We used to make the call below to Elasticsearch to get configuration information during kibana index management jobs. After moving from 5.x to 6.x I get this error: 2019-02-11 21:00:51,450 ERROR Unable to connect to Ela… --- ## [Problem with retrieving data from Elasticsearch by Spark](https://discuss.elastic.co/t/problem-with-retrieving-data-from-elasticsearch-by-spark/166790)
**Author:** [@Yasmeenc](https://discuss.elastic.co/u/Yasmeenc)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 9:42pm UTC](https://discuss.elastic.co/t/problem-with-retrieving-data-from-elasticsearch-by-spark/166790 "2019-02-11T21:42:58Z")
Hi Team, I'm quite new to ES and ES-Hadoop. The code for pulling the data out from ES is like below: es\_read\_conf = { "es.nodes" : "", "es.port" : "80", "es.resource" : 'temprollover/rollover… --- ## [Cannot connect to elasticsearch from hive using Aws instance and Docker Container](https://discuss.elastic.co/t/cannot-connect-to-elasticsearch-from-hive-using-aws-instance-and-docker-container/165645)
**Author:** [@sanjayap](https://discuss.elastic.co/u/sanjayap)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 9:40pm UTC](https://discuss.elastic.co/t/cannot-connect-to-elasticsearch-from-hive-using-aws-instance-and-docker-container/165645 "2019-02-11T21:40:19Z")
Hi, I am running 2 different docker containers on AWS instance. One for HDP sandbox and other for elasticsearch. Both the containers are up and running on the respective port (i.e., HDP on 8080 and elasticsearch on 9400… --- ## [Repository-hdfs with kerberos failed](https://discuss.elastic.co/t/repository-hdfs-with-kerberos-failed/166491)
**Author:** [@iQIYI](https://discuss.elastic.co/u/iQIYI)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 9:36pm UTC](https://discuss.elastic.co/t/repository-hdfs-with-kerberos-failed/166491 "2019-02-11T21:36:07Z")
Hi, I want to create a secure repository-hdfs with kerberos. Place the keytab file in /etc/elasticsearch/repository-hdfs/krb5.keytab Edit the realm info in /etc/krb5.conf After done the above two steps, I can use ki… --- ## [Query about elasticsearch nodes](https://discuss.elastic.co/t/query-about-elasticsearch-nodes/167879)
**Author:** [@cghosh](https://discuss.elastic.co/u/cghosh)\ **Replies:** 3\ **Last updated:** [February 11, 2019, 8:04pm UTC](https://discuss.elastic.co/t/query-about-elasticsearch-nodes/167879 "2019-02-11T20:04:15Z")
Hi, I have 4 instances. One of the instance has 3 nodes like below and others has only one node. Instance 1: /var/lib/elasticsearch/elasticsearch/nodes/0 /var/lib/elasticsearch/elasticsearch/nodes/1 /var/lib/elastic… --- ## [How to reindex and add new field?](https://discuss.elastic.co/t/how-to-reindex-and-add-new-field/167660)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 7:55pm UTC](https://discuss.elastic.co/t/how-to-reindex-and-add-new-field/167660 "2019-02-11T19:55:39Z")
Hi, Currently I have two versions of a field in different indexes. Here I use session.id and there I use sessionId. I will make them equal in logstash for new values, but how can I change the old data via reindex? I w… --- ## [Elasticsearch could not store watch with watch id](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619)
**Author:** [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Replies:** 15\ **Last updated:** [February 11, 2019, 6:27pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619 "2019-02-11T18:27:46Z")
Hi, Am using ELK version 6.5.4. Running with trial license. I am getting the below ERROR message. could not store triggered watch with id \[\_d6Y1y0bQ4mypPMdh3YZzg\_xpack\_license\_expiration\_89c0aa03-0208-400b-a351-db9cf5… --- ## [Asynchronous Reindex request in ElasticSearch 6.3 is not working](https://discuss.elastic.co/t/asynchronous-reindex-request-in-elasticsearch-6-3-is-not-working/167910)
**Author:** [@Harshraj\_Shinde](https://discuss.elastic.co/u/Harshraj_Shinde)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 6:07pm UTC](https://discuss.elastic.co/t/asynchronous-reindex-request-in-elasticsearch-6-3-is-not-working/167910 "2019-02-11T18:07:14Z")
Hello Guys, I have been trying this code ReindexRequestBuilder builder = ReindexAction.INSTANCE.newRequestBuilder(getClient()) .source(sourceIndexName).abortOnVersionConflict(false) .de… --- ## [Bool must and must\_not as arrays in documentation](https://discuss.elastic.co/t/bool-must-and-must-not-as-arrays-in-documentation/167909)
**Author:** [@frankshad](https://discuss.elastic.co/u/frankshad)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 6:02pm UTC](https://discuss.elastic.co/t/bool-must-and-must-not-as-arrays-in-documentation/167909 "2019-02-11T18:02:34Z")
I have just discovered that must and must\_not in bool queries can be arrays. The documentation page doesn't say this and all the examples show an object for these and an array for should, so I assumed that was what it sh… --- ## [Geo\_shape query no longer working in 6.6.0](https://discuss.elastic.co/t/geo-shape-query-no-longer-working-in-6-6-0/167682)
**Author:** [@frankshad](https://discuss.elastic.co/u/frankshad)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 5:56pm UTC](https://discuss.elastic.co/t/geo-shape-query-no-longer-working-in-6-6-0/167682 "2019-02-11T17:56:26Z")
I tried upgrading my development system from 6.4 to 6.6.0. After this, point (in a document) in polygon (geo\_shape in the query) queries ("give me everyone located in this area") no longer work - no error or anything, ju… --- ## [How to detect when a cluster of elasticsearch is down?](https://discuss.elastic.co/t/how-to-detect-when-a-cluster-of-elasticsearch-is-down/167902)
**Author:** [@Lucas\_Fernandez](https://discuss.elastic.co/u/Lucas_Fernandez)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 5:29pm UTC](https://discuss.elastic.co/t/how-to-detect-when-a-cluster-of-elasticsearch-is-down/167902 "2019-02-11T17:29:28Z")
I don't know how to do a snapshot every one hour. And how to detect when the all nodes of a cluster of elasticsearch is down. --- ## [Cannot get cross cluster search working for Elasticsearch 6.4 on Kubernetes](https://discuss.elastic.co/t/cannot-get-cross-cluster-search-working-for-elasticsearch-6-4-on-kubernetes/167897)
**Author:** [@preddy75](https://discuss.elastic.co/u/preddy75)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 5:04pm UTC](https://discuss.elastic.co/t/cannot-get-cross-cluster-search-working-for-elasticsearch-6-4-on-kubernetes/167897 "2019-02-11T17:04:07Z")
We cannot get cross cluster search to work on Kubernetes. We cannot use this feature without exposing the remote nodes on the target Kubernetes cluster by way of Hostnetworking (which opens the ip and port of the Elastic… --- ## [Shards stuck in unassigned state](https://discuss.elastic.co/t/shards-stuck-in-unassigned-state/167874)
**Author:** [@Maddy\_M](https://discuss.elastic.co/u/Maddy_M)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 4:54pm UTC](https://discuss.elastic.co/t/shards-stuck-in-unassigned-state/167874 "2019-02-11T16:54:56Z")
Hi, Cluster hit the circuit breaker, so we increased the limit temporarily. But we restart one of the node and that node is stuck at 500 shards and unable to move further into initialize other shards. There are multip… --- ## [API'S](https://discuss.elastic.co/t/apis/167893)
**Author:** [@Dv\_Thiyanesh](https://discuss.elastic.co/u/Dv_Thiyanesh)\ **Replies:** 5\ **Last updated:** [February 11, 2019, 4:52pm UTC](https://discuss.elastic.co/t/apis/167893 "2019-02-11T16:52:45Z")
Hello All, Can we use Java to only read data from elasticsearch that we are already ingested logs file to elasticsearch? not to index from java and retrieving, only to read already created logs file's instead of kibana… --- ## [IDs with special characters](https://discuss.elastic.co/t/ids-with-special-characters/167701)
**Author:** [@kjamesb](https://discuss.elastic.co/u/kjamesb)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 4:32pm UTC](https://discuss.elastic.co/t/ids-with-special-characters/167701 "2019-02-11T16:32:22Z")
I'm setting up elasticsearch with my java spring app. When I'm attempting to get records, I'm having an issue with retrieval and it seems to be due to the ID's of my records. For example, here's an Id of a record: "GD+… --- ## [Problem to match patterns in phrases with Regexp Query](https://discuss.elastic.co/t/problem-to-match-patterns-in-phrases-with-regexp-query/167887)
**Author:** [@gustavovaliati](https://discuss.elastic.co/u/gustavovaliati)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 3:42pm UTC](https://discuss.elastic.co/t/problem-to-match-patterns-in-phrases-with-regexp-query/167887 "2019-02-11T15:42:32Z")
Hi guys! I am looking for a recommendation to build a search. In my database, I have dynamic patterns that should be matched and others that must not. For example, I have these two documents: { "\_id" : 1, "descri… --- ## [Completion of particular terms with proximity matching \[5.6\]](https://discuss.elastic.co/t/completion-of-particular-terms-with-proximity-matching-5-6/167890)
**Author:** [@Shirogami](https://discuss.elastic.co/u/Shirogami)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 3:46pm UTC](https://discuss.elastic.co/t/completion-of-particular-terms-with-proximity-matching-5-6/167890 "2019-02-11T15:46:42Z")
Hi, I would like to be able to query for quick b\* f\* in an index containing : quick brown fox, q brown fox, q brown asd eoigj dfg fox. This query must just return quick brown fox. Additionally, a search for q brown fo… --- ## [ES delete logs between timestamp range](https://discuss.elastic.co/t/es-delete-logs-between-timestamp-range/167876)
**Author:** [@shaheryar](https://discuss.elastic.co/u/shaheryar)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 3:28pm UTC](https://discuss.elastic.co/t/es-delete-logs-between-timestamp-range/167876 "2019-02-11T15:28:08Z")
Can any body point out what's wrong with the query? POST logstash-2019.02/\_delete\_by\_query { "query": { "range":{ "@timestamp" : { "gte": "2019-02-11T19:00:00Z" } } } } Output { "took": 0, "timed\_out": fals… --- ## [Remove pipeline using field wildcard](https://discuss.elastic.co/t/remove-pipeline-using-field-wildcard/167849)
**Author:** [@manikandanid](https://discuss.elastic.co/u/manikandanid)\ **Replies:** 6\ **Last updated:** [February 11, 2019, 3:16pm UTC](https://discuss.elastic.co/t/remove-pipeline-using-field-wildcard/167849 "2019-02-11T15:16:16Z")
I want to remove a field starting with ElapsedTimeAt\* in my remove processor pipeline. Is it possible? --- ## [Understanding Disk-based Shard Allocation better](https://discuss.elastic.co/t/understanding-disk-based-shard-allocation-better/167746)
**Author:** [@kghbln](https://discuss.elastic.co/u/kghbln)\ **Replies:** 10\ **Last updated:** [February 11, 2019, 2:45pm UTC](https://discuss.elastic.co/t/understanding-disk-based-shard-allocation-better/167746 "2019-02-11T14:45:43Z")
Even after reading the docu I have no idea what to set to my "elasticsearch.yml" file to prevent the following pain which I am getting all at the same time: low disk watermark \[20gb\] exceeded on ... high disk watermark… --- ## [Nest using system indices](https://discuss.elastic.co/t/nest-using-system-indices/167684)
**Author:** [@Marc-Antoine\_J](https://discuss.elastic.co/u/Marc-Antoine_J)\ **Replies:** 3\ **Last updated:** [February 11, 2019, 1:28pm UTC](https://discuss.elastic.co/t/nest-using-system-indices/167684 "2019-02-11T13:28:51Z")
Hi all, I'm trying to get my cluster stats out of the .monitoring-\* index with NEST and I can't get a result out so I was wondering if it does not allow querying system indices? TermQuery termQuery = new TermQuery { … --- ## [Increase Elastic Heap After Installation](https://discuss.elastic.co/t/increase-elastic-heap-after-installation/167837)
**Author:** [@joubeh](https://discuss.elastic.co/u/joubeh)\ **Replies:** 7\ **Last updated:** [February 11, 2019, 1:21pm UTC](https://discuss.elastic.co/t/increase-elastic-heap-after-installation/167837 "2019-02-11T13:21:32Z")
Hello, please note that I've installed ES using windows MSI I'm trying to increase heap size using jvm.options as below JVM configuration ################################################################ ## IMPORTANT: … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1397) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1399)