# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1002 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1003 --- ## [De-Duplication Query](https://discuss.elastic.co/t/de-duplication-query/221441)
**Author:** [@mrbarret](https://discuss.elastic.co/u/mrbarret)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 1:47pm UTC](https://discuss.elastic.co/t/de-duplication-query/221441 "2020-02-28T13:47:59Z")
I have a collection of documents where each document looks like { "\_id": ... , "Author": ..., "Content": ...., "DateTime": ... } I would like to issue one query to the collection so that I get in response the olde… --- ## [Elasticsearch Prometheus exporter for Openshift](https://discuss.elastic.co/t/elasticsearch-prometheus-exporter-for-openshift/221440)
**Author:** [@Tomas\_Bartek](https://discuss.elastic.co/u/Tomas_Bartek)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 1:46pm UTC](https://discuss.elastic.co/t/elasticsearch-prometheus-exporter-for-openshift/221440 "2020-02-28T13:46:53Z")
Hi, I wonder why is Openshift for the Elasticsearch using/recommending plugin based Prometheus exporter over proxy (standalone) exporter from the official Prometheus exporter list. Plugin-based exporter: Proxy expo… --- ## [Elasticsearch Ranger Kerbeos](https://discuss.elastic.co/t/elasticsearch-ranger-kerbeos/221429)
**Author:** [@hsl](https://discuss.elastic.co/u/hsl)\ **Replies:** 1\ **Last updated:** [February 28, 2020, 12:59pm UTC](https://discuss.elastic.co/t/elasticsearch-ranger-kerbeos/221429 "2020-02-28T12:59:10Z")
I want to use Ranger to control the access to elasticsearch, so I intall elasticsearch 7.1.1, and open kerberos for es. after install elasticsearch-ranger plugin, it can Synchronize policy from ranger, and kerberos Certi… --- ## [Edge-ngram and irrelevant scoring](https://discuss.elastic.co/t/edge-ngram-and-irrelevant-scoring/221397)
**Author:** [@slim-souissi](https://discuss.elastic.co/u/slim-souissi)\ **Replies:** 1\ **Last updated:** [February 28, 2020, 12:38pm UTC](https://discuss.elastic.co/t/edge-ngram-and-irrelevant-scoring/221397 "2020-02-28T12:38:25Z")
Hello , I'm applying edge-ngram as index toxenizer . My problem is that when a term matches , it matches so many times on the same field . As a consequence , the score become too high and irrelevant. Here is an examp… --- ## [Cannot regenerate the the license](https://discuss.elastic.co/t/cannot-regenerate-the-the-license/221417)
**Author:** [@Vivek\_Vimal](https://discuss.elastic.co/u/Vivek_Vimal)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 11:36am UTC](https://discuss.elastic.co/t/cannot-regenerate-the-the-license/221417 "2020-02-28T11:36:23Z")
\[2020-02-28T15:34:48,940\]\[ERROR\]\[o.e.x.s.a.f.SecurityActionFilter\] \[O9MrWj0\] blocking \[cluster:monitor/stats\] operation due to expired license. Cluster health, cluster stats and indices stats operations are blocked on … --- ## [Please don't deprecate sparse vector fields](https://discuss.elastic.co/t/please-dont-deprecate-sparse-vector-fields/219063)
**Author:** [@George\_Kinsman](https://discuss.elastic.co/u/George_Kinsman)\ **Replies:** 1\ **Last updated:** [February 13, 2020, 3:29pm UTC](https://discuss.elastic.co/t/please-dont-deprecate-sparse-vector-fields/219063 "2020-02-13T15:29:55Z")
Hi there! I noticed that support for sparse vectors is being deprecated due to lack of interest and use cases: https://github.com/elastic/elasticsearch/pull/48781. This is a great article that discusses the trade off b… --- ## [Regarding Bulk Queue Size](https://discuss.elastic.co/t/regarding-bulk-queue-size/221413)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 11:26am UTC](https://discuss.elastic.co/t/regarding-bulk-queue-size/221413 "2020-02-28T11:26:02Z")
Hi, Can I ask, is the heap consumption of the bulk queue simply a factor of queue length multiplied by bulk payload size? I ask because I've seen users report very large bulk queue sizes and am wondering how that impact… --- ## [Reading Nested data from ElasticSearch via Spark Scala](https://discuss.elastic.co/t/reading-nested-data-from-elasticsearch-via-spark-scala/221403)
**Author:** [@Apurw](https://discuss.elastic.co/u/Apurw)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 10:36am UTC](https://discuss.elastic.co/t/reading-nested-data-from-elasticsearch-via-spark-scala/221403 "2020-02-28T10:36:06Z")
I am trying to read data from Elasticsearch via Spark Scala: Scala 2.11.8, Spark 2.3.0, Elasticsearch 5.6.8 To Connect -- spark2-shell --jars elasticsearch-spark-20\_2.11-5.6.8.jar val df = spark.read.format("org.elas… --- ## [ElasticSearch Cutodd Limit](https://discuss.elastic.co/t/elasticsearch-cutodd-limit/221390)
**Author:** [@Aditya2](https://discuss.elastic.co/u/Aditya2)\ **Replies:** 1\ **Last updated:** [February 28, 2020, 10:05am UTC](https://discuss.elastic.co/t/elasticsearch-cutodd-limit/221390 "2020-02-28T10:05:09Z")
I am trying to push data from a database to ElasticSearch. The records is in Json format and each records is of about 300 lines. I am trying to push about 1000 records. The ElasticSearch shows only 9-10 records. Is there… --- ## [Elasticsearch default credentials](https://discuss.elastic.co/t/elasticsearch-default-credentials/221384)
**Author:** [@Amine\_Maalfi](https://discuss.elastic.co/u/Amine_Maalfi)\ **Replies:** 2\ **Last updated:** [February 28, 2020, 9:55am UTC](https://discuss.elastic.co/t/elasticsearch-default-credentials/221384 "2020-02-28T09:55:10Z")
Hello guys, i've found that the default credentials for ES are elastic:changeme but it's not working for me! am i missing something? the error : { "error": { "root\_cause": \[ { "t… --- ## [How to find percentage using doc\_count in elastic query](https://discuss.elastic.co/t/how-to-find-percentage-using-doc-count-in-elastic-query/221394)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 9:17am UTC](https://discuss.elastic.co/t/how-to-find-percentage-using-doc-count-in-elastic-query/221394 "2020-02-28T09:17:17Z")
I am having an elastic result like below. { "key" : "2020-02-18T09:10:14.453Z-2020-02-26T09:10:14.453Z", "from" : 1.582017014453E12, "from\_as\_string" : "2020-02-18T09:10:14.453Z", "to" : 1.582708214453E12, "to\_as\_s… --- ## [Elasticsearch \_sql string containing Apostrophe](https://discuss.elastic.co/t/elasticsearch-sql-string-containing-apostrophe/221300)
**Author:** [@vikasp](https://discuss.elastic.co/u/vikasp)\ **Replies:** 1\ **Last updated:** [February 28, 2020, 8:57am UTC](https://discuss.elastic.co/t/elasticsearch-sql-string-containing-apostrophe/221300 "2020-02-28T08:57:26Z")
Hi, I have a query: GET /test\_sql/\_search { "query": { "bool": { "must": \[ { "match": { "tenant\_name.keyword": "KNOXVILLE'S CORPORATION" } } \] } } } The above query works fine. Now, I try the same thing… --- ## [Hardware requirements to build an ELK cluster with 5000 agents](https://discuss.elastic.co/t/hardware-requirements-to-build-an-elk-cluster-with-5000-agents/221327)
**Author:** [@premierpsp](https://discuss.elastic.co/u/premierpsp)\ **Replies:** 3\ **Last updated:** [February 28, 2020, 8:09am UTC](https://discuss.elastic.co/t/hardware-requirements-to-build-an-elk-cluster-with-5000-agents/221327 "2020-02-28T08:09:34Z")
That is. I need to monitor 5000 servers with metricbeat and I need help to know how many nodes and hardware is recommended. Thank You! --- ## [How to query keyword nested field that can be an integer values in ElasticSearch?](https://discuss.elastic.co/t/how-to-query-keyword-nested-field-that-can-be-an-integer-values-in-elasticsearch/221381)
**Author:** [@111230](https://discuss.elastic.co/u/111230)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 8:00am UTC](https://discuss.elastic.co/t/how-to-query-keyword-nested-field-that-can-be-an-integer-values-in-elasticsearch/221381 "2020-02-28T08:00:18Z")
This is example of field in ES document. How can i query 'prices.value' that can be only integers? In this case, value is "150.99" and can fully be converted to integer. But sometimes in can be a text something like "va… --- ## [ERROR Unable to connect to Elasticsearch cluster. Error: 'Elasticsearch' object has no attribute 'deprecation'](https://discuss.elastic.co/t/error-unable-to-connect-to-elasticsearch-cluster-error-elasticsearch-object-has-no-attribute-deprecation/221180)
**Author:** [@Gokulnath](https://discuss.elastic.co/u/Gokulnath)\ **Replies:** 2\ **Last updated:** [February 28, 2020, 6:01am UTC](https://discuss.elastic.co/t/error-unable-to-connect-to-elasticsearch-cluster-error-elasticsearch-object-has-no-attribute-deprecation/221180 "2020-02-28T06:01:57Z")
I have installed elasticsearch version 7.2.1 and curator version is 5.6.0, then i updated the curator to version 5.7.6.because of version issue. Now when i try to run a job means im getting this error. "ERROR Unable… --- ## [Fluentd connection to Elasticsearch(cluster)](https://discuss.elastic.co/t/fluentd-connection-to-elasticsearch-cluster/221229)
**Author:** [@Suat\_Bey](https://discuss.elastic.co/u/Suat_Bey)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 11:55am UTC](https://discuss.elastic.co/t/fluentd-connection-to-elasticsearch-cluster/221229 "2020-02-27T11:55:38Z")
Hi, We are trying to send rsyslog from fluentd(td-agent) to cluster elaskticsearch (with xpack), but getting an error . /etc/td-agent/td-agent.conf \ @type syslog port 5140 bind 0.0.0.0 tag system \ **Author:** [@chris99](https://discuss.elastic.co/u/chris99)\ **Replies:** 1\ **Last updated:** [February 28, 2020, 3:23am UTC](https://discuss.elastic.co/t/one-large-index-vs-split-indexes/221353 "2020-02-28T03:23:17Z") Overview We have a multitenant transactional SAAS application. I am busy building a prototype implementation for ElasticSeach. The idea is to index our transactional data then use it for fast search and analytical aggre… --- ## [Degraded Indexing Performance on v7.3.1 (from v5.6.10)](https://discuss.elastic.co/t/degraded-indexing-performance-on-v7-3-1-from-v5-6-10/221062)
**Author:** [@Jack\_Farrelly](https://discuss.elastic.co/u/Jack_Farrelly)\ **Replies:** 5\ **Last updated:** [February 28, 2020, 1:11am UTC](https://discuss.elastic.co/t/degraded-indexing-performance-on-v7-3-1-from-v5-6-10/221062 "2020-02-28T01:11:36Z")
Hey, We recently upgraded from ES version 5.6.10 to 7.3.1 and noticed indexing performance seems much worse. We could index at a rate of around 500k op/s, and now we peak at around 200k op/s. During indexing, we get bu… --- ## [Filtered Alias not working on Inner Hits](https://discuss.elastic.co/t/filtered-alias-not-working-on-inner-hits/221344)
**Author:** [@acuitymd](https://discuss.elastic.co/u/acuitymd)\ **Replies:** 0\ **Last updated:** [February 28, 2020, 12:57am UTC](https://discuss.elastic.co/t/filtered-alias-not-working-on-inner-hits/221344 "2020-02-28T00:57:27Z")
Hi, I have a multitenant architecture where there is a large set of "master" data between each tenant, then smaller pieces of tenant-specific data. To query only for a specific tenant, I have a filtered alias where I'm… --- ## [What is the recommended Maven repository for x-pack-transport compatible with ES 7.6?](https://discuss.elastic.co/t/what-is-the-recommended-maven-repository-for-x-pack-transport-compatible-with-es-7-6/221127)
**Author:** [@preeti20](https://discuss.elastic.co/u/preeti20)\ **Replies:** 2\ **Last updated:** [February 28, 2020, 12:42am UTC](https://discuss.elastic.co/t/what-is-the-recommended-maven-repository-for-x-pack-transport-compatible-with-es-7-6/221127 "2020-02-28T00:42:59Z")
Maven Central appears to be missing the x-pack-transport 7.6 https://mvnrepository.com/artifact/org.elasticsearch.client/x-pack-transport. The latest in maven is 6.0.0 But i see x-pack-transport 7.6 in https://artifact… --- ## [Can't produce a geo\_point or geo\_shape data type for JSON geospatial data via Elasticsearch viewable on Kibana](https://discuss.elastic.co/t/cant-produce-a-geo-point-or-geo-shape-data-type-for-json-geospatial-data-via-elasticsearch-viewable-on-kibana/221337)
**Author:** [@fburgess](https://discuss.elastic.co/u/fburgess)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 11:01pm UTC](https://discuss.elastic.co/t/cant-produce-a-geo-point-or-geo-shape-data-type-for-json-geospatial-data-via-elasticsearch-viewable-on-kibana/221337 "2020-02-27T23:01:01Z")
OS: CentOS 7 OS: CentOS 7 \[root@localhost logstash\]# curl -XGET 'http://localhost:9200' { "name" : "primary-node", "cluster\_name" : "elasticsearch", "cluster\_uuid" : "0tXluLtmSmyY9\_hj-XOfrQ", "version" : { "numbe… --- ## [Migrate SSAS Olap to the Elasticsearch](https://discuss.elastic.co/t/migrate-ssas-olap-to-the-elasticsearch/220265)
**Author:** [@luis2](https://discuss.elastic.co/u/luis2)\ **Replies:** 5\ **Last updated:** [February 27, 2020, 10:16pm UTC](https://discuss.elastic.co/t/migrate-ssas-olap-to-the-elasticsearch/220265 "2020-02-27T22:16:28Z")
Hi, I work at a company that is thinking about migrate his BI structure ( SSAS Olap and Power BI ) to the ElasticSearch. Our BI work basically with aggregations as sum, max, min, first, last, average at aggregation in m… --- ## [Creating an array of ids using Elasticsearch](https://discuss.elastic.co/t/creating-an-array-of-ids-using-elasticsearch/221332)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 10:05pm UTC](https://discuss.elastic.co/t/creating-an-array-of-ids-using-elasticsearch/221332 "2020-02-27T22:05:09Z")
Background information: I'm using Elastic APM to collect API data and send it to Elasticsearch. Now I would like to run a watcher that searches the data for latency. I would like to group all the trace.id of all the doc… --- ## [Indexes not migrating from hot to cold](https://discuss.elastic.co/t/indexes-not-migrating-from-hot-to-cold/220915)
**Author:** [@amcneill3](https://discuss.elastic.co/u/amcneill3)\ **Replies:** 1\ **Last updated:** [February 27, 2020, 9:59pm UTC](https://discuss.elastic.co/t/indexes-not-migrating-from-hot-to-cold/220915 "2020-02-27T21:59:53Z")
Hi. I have a Hot Cold setup (two nodes hot, one node cold). It does not appear any of my indexes in hot ever migrate to cold, given my ILM policy. For example: But index I expect to be cold by now is still in hot… --- ## [Adjusting custom mapping for new fields](https://discuss.elastic.co/t/adjusting-custom-mapping-for-new-fields/221085)
**Author:** [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Replies:** 1\ **Last updated:** [February 27, 2020, 9:59pm UTC](https://discuss.elastic.co/t/adjusting-custom-mapping-for-new-fields/221085 "2020-02-27T21:59:05Z")
I'm looking for the best approaches to adjust an index mapping every time there are new fields added to it (after a vendor or a beat update for example). Currently, the filebeat (7.6) googlecloud module (https://www.ela… --- ## [ES7: Do we need cluster.initial\_master\_nodes when a node restarts?](https://discuss.elastic.co/t/es7-do-we-need-cluster-initial-master-nodes-when-a-node-restarts/221320)
**Author:** [@UDixit](https://discuss.elastic.co/u/UDixit)\ **Replies:** 3\ **Last updated:** [February 27, 2020, 8:32pm UTC](https://discuss.elastic.co/t/es7-do-we-need-cluster-initial-master-nodes-when-a-node-restarts/221320 "2020-02-27T20:32:15Z")
As per the documents, we need Cluster.initial\_master\_nodes only for the first bootstrap of the cluster. How does a node join the cluster when it restarts? Does it only need the sed\_hosts to discover others? What if the… --- ## [How to speed up script query](https://discuss.elastic.co/t/how-to-speed-up-script-query/221125)
**Author:** [@bmax](https://discuss.elastic.co/u/bmax)\ **Replies:** 2\ **Last updated:** [February 27, 2020, 7:46pm UTC](https://discuss.elastic.co/t/how-to-speed-up-script-query/221125 "2020-02-27T19:46:20Z")
Hello, I have an mapping for a field visit\_times which is a date type and contains many epoch timestamps. I have some complex queries such as checking the dayOfWeek of any of the values within that array or even checkin… --- ## ["failed shard on node... ...Data too large, data for \[\\] would be" only for 3 most recent .monitoring-es indices](https://discuss.elastic.co/t/failed-shard-on-node-data-too-large-data-for-transport-request-would-be-only-for-3-most-recent-monitoring-es-indices/218827)
**Author:** [@mattsdevop](https://discuss.elastic.co/u/mattsdevop)\ **Replies:** 8\ **Last updated:** [February 27, 2020, 7:40pm UTC](https://discuss.elastic.co/t/failed-shard-on-node-data-too-large-data-for-transport-request-would-be-only-for-3-most-recent-monitoring-es-indices/218827 "2020-02-27T19:40:41Z")
Looking for some direction with this unique issue in our production cluster. Hoping that @HenningAndersen @dadoonet @Christian\_Dahlqvist and/or @Armin\_Braun might be able to poke in here and offer feedback. Details/beha… --- ## [Not able to query a wildcard](https://discuss.elastic.co/t/not-able-to-query-a-wildcard/220896)
**Author:** [@prasad98](https://discuss.elastic.co/u/prasad98)\ **Replies:** 6\ **Last updated:** [February 27, 2020, 7:28pm UTC](https://discuss.elastic.co/t/not-able-to-query-a-wildcard/220896 "2020-02-27T19:28:11Z")
Hi, I'm trying to query a document where it has field mapping\_type and product.categories\[0\].name field which I want to query it using a wildcard. I tried to run the query in this way: GET /products/\_search { "quer… --- ## [Does AWS ElasticSearch v7.1 supports script\_fields in queries?](https://discuss.elastic.co/t/does-aws-elasticsearch-v7-1-supports-script-fields-in-queries/220569)
**Author:** [@mhnamaki](https://discuss.elastic.co/u/mhnamaki)\ **Replies:** 5\ **Last updated:** [February 27, 2020, 7:22pm UTC](https://discuss.elastic.co/t/does-aws-elasticsearch-v7-1-supports-script-fields-in-queries/220569 "2020-02-27T19:22:49Z")
Hello, To return the distance information for each document and the query search origin, I'm using the following snippet: "script\_fields": { "distance": { "script": "doc\['location\_lat\_lon'\].arcDistance(40,-105)" } … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1001) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1003)