# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1677 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1678 --- ## [Read number of document marked as delete from stats](https://discuss.elastic.co/t/read-number-of-document-marked-as-delete-from-stats/133219)
**Author:** [@Rah](https://discuss.elastic.co/u/Rah)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 8:14pm UTC](https://discuss.elastic.co/t/read-number-of-document-marked-as-delete-from-stats/133219 "2018-05-24T20:14:27Z")
Hi, Is there a way to read number of documents that are marked for delete in an index as Java APIs.? stats gives below -\> }, "all": { "primaries": { "docs": { "count": 11, "deleted": 2 }, "store": { "size\_in\_b… --- ## [Can we have concatenation of multiple fields in "es.mapping.id"](https://discuss.elastic.co/t/can-we-have-concatenation-of-multiple-fields-in-es-mapping-id/133215)
**Author:** [@divyanshi](https://discuss.elastic.co/u/divyanshi)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 7:18pm UTC](https://discuss.elastic.co/t/can-we-have-concatenation-of-multiple-fields-in-es-mapping-id/133215 "2018-05-24T19:18:54Z")
Can we have concatenation of multiple fields in "es.mapping.id"? If yes how to do that? I am writing to ES using scala spark code and want to create the id with concatenation of two existing fields. --- ## [ElasticSearch migration from Version 0.19.7 to AWS ES 1.5](https://discuss.elastic.co/t/elasticsearch-migration-from-version-0-19-7-to-aws-es-1-5/133197)
**Author:** [@abinesh](https://discuss.elastic.co/u/abinesh)\ **Replies:** 3\ **Last updated:** [May 24, 2018, 7:12pm UTC](https://discuss.elastic.co/t/elasticsearch-migration-from-version-0-19-7-to-aws-es-1-5/133197 "2018-05-24T19:12:44Z")
I am trying to migrate data from ElasticSearch installed in our datacenter to ElasticSearch cluster in AWS. Source: ElasticSearch Cluster in current Datacenter: Version: 0.19.7 Number of Records: 100k Number of Nodes:… --- ## [Upgrading elasticsearch from 2.x to 6.x in Docker containers](https://discuss.elastic.co/t/upgrading-elasticsearch-from-2-x-to-6-x-in-docker-containers/133213)
**Author:** [@uliana\_andreeva](https://discuss.elastic.co/u/uliana_andreeva)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 7:12pm UTC](https://discuss.elastic.co/t/upgrading-elasticsearch-from-2-x-to-6-x-in-docker-containers/133213 "2018-05-24T19:12:19Z")
Hi! I'm trying to upgrade elasticsearch from 2.4.4 to 6.2.2. According to https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade.html, "To upgrade an Elasticsearch cluster running 2.x, you hav… --- ## [Xpack Query Match Not working](https://discuss.elastic.co/t/xpack-query-match-not-working/133212)
**Author:** [@djtecha](https://discuss.elastic.co/u/djtecha)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 7:07pm UTC](https://discuss.elastic.co/t/xpack-query-match-not-working/133212 "2018-05-24T19:07:50Z")
Hello, I'm trying to add a query match for a role in XPACK and it doesn't seem to be working. I believe this is due to the match part getting wrapped in quotes. POST \_xpack/security/role/kibana\_readall { "cluster":… --- ## [Write Performance at Scale](https://discuss.elastic.co/t/write-performance-at-scale/132894)
**Author:** [@jdebois](https://discuss.elastic.co/u/jdebois)\ **Replies:** 4\ **Last updated:** [May 24, 2018, 6:49pm UTC](https://discuss.elastic.co/t/write-performance-at-scale/132894 "2018-05-24T18:49:42Z")
Problem: Updating 100,000 records at once takes 30 minutes to complete Environment: 2 Elastic VMs 16VCPU 60GB RAM each 1 TB SSD each 10G connection between nodes Xms Xmx set to 24G "refresh\_interval": "30s", "number\_o… --- ## [What is future plan of scripted aggregations](https://discuss.elastic.co/t/what-is-future-plan-of-scripted-aggregations/132964)
**Author:** [@palashkulsh](https://discuss.elastic.co/u/palashkulsh)\ **Replies:** 6\ **Last updated:** [May 24, 2018, 6:28pm UTC](https://discuss.elastic.co/t/what-is-future-plan-of-scripted-aggregations/132964 "2018-05-24T18:28:58Z")
hello team, We are planning to use scripted aggregations feature of elasticsearch. But are apprehensive about it because doc \[ https://www.elastic.co/guide/en/elasticsearch/reference/6.x/search-aggregations-metrics-scri… --- ## [Allocating data to specific shard and node](https://discuss.elastic.co/t/allocating-data-to-specific-shard-and-node/133185)
**Author:** [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)\ **Replies:** 1\ **Last updated:** [May 24, 2018, 6:16pm UTC](https://discuss.elastic.co/t/allocating-data-to-specific-shard-and-node/133185 "2018-05-24T18:16:05Z")
Hi, I can see that it is possible to allocate shards to particular nodes, and (some) routing of data is also possible, but... how can I ensure that data will always end up in their own shard? Scenario: I have 3 nodes … --- ## [DeleteByQueryRequestBuilder no longer allows setting routing?](https://discuss.elastic.co/t/deletebyqueryrequestbuilder-no-longer-allows-setting-routing/132003)
**Author:** [@orogvany](https://discuss.elastic.co/u/orogvany)\ **Replies:** 1\ **Last updated:** [May 24, 2018, 5:56pm UTC](https://discuss.elastic.co/t/deletebyqueryrequestbuilder-no-longer-allows-setting-routing/132003 "2018-05-24T17:56:56Z")
I'm (finally) updating from ES 2.old to 6.2.4. However, it seems like the ability to set routing on a delete-by-query request via the DeleteByQueryRequestBuilder has gone missing? In the REST docs, it shows that routin… --- ## [Where can i find Examples of watches related to SOC so that i can use ELK as SIEM, any Reference which can help me to build a SIEM using ELK](https://discuss.elastic.co/t/where-can-i-find-examples-of-watches-related-to-soc-so-that-i-can-use-elk-as-siem-any-reference-which-can-help-me-to-build-a-siem-using-elk/131223)
**Author:** [@addanuj](https://discuss.elastic.co/u/addanuj)\ **Replies:** 7\ **Last updated:** [May 24, 2018, 5:56pm UTC](https://discuss.elastic.co/t/where-can-i-find-examples-of-watches-related-to-soc-so-that-i-can-use-elk-as-siem-any-reference-which-can-help-me-to-build-a-siem-using-elk/131223 "2018-05-24T17:56:12Z")
Hi I am trying to convert SPLUNK correlation rules to ELK using watchers, anyone has any knowledgebase which can help me take references of X-Pack (watches), any blog or any website, any repository, which contains simil… --- ## [ML Job - Edit JSON vs Create with gui?](https://discuss.elastic.co/t/ml-job-edit-json-vs-create-with-gui/133175)
**Author:** [@jc034240](https://discuss.elastic.co/u/jc034240)\ **Replies:** 2\ **Last updated:** [May 24, 2018, 5:18pm UTC](https://discuss.elastic.co/t/ml-job-edit-json-vs-create-with-gui/133175 "2018-05-24T17:18:04Z")
Does anyone know of a way through Kibana to create an ML job via JSON (vs going through gui), or editing the JSON after the ML job is created? --- ## [Get index pattern ID?](https://discuss.elastic.co/t/get-index-pattern-id/132598)
**Author:** [@sdndude](https://discuss.elastic.co/u/sdndude)\ **Replies:** 2\ **Last updated:** [May 24, 2018, 5:01pm UTC](https://discuss.elastic.co/t/get-index-pattern-id/132598 "2018-05-24T17:01:54Z")
Is there an API to get the index pattern IDs? I had to delete the index pattern that I had because the mappings changed and now none of my visualizations work. I would hate to have to re-create them when I can just edi… --- ## [Remove old document versions](https://discuss.elastic.co/t/remove-old-document-versions/133126)
**Author:** [@atom](https://discuss.elastic.co/u/atom)\ **Replies:** 2\ **Last updated:** [May 24, 2018, 4:04pm UTC](https://discuss.elastic.co/t/remove-old-document-versions/133126 "2018-05-24T16:04:52Z")
Hi, i am currently writing a script which makes use of update by query. I am trying to remove a field in all indices and documents older than 7 days. I can see in Kibana that it is working. But i think the information… --- ## [Mapper turns out to be very slow](https://discuss.elastic.co/t/mapper-turns-out-to-be-very-slow/131877)
**Author:** [@machong](https://discuss.elastic.co/u/machong)\ **Replies:** 6\ **Last updated:** [May 24, 2018, 3:52pm UTC](https://discuss.elastic.co/t/mapper-turns-out-to-be-very-slow/131877 "2018-05-24T15:52:36Z")
I'm using eshadoop lib to do some data process work. the hadoop application seems to get stuck when the mapper progress reached 80%. I tried to add some debug message into mapper and found that the program is not blocked… --- ## [Termquery is failing](https://discuss.elastic.co/t/termquery-is-failing/133174)
**Author:** [@kish](https://discuss.elastic.co/u/kish)\ **Replies:** 3\ **Last updated:** [May 24, 2018, 3:41pm UTC](https://discuss.elastic.co/t/termquery-is-failing/133174 "2018-05-24T15:41:24Z")
Hello - I have the following query searchResult, err := elasticSearchClient.Search(). Index("cscfcounter"). // search in index Query(elastic.NewTermQuery("NodeId", rn)). // specify the query… --- ## [Supported K8s Persisted Volume Storage Classes](https://discuss.elastic.co/t/supported-k8s-persisted-volume-storage-classes/133186)
**Author:** [@danwhitacre](https://discuss.elastic.co/u/danwhitacre)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 3:23pm UTC](https://discuss.elastic.co/t/supported-k8s-persisted-volume-storage-classes/133186 "2018-05-24T15:23:46Z")
Hey, I am actively deploying the elasticsearch container into k8s and am wondering what the supported storage classes for persisted volumes are? Specifically I am wondering what provisioners are supported as listed here… --- ## [Find differenence of a fields from 2 bucket values](https://discuss.elastic.co/t/find-differenence-of-a-fields-from-2-bucket-values/133166)
**Author:** [@leela\_kumili](https://discuss.elastic.co/u/leela_kumili)\ **Replies:** 1\ **Last updated:** [May 24, 2018, 3:18pm UTC](https://discuss.elastic.co/t/find-differenence-of-a-fields-from-2-bucket-values/133166 "2018-05-24T15:18:47Z")
Hi, I have a requirement where I want to find number total number of hits for for each 1 hours interval for last 2 hours and find diff between total hits of current hours to previous hours for each action , how can i bu… --- ## [Do elasticsearch doument participate in all OR conditions in bool query.?](https://discuss.elastic.co/t/do-elasticsearch-doument-participate-in-all-or-conditions-in-bool-query/133031)
**Author:** [@san88922](https://discuss.elastic.co/u/san88922)\ **Replies:** 3\ **Last updated:** [May 24, 2018, 2:37pm UTC](https://discuss.elastic.co/t/do-elasticsearch-doument-participate-in-all-or-conditions-in-bool-query/133031 "2018-05-24T14:37:25Z")
I am new to Elasticsearch, And we are working on a requirement,where document in the elasticsearch will be fetched based on the match types like fuzzyMatch,Wordmatch etc... We are facing performance issues here, when do… --- ## [Copying Value From Existing Field](https://discuss.elastic.co/t/copying-value-from-existing-field/133013)
**Author:** [@Varan](https://discuss.elastic.co/u/Varan)\ **Replies:** 3\ **Last updated:** [May 24, 2018, 2:29pm UTC](https://discuss.elastic.co/t/copying-value-from-existing-field/133013 "2018-05-24T14:29:09Z")
Hi, I have an index with data already indexed. I want to add a multi-field to an already existing field and to copy the content of the said field in it. PUT my\_index { "mappings": { "\_doc": { "properties":… --- ## [Create/Delete operations value](https://discuss.elastic.co/t/create-delete-operations-value/133029)
**Author:** [@Anton\_Garakh](https://discuss.elastic.co/u/Anton_Garakh)\ **Replies:** 3\ **Last updated:** [May 24, 2018, 2:26pm UTC](https://discuss.elastic.co/t/create-delete-operations-value/133029 "2018-05-24T14:26:49Z")
I'm wondering is it normal to create/delete indexes, let's say, once per 10 minutes. Is it memory or cpu consuming? Does elastic allocates storage and if so, how much is time waiting for this storage being available onc… --- ## [Elasticsearch same query has different performance](https://discuss.elastic.co/t/elasticsearch-same-query-has-different-performance/132765)
**Author:** [@ans76](https://discuss.elastic.co/u/ans76)\ **Replies:** 4\ **Last updated:** [May 24, 2018, 2:23pm UTC](https://discuss.elastic.co/t/elasticsearch-same-query-has-different-performance/132765 "2018-05-24T14:23:36Z")
I have 2 nodes in my cluster. n1: \[master:true data:true\] 8GB RAM 500GB Disk space n2: \[master:false data:true\] 8GB RAM 500GB Disk space Only one index for test(about 20000 docs), 5shards, 1replica Run the same quer… --- ## [Elastic cpu and memory spikes suddenly](https://discuss.elastic.co/t/elastic-cpu-and-memory-spikes-suddenly/133164)
**Author:** [@elasticapp](https://discuss.elastic.co/u/elasticapp)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 1:58pm UTC](https://discuss.elastic.co/t/elastic-cpu-and-memory-spikes-suddenly/133164 "2018-05-24T13:58:07Z")
Hi, My app uses elastic search for retrieving some results. For some reason, it sometimes, for unknown reason spikes 100% and even more. \[43\] here are the stats, can anyone tell what wrong with my indices? { "\_shard… --- ## [Best practice for creating pipelines](https://discuss.elastic.co/t/best-practice-for-creating-pipelines/133163)
**Author:** [@cloudvar](https://discuss.elastic.co/u/cloudvar)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 1:57pm UTC](https://discuss.elastic.co/t/best-practice-for-creating-pipelines/133163 "2018-05-24T13:57:48Z")
Best practice for creating pipelines in a microservice environment. Should I go with each project has their our pipeline and then add a jenkins job that deploy as an when things change. Or based on techology. One pipel… --- ## [What Analyzers if any does the meta data use?](https://discuss.elastic.co/t/what-analyzers-if-any-does-the-meta-data-use/133027)
**Author:** [@James\_Land](https://discuss.elastic.co/u/James_Land)\ **Replies:** 2\ **Last updated:** [May 24, 2018, 1:43pm UTC](https://discuss.elastic.co/t/what-analyzers-if-any-does-the-meta-data-use/133027 "2018-05-24T13:43:55Z")
I was trying to run some Term queries on an elastic index. I noticed that unlike all the other fields in my query the \_id field did not seem to get tokenized at all. Just wanted to confirm this and see if someone could … --- ## [Weird Connection error](https://discuss.elastic.co/t/weird-connection-error/133103)
**Author:** [@ardel](https://discuss.elastic.co/u/ardel)\ **Replies:** 1\ **Last updated:** [May 24, 2018, 1:04pm UTC](https://discuss.elastic.co/t/weird-connection-error/133103 "2018-05-24T13:04:07Z")
Hello Everyone, Is there any one here who can help with my curator problem. The issue is that, when I execute delete\_indices command I can see that there's no result but I think it's normal because there's no connectio… --- ## [Default behavior of xpack.security.enabled](https://discuss.elastic.co/t/default-behavior-of-xpack-security-enabled/133127)
**Author:** [@Tim-Erwin](https://discuss.elastic.co/u/Tim-Erwin)\ **Replies:** 2\ **Last updated:** [May 24, 2018, 12:57pm UTC](https://discuss.elastic.co/t/default-behavior-of-xpack-security-enabled/133127 "2018-05-24T12:57:42Z")
After upgrading an elasticsearch 2 instance to 5.6.9 I got an error "missing authentication token for REST request". I fixed that by setting "xpack.security.enabled: false". However, I'm also sucessfully running an elas… --- ## [Missing data in ES](https://discuss.elastic.co/t/missing-data-in-es/132813)
**Author:** [@Jyoti\_rs](https://discuss.elastic.co/u/Jyoti_rs)\ **Replies:** 3\ **Last updated:** [May 24, 2018, 12:46pm UTC](https://discuss.elastic.co/t/missing-data-in-es/132813 "2018-05-24T12:46:07Z")
I am using Elastic search version 2.2.0 and i published the data and i could see the response id but when i am trying to find the data in Kibana.The data is missing and not visible in kibana. Any quick support is appre… --- ## [Find "star wars" when i'm looking for "starwars"](https://discuss.elastic.co/t/find-star-wars-when-im-looking-for-starwars/133026)
**Author:** [@Arth](https://discuss.elastic.co/u/Arth)\ **Replies:** 4\ **Last updated:** [May 24, 2018, 12:41pm UTC](https://discuss.elastic.co/t/find-star-wars-when-im-looking-for-starwars/133026 "2018-05-24T12:41:21Z")
Hi, I have 200 products with the trade name "star wars" (separated by white space) but I don't get results when I'm searching "starwars" (all of it in lower case letters, not "StarWars") ? Is there a way please ? --- ## [Spark ES connector is taking time](https://discuss.elastic.co/t/spark-es-connector-is-taking-time/133137)
**Author:** [@kedarsdixit](https://discuss.elastic.co/u/kedarsdixit)\ **Replies:** 0\ **Last updated:** [May 24, 2018, 12:06pm UTC](https://discuss.elastic.co/t/spark-es-connector-is-taking-time/133137 "2018-05-24T12:06:17Z")
Hi, We are using Spark ES connector to get the data from Elasticsearch, for some reason when we put the query which is comprised of 1 bool and 2 range queries, the query takes quite longer than expected. However the Tra… --- ## [Discover: Request Timeout after 30000ms: ES](https://discuss.elastic.co/t/discover-request-timeout-after-30000ms-es/131400)
**Author:** [@rahulnama](https://discuss.elastic.co/u/rahulnama)\ **Replies:** 5\ **Last updated:** [May 24, 2018, 10:41am UTC](https://discuss.elastic.co/t/discover-request-timeout-after-30000ms-es/131400 "2018-05-24T10:41:31Z")
Hello Team Greetings of the day! I'm using ES on a node. Node Specifications: 2 VCPUs, 50GB DISK, 8 GB RAM Indices 4 Every Indice has less than 100mb of data. When I'm trying to visualise an indice data in discove… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1676) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1678)