# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=803 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 804 --- ## [Object mapping for \[Data\] tried to parse field \[Data\] as object, but found a concrete value](https://discuss.elastic.co/t/object-mapping-for-data-tried-to-parse-field-data-as-object-but-found-a-concrete-value/249476)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 0\ **Last updated:** [September 22, 2020, 8:29am UTC](https://discuss.elastic.co/t/object-mapping-for-data-tried-to-parse-field-data-as-object-but-found-a-concrete-value/249476 "2020-09-22T08:29:25Z")
elasticsearch - Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"mall\_log-2020.09", :routing=\>nil, :\_type=\>"\_doc"}, #LogStash::Event:0x52c34e1a\], :response=\>{"index"=\>{"\_in… --- ## [Can I predict how much memory do I need?](https://discuss.elastic.co/t/can-i-predict-how-much-memory-do-i-need/249431)
**Author:** [@monit\_pakubuwono](https://discuss.elastic.co/u/monit_pakubuwono)\ **Replies:** 1\ **Last updated:** [September 22, 2020, 7:56am UTC](https://discuss.elastic.co/t/can-i-predict-how-much-memory-do-i-need/249431 "2020-09-22T07:56:30Z")
Hi, I am sorry if my question is silly, I am actually a chemical engineer that want to use Elastic Search on my own mobile app. I want to make deployment using Elastic Cloud. there are storage and memory, I can predict… --- ## [Could not locate that index-pattern (id: filebeat-\*)](https://discuss.elastic.co/t/could-not-locate-that-index-pattern-id-filebeat/249447)
**Author:** [@Mobin](https://discuss.elastic.co/u/Mobin)\ **Replies:** 1\ **Last updated:** [September 22, 2020, 7:34am UTC](https://discuss.elastic.co/t/could-not-locate-that-index-pattern-id-filebeat/249447 "2020-09-22T07:34:56Z")
Dear Team, I am new to Elastic stack and recently I created a elastic stack setup using file beat. My intention is to monitor the Linux server logs, So I installed Elasticsearch, Logstash,filebeat, and Kibana on server … --- ## [Indicies turn yellow from specific date - only 1% of Data is shown in Kibana](https://discuss.elastic.co/t/indicies-turn-yellow-from-specific-date-only-1-of-data-is-shown-in-kibana/249364)
**Author:** [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\ **Replies:** 3\ **Last updated:** [September 22, 2020, 7:21am UTC](https://discuss.elastic.co/t/indicies-turn-yellow-from-specific-date-only-1-of-data-is-shown-in-kibana/249364 "2020-09-22T07:21:31Z")
Hello Community first of all - some infos about the cluster i'm using: 3 Nodes - 1 Master - 2 Data Nodes (All of them running CentOS 7) Kibana Version 7.8 Elasticsearch Version 7.8 Logstash Version 7.8 Filebeat Vers… --- ## [Rollup Jobs extremly high CPU USage](https://discuss.elastic.co/t/rollup-jobs-extremly-high-cpu-usage/249359)
**Author:** [@michaelpietzsch](https://discuss.elastic.co/u/michaelpietzsch)\ **Replies:** 2\ **Last updated:** [September 22, 2020, 7:20am UTC](https://discuss.elastic.co/t/rollup-jobs-extremly-high-cpu-usage/249359 "2020-09-22T07:20:35Z")
Hi Guys and Gals, i've got a rollup job running on my cluster. Once its active CPU Spikes extremely. The cluster consists of 6 Dedicated data nodes. How can this happen? Regards Michael --- ## [Sending logs to elsatic-stack from another eks application cluster via filebeat](https://discuss.elastic.co/t/sending-logs-to-elsatic-stack-from-another-eks-application-cluster-via-filebeat/248520)
**Author:** [@Raman\_Sawhney](https://discuss.elastic.co/u/Raman_Sawhney)\ **Replies:** 2\ **Last updated:** [September 22, 2020, 7:03am UTC](https://discuss.elastic.co/t/sending-logs-to-elsatic-stack-from-another-eks-application-cluster-via-filebeat/248520 "2020-09-22T07:03:24Z")
Hello Team, just wanted to check on one thing may be it sound silly but i am asking. Here is the scenerio that we have one eks cluster where we have installed elastic-stack all the pods and everything. If i want to sen… --- ## [Records more than 10,000 with pagination](https://discuss.elastic.co/t/records-more-than-10-000-with-pagination/249398)
**Author:** [@Pankaj\_Jadhav](https://discuss.elastic.co/u/Pankaj_Jadhav)\ **Replies:** 4\ **Last updated:** [September 22, 2020, 6:58am UTC](https://discuss.elastic.co/t/records-more-than-10-000-with-pagination/249398 "2020-09-22T06:58:10Z")
Hi Team, Am having more than 10,000 + records under elastic. Now my requirement is i've to show records to user with pagination. Am able to show records upto 10,000 with back end logic like elastic query { "from" : … --- ## [Performance for small index](https://discuss.elastic.co/t/performance-for-small-index/249329)
**Author:** [@Mathmarc](https://discuss.elastic.co/u/Mathmarc)\ **Replies:** 6\ **Last updated:** [September 22, 2020, 5:54am UTC](https://discuss.elastic.co/t/performance-for-small-index/249329 "2020-09-22T05:54:55Z")
Hi everyone, I have multiple question about the optimization of a cluster with: running on c5.xlarge (4cpu 8GB ram) huge read req/s and small write req/s. small index 20GB to 30GB. experimenting shard between 1GB to … --- ## [Sending syslog logs through logstash](https://discuss.elastic.co/t/sending-syslog-logs-through-logstash/249442)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 1\ **Last updated:** [September 22, 2020, 5:07am UTC](https://discuss.elastic.co/t/sending-syslog-logs-through-logstash/249442 "2020-09-22T05:07:04Z")
I've configure rsyslog server logs are coming at the server side input { tcp { port =\> 10514 } } output { elasticsearch { hosts =\> "localhost:9200" index =\> "client-syslog-%{+YYYY.MM.dd}" } } the rsyslog logs are com… --- ## [Anomaly detection using derivative aggregation](https://discuss.elastic.co/t/anomaly-detection-using-derivative-aggregation/249207)
**Author:** [@heric](https://discuss.elastic.co/u/heric)\ **Replies:** 3\ **Last updated:** [September 22, 2020, 4:38am UTC](https://discuss.elastic.co/t/anomaly-detection-using-derivative-aggregation/249207 "2020-09-22T04:38:15Z")
Hi All, I am trying to define anomaly detection using derivative aggregation because the value that im interested in is accumulated counter ( increasing in each interval ). Below are the anomaly detector and datafeed t… --- ## [How to prevent Elasticsearch match on only one character with multi\_match](https://discuss.elastic.co/t/how-to-prevent-elasticsearch-match-on-only-one-character-with-multi-match/249441)
**Author:** [@EHH](https://discuss.elastic.co/u/EHH)\ **Replies:** 0\ **Last updated:** [September 22, 2020, 4:15am UTC](https://discuss.elastic.co/t/how-to-prevent-elasticsearch-match-on-only-one-character-with-multi-match/249441 "2020-09-22T04:15:43Z")
Hi there, I'm using elasiticsearch-dsl and SmartCN analyzer: from elasticsearch\_dsl import analyzer analyzer\_cn = analyzer( 'smartcn', tokenizer=tokenizer('smartcn\_tokenizer'), filter=\['lowercase'\] ) And I… --- ## [How to only query on data nodes by elasticsearch-py?](https://discuss.elastic.co/t/how-to-only-query-on-data-nodes-by-elasticsearch-py/249293)
**Author:** [@derobukal](https://discuss.elastic.co/u/derobukal)\ **Replies:** 3\ **Last updated:** [September 22, 2020, 4:00am UTC](https://discuss.elastic.co/t/how-to-only-query-on-data-nodes-by-elasticsearch-py/249293 "2020-09-22T04:00:39Z")
I am using elasticsearch-py, which version is 7.5.1. I have 3 master nodes and 6 data nodes, master nodes have 2gb heap and data nodes have 30 gb heap. How can I only query data on my data nodes? I read the source code… --- ## [Invalid response for \_forcemerge, \_cache/clear APIs when unassigned shards](https://discuss.elastic.co/t/invalid-response-for-forcemerge-cache-clear-apis-when-unassigned-shards/248901)
**Author:** [@setiah](https://discuss.elastic.co/u/setiah)\ **Replies:** 1\ **Last updated:** [September 21, 2020, 11:58pm UTC](https://discuss.elastic.co/t/invalid-response-for-forcemerge-cache-clear-apis-when-unassigned-shards/248901 "2020-09-21T23:58:42Z")
Hi, Issue When \_forcemerge is tried on an index or a group of indices that has some unassigned shards, the response returned does not indicate failures for unassigned shards. Even the total, failed and successful count… --- ## [Adding new field to the mapping template](https://discuss.elastic.co/t/adding-new-field-to-the-mapping-template/249102)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 2\ **Last updated:** [September 21, 2020, 9:22pm UTC](https://discuss.elastic.co/t/adding-new-field-to-the-mapping-template/249102 "2020-09-21T21:22:49Z")
Hi, From time to time, I need to add a new field to my templates (mostly daily indices). Is there an easy way to apply that to an existing index? Thank you! --- ## [Update returned values in response](https://discuss.elastic.co/t/update-returned-values-in-response/249376)
**Author:** [@Cyril\_Guiet](https://discuss.elastic.co/u/Cyril_Guiet)\ **Replies:** 1\ **Last updated:** [September 21, 2020, 9:08pm UTC](https://discuss.elastic.co/t/update-returned-values-in-response/249376 "2020-09-21T21:08:01Z")
Hello ! after my query I would like to know if it's possible to update the "instanceof" values by there labels equivalents ( municipalities and cities of Serbia == Q783930 ) . Thanks :slight\_smile: returned answer sam… --- ## [Automatic deletion of read only indices with 0 documents](https://discuss.elastic.co/t/automatic-deletion-of-read-only-indices-with-0-documents/249378)
**Author:** [@mjedynak](https://discuss.elastic.co/u/mjedynak)\ **Replies:** 1\ **Last updated:** [September 21, 2020, 8:57pm UTC](https://discuss.elastic.co/t/automatic-deletion-of-read-only-indices-with-0-documents/249378 "2020-09-21T20:57:59Z")
I have index lifecycle policy (hot phase 1 month -\> warm phase 1 year -\> delete phase) My index has 2 primary shards + 1 replica (4 shards in total). Sometimes it happens that my index has 0 documents and it rolls over… --- ## [Matrix aggregate question](https://discuss.elastic.co/t/matrix-aggregate-question/249421)
**Author:** [@Aldo\_Sarmiento](https://discuss.elastic.co/u/Aldo_Sarmiento)\ **Replies:** 0\ **Last updated:** [September 21, 2020, 8:41pm UTC](https://discuss.elastic.co/t/matrix-aggregate-question/249421 "2020-09-21T20:41:52Z")
Hello I'm having a hard time figuring out if the aggregate functions would be able to achieve what I'm trying to do. It's hard to explain in text format, but I created a simple image: https://imgur.com/a/YPXmrBI Basical… --- ## [High Heap Usage](https://discuss.elastic.co/t/high-heap-usage/249060)
**Author:** [@yasir\_khatri](https://discuss.elastic.co/u/yasir_khatri)\ **Replies:** 3\ **Last updated:** [September 21, 2020, 8:31pm UTC](https://discuss.elastic.co/t/high-heap-usage/249060 "2020-09-21T20:31:43Z")
We are using single node of elasticsearch v7.9.1 with 3.0 Tb of data. The rollover size is 10 gb 3 shards per index, that makes in 3.33 gb per shard. I need help troubleshooting these issues: The heap usage is always … --- ## [Index only and do not store in source](https://discuss.elastic.co/t/index-only-and-do-not-store-in-source/249283)
**Author:** [@Ankit\_Mangal](https://discuss.elastic.co/u/Ankit_Mangal)\ **Replies:** 2\ **Last updated:** [September 21, 2020, 7:31pm UTC](https://discuss.elastic.co/t/index-only-and-do-not-store-in-source/249283 "2020-09-21T19:31:37Z")
I was looking for a feature where i can pass first name, last name as plain string to Elasticsearch and just index these fields so i can search but do not store them in source (Due to PII data). Came across this copy\_to … --- ## [Continuous transform doesn't use checkpoint timestamp to filter search](https://discuss.elastic.co/t/continuous-transform-doesnt-use-checkpoint-timestamp-to-filter-search/249178)
**Author:** [@tbrown\_monarx](https://discuss.elastic.co/u/tbrown_monarx)\ **Replies:** 3\ **Last updated:** [September 21, 2020, 7:33pm UTC](https://discuss.elastic.co/t/continuous-transform-doesnt-use-checkpoint-timestamp-to-filter-search/249178 "2020-09-21T19:33:11Z")
On ES 7.7.1 I recently created a transform with an empty (match\_all) query that pivots with a group\_by including three terms and one histogram (not a date\_histogram which I understand has some issues with transforms). W… --- ## [Can't get AD ldap to work](https://discuss.elastic.co/t/cant-get-ad-ldap-to-work/249174)
**Author:** [@Jay9x](https://discuss.elastic.co/u/Jay9x)\ **Replies:** 9\ **Last updated:** [September 21, 2020, 7:26pm UTC](https://discuss.elastic.co/t/cant-get-ad-ldap-to-work/249174 "2020-09-21T19:26:01Z")
Keep getting this error in the elasticsearch.log. Authentication to realm ldap failed - authenticate failed (Caused by LDAPException(resultCode=49 (invalid credentials), diagnosticMessage='80090308: LdapErr: DSID-0C0904… --- ## [Pipeline Delay 7.9](https://discuss.elastic.co/t/pipeline-delay-7-9/249148)
**Author:** [@AleksandrN](https://discuss.elastic.co/u/AleksandrN)\ **Replies:** 10\ **Last updated:** [September 21, 2020, 5:44pm UTC](https://discuss.elastic.co/t/pipeline-delay-7-9/249148 "2020-09-21T17:44:51Z")
Hello everyone, I have such work bench: 3 nodes of Elastic 1 node of Logstash 1 node of Kibana 1 Winlogbeat agent The Winlogbeat agent ships logs from the Windows machine, that collecting logs from all of my company… --- ## [How to install ELK in a raspberry PI](https://discuss.elastic.co/t/how-to-install-elk-in-a-raspberry-pi/249281)
**Author:** [@javidr](https://discuss.elastic.co/u/javidr)\ **Replies:** 5\ **Last updated:** [September 21, 2020, 6:04pm UTC](https://discuss.elastic.co/t/how-to-install-elk-in-a-raspberry-pi/249281 "2020-09-21T18:04:18Z")
Hi Im trying to install ELK in a raspberry PI and i have little success by now. I have installed the package via apt-get install, and the java version is not compatible. I have downgraded java to jre8 and i have problem… --- ## [Master Elasticsearch node on same host as data node?](https://discuss.elastic.co/t/master-elasticsearch-node-on-same-host-as-data-node/249149)
**Author:** [@dshih](https://discuss.elastic.co/u/dshih)\ **Replies:** 5\ **Last updated:** [September 21, 2020, 6:03pm UTC](https://discuss.elastic.co/t/master-elasticsearch-node-on-same-host-as-data-node/249149 "2020-09-21T18:03:11Z")
Is there any benefit to enabling Elasticsearch master nodes if they will run on the same hosts as a data nodes? Background My company ships a product with a hosted (AWS-based) and on-premises offering. For hosted, we … --- ## [How to query for anomalous state changes based on duration](https://discuss.elastic.co/t/how-to-query-for-anomalous-state-changes-based-on-duration/248551)
**Author:** [@ehausig1](https://discuss.elastic.co/u/ehausig1)\ **Replies:** 2\ **Last updated:** [September 21, 2020, 5:44pm UTC](https://discuss.elastic.co/t/how-to-query-for-anomalous-state-changes-based-on-duration/248551 "2020-09-21T17:44:06Z")
I'm trying to determine if there is a way that I can query an Elasticsearch index for a particular data anomaly we sometimes encounter. Here is a contrived example of the dataset (with some annotations): \[ { "\_id… --- ## [ES-Hadoop compatibility with Spark 2.4.4](https://discuss.elastic.co/t/es-hadoop-compatibility-with-spark-2-4-4/249393)
**Author:** [@Jonas555](https://discuss.elastic.co/u/Jonas555)\ **Replies:** 0\ **Last updated:** [September 21, 2020, 4:08pm UTC](https://discuss.elastic.co/t/es-hadoop-compatibility-with-spark-2-4-4/249393 "2020-09-21T16:08:55Z")
Hello, We have an EMR v5.28 cluster that comes with Spark 2.4.4. Our understanding is that Elasticsearch-Hadoop is not compatible with Spark 2.4.4 since Spark is compiled with Scala 2.12, and ES-Hadoop requires Scala 2.… --- ## [Update values by curl cmd](https://discuss.elastic.co/t/update-values-by-curl-cmd/249386)
**Author:** [@Cyril\_Guiet](https://discuss.elastic.co/u/Cyril_Guiet)\ **Replies:** 2\ **Last updated:** [September 21, 2020, 3:59pm UTC](https://discuss.elastic.co/t/update-values-by-curl-cmd/249386 "2020-09-21T15:59:18Z")
Hello ! Can I update the "instanceof" values by there equivalents labels ( municipalities and cities of Serbia == Q783930 ) by curl API ? Thanks :slight\_smile: returned answer sample: { "\_index" : "enwiki\_latest", … --- ## [Big logs files with TFS elasticsearch](https://discuss.elastic.co/t/big-logs-files-with-tfs-elasticsearch/248806)
**Author:** [@Freud](https://discuss.elastic.co/u/Freud)\ **Replies:** 6\ **Last updated:** [September 21, 2020, 2:54pm UTC](https://discuss.elastic.co/t/big-logs-files-with-tfs-elasticsearch/248806 "2020-09-21T14:54:55Z")
Hi, We have a TFS 2018 and since a few days elasticsearch create 5Go logs file every days. I don't know what is the matter with it. The log contain 2 type of WARN "cluster.action.shard" and "indices.cluster" Can some… --- ## [How to add two new fields to a document](https://discuss.elastic.co/t/how-to-add-two-new-fields-to-a-document/249259)
**Author:** [@Rui\_Goncalves](https://discuss.elastic.co/u/Rui_Goncalves)\ **Replies:** 2\ **Last updated:** [September 21, 2020, 2:16pm UTC](https://discuss.elastic.co/t/how-to-add-two-new-fields-to-a-document/249259 "2020-09-21T14:16:37Z")
Hi, I'm trying to add two new fields to a document: all2\_business\_index.car, car being an array. The code below throws a null\_pointer\_exception How canI fix this, thank you Blockquote "script\_stack": \[ … --- ## [Precise distinct count](https://discuss.elastic.co/t/precise-distinct-count/249374)
**Author:** [@ion\_utale](https://discuss.elastic.co/u/ion_utale)\ **Replies:** 0\ **Last updated:** [September 21, 2020, 2:02pm UTC](https://discuss.elastic.co/t/precise-distinct-count/249374 "2020-09-21T14:02:29Z")
Hi, i am just frustrated that i am unable to do an distinct count. on 6.8 Cardinality is just an approximation. I have over 20k buckets and cardinaly with precision\_threshold = 40000 is still retuning me wrong value… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=802) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=804)