# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=610 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 611 --- ## [Fetch and store data from an index every hour](https://discuss.elastic.co/t/fetch-and-store-data-from-an-index-every-hour/278332)
**Author:** [@mrsac7](https://discuss.elastic.co/u/mrsac7)\ **Replies:** 2\ **Last updated:** [July 13, 2021, 7:11am UTC](https://discuss.elastic.co/t/fetch-and-store-data-from-an-index-every-hour/278332 "2021-07-13T07:11:01Z")
Dear community, I want to create a script that can - Fetch data from a given source index Process the data and store into a destination index The script will be scheduled to run once every hour. The data contains t… --- ## [Questions about query latency](https://discuss.elastic.co/t/questions-about-query-latency/278483)
**Author:** [@yeziblo](https://discuss.elastic.co/u/yeziblo)\ **Replies:** 4\ **Last updated:** [July 13, 2021, 6:13am UTC](https://discuss.elastic.co/t/questions-about-query-latency/278483 "2021-07-13T06:13:38Z")
Hi everyone. I have a strange problem. We have an ES cluster with three nodes, each node's heap memory is 8 GB and the es version is 7.8. Now I have two indexes: test@20210711125233 and test@20210712075544. There mapp… --- ## [Elasticsearch GC timeout on data node](https://discuss.elastic.co/t/elasticsearch-gc-timeout-on-data-node/277943)
**Author:** [@atakjudin](https://discuss.elastic.co/u/atakjudin)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 4:50am UTC](https://discuss.elastic.co/t/elasticsearch-gc-timeout-on-data-node/277943 "2021-07-13T04:50:54Z")
Hello, I try to find out an answer to my problem but I don't find a solution in existing threads I have read. I have a cluster of 7 elasticsearch nodes (2 master, 4 ingest, all data). The latest node was added two week… --- ## [The problems of memory of ES7.12](https://discuss.elastic.co/t/the-problems-of-memory-of-es7-12/278030)
**Author:** [@lichuandev](https://discuss.elastic.co/u/lichuandev)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 4:42am UTC](https://discuss.elastic.co/t/the-problems-of-memory-of-es7-12/278030 "2021-07-13T04:42:49Z")
After I upgraded to version 7.13 and used the default configuration, things got better, but I still had to restart it once a week.Version 7.4 used previously did not have this problem.Is there any good solution? Thank y… --- ## [Network Management System with ELK (Like Open NMS etc)](https://discuss.elastic.co/t/network-management-system-with-elk-like-open-nms-etc/278485)
**Author:** [@mohammadawaisjavaid](https://discuss.elastic.co/u/mohammadawaisjavaid)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 4:41am UTC](https://discuss.elastic.co/t/network-management-system-with-elk-like-open-nms-etc/278485 "2021-07-13T04:41:27Z")
Hi all, I have configured 7.10.0 Stack Management. Now I wished to see the devices attached to my network in Elasticsearch. In this regard I am searching for a integration that helps me visualize the whole network in KI… --- ## [Search After NEST - ElasticSearch](https://discuss.elastic.co/t/search-after-nest-elasticsearch/278478)
**Author:** [@NoviceESCoder](https://discuss.elastic.co/u/NoviceESCoder)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 12:06am UTC](https://discuss.elastic.co/t/search-after-nest-elasticsearch/278478 "2021-07-13T00:06:20Z")
After realizing that the search api in ElasticSearch will only handle up to 10,000 results, my next option to retrieve all the logs (more than 10,000 results) was through creating a point in time id and passing that id t… --- ## [Where do we define the password for cert generated using elastic certutil in yml file?](https://discuss.elastic.co/t/where-do-we-define-the-password-for-cert-generated-using-elastic-certutil-in-yml-file/278448)
**Author:** [@devashishsingh](https://discuss.elastic.co/u/devashishsingh)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 11:55pm UTC](https://discuss.elastic.co/t/where-do-we-define-the-password-for-cert-generated-using-elastic-certutil-in-yml-file/278448 "2021-07-12T23:55:18Z")
Hello, In order to enable SSL security, I did follow steps and generated the certificate first. While generating it asked for a password(to create one for the cert protection) Please enter the desired output file \[elas… --- ## [Data nodes are connected to the master but not to each other during upgrade](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472)
**Author:** [@163satish](https://discuss.elastic.co/u/163satish)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 10:35pm UTC](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472 "2021-07-12T22:35:13Z")
I am doing ES upgrade from 6.8.2 to 7.9.3 and seeing issues with 2 data nodes. Shard movements are going from "index" to "init" stage and logs show that the data nodes are not connected to each other. On the new node I … --- ## [Is there an API that will return how many logs are there](https://discuss.elastic.co/t/is-there-an-api-that-will-return-how-many-logs-are-there/278453)
**Author:** [@NoviceESCoder](https://discuss.elastic.co/u/NoviceESCoder)\ **Replies:** 2\ **Last updated:** [July 12, 2021, 8:18pm UTC](https://discuss.elastic.co/t/is-there-an-api-that-will-return-how-many-logs-are-there/278453 "2021-07-12T20:18:08Z")
Is there an api that returns how many results/logs there are? or like a built in feature with NEST that will tell you how many results there is like i.e. count? Because I have a NEST query search that will do the followi… --- ## [Split a field for a specific pattern](https://discuss.elastic.co/t/split-a-field-for-a-specific-pattern/278465)
**Author:** [@mvasqueznr](https://discuss.elastic.co/u/mvasqueznr)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 7:26pm UTC](https://discuss.elastic.co/t/split-a-field-for-a-specific-pattern/278465 "2021-07-12T19:26:02Z")
Hi. I have a log like this: 12:32:46 2021/06/28 ZBXTRAP 192.168.50.135 PDU INFO: notificationtype INFORM version 1 receivedfrom UDP: \[192.168.50.135\]:10756-\>\[… --- ## [Is it possible for ES to drop every field that's not in mapping upon indexing](https://discuss.elastic.co/t/is-it-possible-for-es-to-drop-every-field-thats-not-in-mapping-upon-indexing/278449)
**Author:** [@R.L](https://discuss.elastic.co/u/R.L)\ **Replies:** 4\ **Last updated:** [July 12, 2021, 6:58pm UTC](https://discuss.elastic.co/t/is-it-possible-for-es-to-drop-every-field-thats-not-in-mapping-upon-indexing/278449 "2021-07-12T18:58:52Z")
Let's say I have a index template mapping which has fields: { "first\_name"..., "last\_name"..., } and someone is uploading docs with fields: { "first\_name": "jon", "last\_name": "connington", "address": "blah… --- ## [Max heap size in nodes elasticsearch](https://discuss.elastic.co/t/max-heap-size-in-nodes-elasticsearch/278444)
**Author:** [@Oskr](https://discuss.elastic.co/u/Oskr)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 5:41pm UTC](https://discuss.elastic.co/t/max-heap-size-in-nodes-elasticsearch/278444 "2021-07-12T17:41:41Z")
Hi, I have a 3-node architecture with the following roles: "roles" : \[ "ingest", "master", "data", "ml" \] currently each server has 32GB of memory, which means that my nodes have 16GB for elastic. In the last few… --- ## [Metricbeat http module nested json](https://discuss.elastic.co/t/metricbeat-http-module-nested-json/278452)
**Author:** [@creativeguitar](https://discuss.elastic.co/u/creativeguitar)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 6:00pm UTC](https://discuss.elastic.co/t/metricbeat-http-module-nested-json/278452 "2021-07-12T18:00:15Z")
I need help regarding elastic search mapping of nested json response. I am using the meatricbeat's http module to query data from our asset management system and I am getting the following response in return "fields": \[… --- ## [Can't reassign shards, all shards currently on same node](https://discuss.elastic.co/t/cant-reassign-shards-all-shards-currently-on-same-node/277794)
**Author:** [@tomx1](https://discuss.elastic.co/u/tomx1)\ **Replies:** 6\ **Last updated:** [July 12, 2021, 5:38pm UTC](https://discuss.elastic.co/t/cant-reassign-shards-all-shards-currently-on-same-node/277794 "2021-07-12T17:38:27Z")
We are maintaining a 10 node cluster (5 hot nodes with SSD storage and 5 warm nodes with SAS storage, latest Elasticsearch version by the way). The shard allocation for a specific index like looks this: GET \_cat/shards… --- ## [Retrieving all records from query search without having to use .size -ElasticsSearch](https://discuss.elastic.co/t/retrieving-all-records-from-query-search-without-having-to-use-size-elasticssearch/278440)
**Author:** [@NoviceESCoder](https://discuss.elastic.co/u/NoviceESCoder)\ **Replies:** 5\ **Last updated:** [July 12, 2021, 4:53pm UTC](https://discuss.elastic.co/t/retrieving-all-records-from-query-search-without-having-to-use-size-elasticssearch/278440 "2021-07-12T16:53:33Z")
I noticed that when running the following dsl query search on the cli console for ElasticSearch that it will only return a few of the records. If I add a "size": 3000, then I will be able to retrieve more than I original… --- ## [Large number of queued refresh threads on a single node, queue count keeps increasing](https://discuss.elastic.co/t/large-number-of-queued-refresh-threads-on-a-single-node-queue-count-keeps-increasing/278435)
**Author:** [@BradVido](https://discuss.elastic.co/u/BradVido)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 2:41pm UTC](https://discuss.elastic.co/t/large-number-of-queued-refresh-threads-on-a-single-node-queue-count-keeps-increasing/278435 "2021-07-12T14:41:48Z")
I have a 3 node cluster on ES 7.8.1 Once in a while, it seems that the refresh queue gets "stuck" on a node (not always the same node), and the queued number of refresh threads keeps increasing, while the active number … --- ## [The new node cannot add to the cluster but creating a separate new cluster instead](https://discuss.elastic.co/t/the-new-node-cannot-add-to-the-cluster-but-creating-a-separate-new-cluster-instead/278373)
**Author:** [@Zongxian\_Feng](https://discuss.elastic.co/u/Zongxian_Feng)\ **Replies:** 5\ **Last updated:** [July 12, 2021, 2:41pm UTC](https://discuss.elastic.co/t/the-new-node-cannot-add-to-the-cluster-but-creating-a-separate-new-cluster-instead/278373 "2021-07-12T14:41:31Z")
Hi. I am trying to add a new node to an existing cluster(only has one node in it). After configuring in .yml and running the elasticsearch on both nodes, the new node creates a new cluster instead of joining the original… --- ## [Query string boolean operators vs. stop words](https://discuss.elastic.co/t/query-string-boolean-operators-vs-stop-words/276535)
**Author:** [@zsonglorman](https://discuss.elastic.co/u/zsonglorman)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 2:34pm UTC](https://discuss.elastic.co/t/query-string-boolean-operators-vs-stop-words/276535 "2021-07-12T14:34:15Z")
Hello, I'm using a query string query with boolean operators as stated in the docs, like this: ((quick AND fox) OR (brown AND fox) OR fox) AND NOT news This works all right, but I noticed that stop words are not ignore… --- ## [How to update elastic search \_source settings?](https://discuss.elastic.co/t/how-to-update-elastic-search-source-settings/278431)
**Author:** [@MarvinLiu](https://discuss.elastic.co/u/MarvinLiu)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 1:45pm UTC](https://discuss.elastic.co/t/how-to-update-elastic-search-source-settings/278431 "2021-07-12T13:45:01Z")
When I try to update the \_source parameter in elastic search,I got an error. Can I change this settings? PUT source-test POST source-test/\_mappings { "\_source" : { "enabled":true } } POST source-test/\_mapping… --- ## [Boost two related fields in a search query](https://discuss.elastic.co/t/boost-two-related-fields-in-a-search-query/278428)
**Author:** [@Antojvlt](https://discuss.elastic.co/u/Antojvlt)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 12:45pm UTC](https://discuss.elastic.co/t/boost-two-related-fields-in-a-search-query/278428 "2021-07-12T12:45:57Z")
Hi, I am new to Elasticsearch and I encounter a difficultly to make my search query to work as expected. I want to search on documents that contain two field: "review\_count" : { "type" : "long" }, "s… --- ## [How to make stopwords path text file work with words that have single quotes](https://discuss.elastic.co/t/how-to-make-stopwords-path-text-file-work-with-words-that-have-single-quotes/278427)
**Author:** [@Zubair\_Ali](https://discuss.elastic.co/u/Zubair_Ali)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 12:34pm UTC](https://discuss.elastic.co/t/how-to-make-stopwords-path-text-file-work-with-words-that-have-single-quotes/278427 "2021-07-12T12:34:12Z")
I have the following stopwords filter: "english\_stop": { "type": "stop", "stopwords\_path": "/usr/local/etc/elasticsearch/stopwords.txt", "updateable": true } These are the words in my st… --- ## [Use predicate expression with NEST in .NET 5](https://discuss.elastic.co/t/use-predicate-expression-with-nest-in-net-5/278420)
**Author:** [@Henrique\_Mauri](https://discuss.elastic.co/u/Henrique_Mauri)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 11:50am UTC](https://discuss.elastic.co/t/use-predicate-expression-with-nest-in-net-5/278420 "2021-07-12T11:50:26Z")
Hi Is it possible to configure in NEST (.NET 5) a way to use a default predicate expression (Default Query Filter) for all NEST queries (GetAsync, GetManyAsync, SearchAsync, ExistsAsync, ...) performed on any index? Th… --- ## [Yum repo doesn't work](https://discuss.elastic.co/t/yum-repo-doesnt-work/278404)
**Author:** [@loposk](https://discuss.elastic.co/u/loposk)\ **Replies:** 2\ **Last updated:** [July 12, 2021, 11:41am UTC](https://discuss.elastic.co/t/yum-repo-doesnt-work/278404 "2021-07-12T11:41:05Z")
Added a repository according to the instructions Install Elasticsearch with RPM | Elasticsearch Guide \[7.13\] | Elastic When yum trying download https://artifacts.elastic.co/packages/7.x/yum/repodata/filelists.xml.gz I g… --- ## [Elasticsearch query DSL as a filter on Kibana](https://discuss.elastic.co/t/elasticsearch-query-dsl-as-a-filter-on-kibana/278415)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 11:16am UTC](https://discuss.elastic.co/t/elasticsearch-query-dsl-as-a-filter-on-kibana/278415 "2021-07-12T11:16:29Z")
Hi guys, i need help with - my challenge is that i have to generate a new field when the query match the cretiria. see the code below: Here I need to add/create a field (references) and fill it with the same value I … --- ## [Value\_count aggregation based on key name pattern](https://discuss.elastic.co/t/value-count-aggregation-based-on-key-name-pattern/278402)
**Author:** [@Ray\_Garcia](https://discuss.elastic.co/u/Ray_Garcia)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 9:21am UTC](https://discuss.elastic.co/t/value-count-aggregation-based-on-key-name-pattern/278402 "2021-07-12T09:21:44Z")
Hi all, Elastic newbie here. I just started playing with ES a week ago. It seems super powerful and it fits perfectly with some aggregated data queries I'm performing. Some of them are already done (and they work flawle… --- ## [Lowercase processor - \[field\] property isn't a string, but of type \[java.util.ArrayList\]](https://discuss.elastic.co/t/lowercase-processor-field-property-isnt-a-string-but-of-type-java-util-arraylist/278340)
**Author:** [@f4d0](https://discuss.elastic.co/u/f4d0)\ **Replies:** 2\ **Last updated:** [July 12, 2021, 9:04am UTC](https://discuss.elastic.co/t/lowercase-processor-field-property-isnt-a-string-but-of-type-java-util-arraylist/278340 "2021-07-12T09:04:11Z")
Hi, Elasticsearch version: 7.13.3 Python module in use: elasticsearch-7.13.3 Pipeline: B\_AutoReply\_pipeline = """ { "processors":\[ { "date": { "field": "Parsed\_Datetime", … --- ## [How to check document consistent with current mapping?](https://discuss.elastic.co/t/how-to-check-document-consistent-with-current-mapping/278387)
**Author:** [@peter5](https://discuss.elastic.co/u/peter5)\ **Replies:** 6\ **Last updated:** [July 12, 2021, 9:00am UTC](https://discuss.elastic.co/t/how-to-check-document-consistent-with-current-mapping/278387 "2021-07-12T09:00:28Z")
i need to know if document is matching current mapping or not (if not match i need to reindex) . eg, i change a field analyzer several times by update mapping . i need to know current document index status is identical … --- ## [Clustering Across Multiple Data Centers](https://discuss.elastic.co/t/clustering-across-multiple-data-centers/278367)
**Author:** [@otoce](https://discuss.elastic.co/u/otoce)\ **Replies:** 2\ **Last updated:** [July 12, 2021, 8:54am UTC](https://discuss.elastic.co/t/clustering-across-multiple-data-centers/278367 "2021-07-12T08:54:47Z")
Hi, thanks again for helping us on Warm nodes on spinning disks not stable, falling out of cluster during recovery. Cluster is working fine since than (8 months). For last 5 months there are three more servers. Total: 3… --- ## [Necessary to index a keyword field for exists query?](https://discuss.elastic.co/t/necessary-to-index-a-keyword-field-for-exists-query/278294)
**Author:** [@abhinavkulkarni](https://discuss.elastic.co/u/abhinavkulkarni)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 8:46am UTC](https://discuss.elastic.co/t/necessary-to-index-a-keyword-field-for-exists-query/278294 "2021-07-12T08:46:26Z")
Hi, Let us say, I have a User schema - name, address, bio and website. The website field is a keyword, but potentially unbounded - that is, the total number of all possible unique websites is limitless. I am interested… --- ## [Node killed after GC doing a Pause Full (G1 Evacuation Pause)](https://discuss.elastic.co/t/node-killed-after-gc-doing-a-pause-full-g1-evacuation-pause/278256)
**Author:** [@Vdm88](https://discuss.elastic.co/u/Vdm88)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 8:41am UTC](https://discuss.elastic.co/t/node-killed-after-gc-doing-a-pause-full-g1-evacuation-pause/278256 "2021-07-12T08:41:48Z")
Hello, We have noticed this. After the GC is doing the Pause Full (G1 Evacuation Pause). The memory is released but the node can not get back in the cluster without a ES Service restart. THE Server replied on the 9200 p… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=609) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=611)