# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1628 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1629 --- ## [Filter with millions of record](https://discuss.elastic.co/t/filter-with-millions-of-record/134561)
**Author:** [@chaudharys31](https://discuss.elastic.co/u/chaudharys31)\ **Replies:** 43\ **Last updated:** [July 5, 2018, 3:00pm UTC](https://discuss.elastic.co/t/filter-with-millions-of-record/134561 "2018-07-05T15:00:40Z")
Can someone explain term filter or any other filter work with millions or 100 K records in the filter? How will be performed and how it will work? --- ## [Filter Aggeration like SQL](https://discuss.elastic.co/t/filter-aggeration-like-sql/138745)
**Author:** [@Puroo\_Jain](https://discuss.elastic.co/u/Puroo_Jain)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 2:16pm UTC](https://discuss.elastic.co/t/filter-aggeration-like-sql/138745 "2018-07-05T14:16:53Z")
How we can filter aggeration value Like we have below SQL query, How we can write that in Elastic SELECT '0-10' AS range, COUNT(AssetId) FROM (SELECT AssetId, SUM(SumOfPowerOffDuration) / 3100 AS Duration from AssetEv… --- ## [Best data type for a big string full of numbers](https://discuss.elastic.co/t/best-data-type-for-a-big-string-full-of-numbers/138741)
**Author:** [@davidediak](https://discuss.elastic.co/u/davidediak)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 2:09pm UTC](https://discuss.elastic.co/t/best-data-type-for-a-big-string-full-of-numbers/138741 "2018-07-05T14:09:54Z")
Hi guys, my docs have a field (called "digits") which is a big string full of numbers separated by space e.g. digits:"123 321 5236243 ...". Each doc have 500k of these digits. I want these searchable, with an exact mat… --- ## [Import from BigQuery to Elasticsearch?](https://discuss.elastic.co/t/import-from-bigquery-to-elasticsearch/138448)
**Author:** [@melissachang](https://discuss.elastic.co/u/melissachang)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 1:43pm UTC](https://discuss.elastic.co/t/import-from-bigquery-to-elasticsearch/138448 "2018-07-05T13:43:59Z")
Is there a recommended/easy way to import data from BigQuery into Elasticsearch? Right now I'm doing BigQuery -\> pandas dataframe -\> dict -\> Elasticsearch, but pandas doesn't handle missing values well. --- ## [Elasticsearch 5.5 - Deleted document count above 50%, and low disk watermark consistently hit](https://discuss.elastic.co/t/elasticsearch-5-5-deleted-document-count-above-50-and-low-disk-watermark-consistently-hit/138596)
**Author:** [@dendle](https://discuss.elastic.co/u/dendle)\ **Replies:** 4\ **Last updated:** [July 5, 2018, 1:32pm UTC](https://discuss.elastic.co/t/elasticsearch-5-5-deleted-document-count-above-50-and-low-disk-watermark-consistently-hit/138596 "2018-07-05T13:32:54Z")
Hello All, I believe I have a storage issue with regards to deleted documents. I was hoping to leave elasticsearch/lucene do its thing and merge out deletes, but i'm not sure why this isnt happening. Its only a proble… --- ## [Weird issue with date sort](https://discuss.elastic.co/t/weird-issue-with-date-sort/137646)
**Author:** [@nicolas-f](https://discuss.elastic.co/u/nicolas-f)\ **Replies:** 6\ **Last updated:** [July 5, 2018, 1:27pm UTC](https://discuss.elastic.co/t/weird-issue-with-date-sort/137646 "2018-07-05T13:27:12Z")
Hi, I have an issue with date field. With this queries: DELETE ds3 PUT ds3 { "mappings":{ "ds2":{ "properties":{ "producerID":{ "type":"keyword" }, … --- ## [Elasticsearch snapshot\_missing\_exception in fetching all snapshots](https://discuss.elastic.co/t/elasticsearch-snapshot-missing-exception-in-fetching-all-snapshots/138722)
**Author:** [@vaidabhishek](https://discuss.elastic.co/u/vaidabhishek)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 1:19pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-missing-exception-in-fetching-all-snapshots/138722 "2018-07-05T13:19:39Z")
I accidentally deleted folder from S3 directory where ES was storing its snapshots. There are still valid snapshots and their folders but running a command GET /\_cat/snapshots/elk?v&s=id gives back: { "error": { … --- ## [java.lang.OutOfMemoryError: Java heap space - GC overhead using visualizations](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-gc-overhead-using-visualizations/138660)
**Author:** [@akapit](https://discuss.elastic.co/u/akapit)\ **Replies:** 15\ **Last updated:** [July 5, 2018, 10:39am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-gc-overhead-using-visualizations/138660 "2018-07-05T10:39:51Z")
I'm running Elastic and kibana on a clean linode instance with 8 GB RAM. Only imported a csv file through logstash into Elasticsearch in a index. Plain text fields with two Dates and some numeric ones. About the index,… --- ## [Web based application](https://discuss.elastic.co/t/web-based-application/138682)
**Author:** [@mkaczor](https://discuss.elastic.co/u/mkaczor)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 12:42pm UTC](https://discuss.elastic.co/t/web-based-application/138682 "2018-07-05T12:42:38Z")
Hello, I did an index in JSON with elasticsearch, (no problem with this). But I need do an web based application and I can't understand how I need to do that. Thank you. --- ## [Is there any limit on Multi search API and Index?](https://discuss.elastic.co/t/is-there-any-limit-on-multi-search-api-and-index/138714)
**Author:** [@rs123](https://discuss.elastic.co/u/rs123)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 12:28pm UTC](https://discuss.elastic.co/t/is-there-any-limit-on-multi-search-api-and-index/138714 "2018-07-05T12:28:15Z")
Is there any limit in multi search API, like at a time in how many indexes I can search? Is there any limit on creating indexes with basic licence, i.e How many indexes can I create for my data with basic licence? --- ## [Ignore TF/IDF in a complex query](https://discuss.elastic.co/t/ignore-tf-idf-in-a-complex-query/137446)
**Author:** [@LONGMAN](https://discuss.elastic.co/u/LONGMAN)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 12:20pm UTC](https://discuss.elastic.co/t/ignore-tf-idf-in-a-complex-query/137446 "2018-07-05T12:20:28Z")
I want to ignore TF/IDF in a complex query. I tried to create custom similarity script script, but no result. Also, this article is useless for me: https://www.elastic.co/guide/en/elasticsearch/guide/master/ignoring-tf… --- ## [The time difference is nine hours](https://discuss.elastic.co/t/the-time-difference-is-nine-hours/138699)
**Author:** [@suhan\_kim](https://discuss.elastic.co/u/suhan_kim)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 11:02am UTC](https://discuss.elastic.co/t/the-time-difference-is-nine-hours/138699 "2018-07-05T11:02:35Z")
hello, The time difference is nine hours. What is the reason? The system is not a reason. --- ## [Cloud and Ingest node/logstash and snmptraps](https://discuss.elastic.co/t/cloud-and-ingest-node-logstash-and-snmptraps/138696)
**Author:** [@agonzalez](https://discuss.elastic.co/u/agonzalez)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 10:48am UTC](https://discuss.elastic.co/t/cloud-and-ingest-node-logstash-and-snmptraps/138696 "2018-07-05T10:48:22Z")
I will need to capture snmp traps in elastic stack cloud instance and as it cant run logstash in cloud is there any way to run logstash in your cloud instances or use elasticsearch ingest node to get snmptraps? --- ## [Userdefined Roles and User](https://discuss.elastic.co/t/userdefined-roles-and-user/138548)
**Author:** [@Chandana](https://discuss.elastic.co/u/Chandana)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 10:35am UTC](https://discuss.elastic.co/t/userdefined-roles-and-user/138548 "2018-07-05T10:35:19Z")
I am using ELK 6.2.4, installed x-pack in all the 3 on windows machine. I tried creating users and roles. Once I created where do they get saved? Can I see those user and the user defined roles in any file? I don't see… --- ## [What Hardware do I need for 100 GB Per day data volume?](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 9:53am UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397 "2018-07-05T09:53:42Z")
I am setting up a cluster and i have following scenario Data Volume 100GB per day (with 3 month retention ) Search scenarios (dashboards only mix, of analyze and not analyzed fields max 10 users) I am going to use 8 se… --- ## [Elastic warm node heap unexpected behavior](https://discuss.elastic.co/t/elastic-warm-node-heap-unexpected-behavior/138680)
**Author:** [@farin99](https://discuss.elastic.co/u/farin99)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 9:49am UTC](https://discuss.elastic.co/t/elastic-warm-node-heap-unexpected-behavior/138680 "2018-07-05T09:49:15Z")
Hi, One of our elasticsearch clusters is showing an unexpected behavior on the heap of its warm nodes. The cluster consists of 40 hot data nodes (8 cores, 64gb ram out of which 30.5gb heap and 1.5tb of local SSD storage… --- ## [I have an issue in installing x-pack the following text appears whenever I try to install it](https://discuss.elastic.co/t/i-have-an-issue-in-installing-x-pack-the-following-text-appears-whenever-i-try-to-install-it/138594)
**Author:** [@Ibra](https://discuss.elastic.co/u/Ibra)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 9:03am UTC](https://discuss.elastic.co/t/i-have-an-issue-in-installing-x-pack-the-following-text-appears-whenever-i-try-to-install-it/138594 "2018-07-05T09:03:44Z")
\-\> Installing x-pack/... Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/x-pack/2.3.1/x-pack-2.3.1.zip ... ERROR: failed to download out of all possible locations..., use --verbose to … --- ## [URGENT: PLEASE HELP Unique list of bigram and trigram combinations then aggregate stats](https://discuss.elastic.co/t/urgent-please-help-unique-list-of-bigram-and-trigram-combinations-then-aggregate-stats/137817)
**Author:** [@Lior\_Krolewicz](https://discuss.elastic.co/u/Lior_Krolewicz)\ **Replies:** 5\ **Last updated:** [July 5, 2018, 8:42am UTC](https://discuss.elastic.co/t/urgent-please-help-unique-list-of-bigram-and-trigram-combinations-then-aggregate-stats/137817 "2018-07-05T08:42:49Z")
I'm currently solving this poorly, not leveraging es. We recently scaled our data and my system crashes. I need an efficient solution to get back online. Thank you! I have a list of 300,000 unique strings with different… --- ## [Running 2 node elasticsearch with docker compose yml file on a single machine](https://discuss.elastic.co/t/running-2-node-elasticsearch-with-docker-compose-yml-file-on-a-single-machine/138668)
**Author:** [@vinaypn](https://discuss.elastic.co/u/vinaypn)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 8:36am UTC](https://discuss.elastic.co/t/running-2-node-elasticsearch-with-docker-compose-yml-file-on-a-single-machine/138668 "2018-07-05T08:36:20Z")
Hi Team, i am trying to run 2 node elasticsearch with a docker compose file. My docker-compose file looks like this: version: '2' services: elasticsearch: image: elasticsearch volumes: - es\_data1:/usr/share/elast… --- ## [Es 5.6.x force merege question about max\_num\_segments/max\_merge\_count/max\_thread\_count](https://discuss.elastic.co/t/es-5-6-x-force-merege-question-about-max-num-segments-max-merge-count-max-thread-count/138298)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 6\ **Last updated:** [July 5, 2018, 8:15am UTC](https://discuss.elastic.co/t/es-5-6-x-force-merege-question-about-max-num-segments-max-merge-count-max-thread-count/138298 "2018-07-05T08:15:53Z")
i see the doc have setting with the indices about force merege index.merge.scheduler.max\_merge\_count index.merge.scheduler.max\_thread\_count with not ssd the docs give index.merge.scheduler.max\_thread\_count to "1" … --- ## [Elastic Benefits of using a single index or multiple index according to a particular criteria(say gender)](https://discuss.elastic.co/t/elastic-benefits-of-using-a-single-index-or-multiple-index-according-to-a-particular-criteria-say-gender/138592)
**Author:** [@Ayush\_Nenawati](https://discuss.elastic.co/u/Ayush_Nenawati)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 7:49am UTC](https://discuss.elastic.co/t/elastic-benefits-of-using-a-single-index-or-multiple-index-according-to-a-particular-criteria-say-gender/138592 "2018-07-05T07:49:35Z")
What is the query performance difference in querying to an index with all user data and querying to a particular index with half of the data? For instance if i have two indexes male and female. How much query improvemen… --- ## [Insert data successfully, but i can not find it](https://discuss.elastic.co/t/insert-data-successfully-but-i-can-not-find-it/138466)
**Author:** [@benhairui](https://discuss.elastic.co/u/benhairui)\ **Replies:** 9\ **Last updated:** [July 5, 2018, 7:29am UTC](https://discuss.elastic.co/t/insert-data-successfully-but-i-can-not-find-it/138466 "2018-07-05T07:29:33Z")
I insert data into es successfully, but when i add two fields, the return is successful, but when i search it with id, i find it without the two fields added. Help me please? thank you! --- ## [Expired https certificate elasticsearch](https://discuss.elastic.co/t/expired-https-certificate-elasticsearch/138633)
**Author:** [@fhalde](https://discuss.elastic.co/u/fhalde)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 7:14am UTC](https://discuss.elastic.co/t/expired-https-certificate-elasticsearch/138633 "2018-07-05T07:14:53Z")
wondering what will happen to my elasticsearch cluster if https certificate gets expired \[it's used for transport and https\] --- ## [Watcher\_help](https://discuss.elastic.co/t/watcher-help/138476)
**Author:** [@newlearner](https://discuss.elastic.co/u/newlearner)\ **Replies:** 3\ **Last updated:** [July 5, 2018, 7:05am UTC](https://discuss.elastic.co/t/watcher-help/138476 "2018-07-05T07:05:33Z")
As I am new to elastic and wacther. Is the below watcher is correct to trigger for every 15 min for cduration \>= 300000 { "trigger": { "schedule": { "interval": "15m" } }, "input": { "search": { "request": { "… --- ## [Can I move disk from node to another with data?](https://discuss.elastic.co/t/can-i-move-disk-from-node-to-another-with-data/138626)
**Author:** [@yossib](https://discuss.elastic.co/u/yossib)\ **Replies:** 4\ **Last updated:** [July 5, 2018, 6:29am UTC](https://discuss.elastic.co/t/can-i-move-disk-from-node-to-another-with-data/138626 "2018-07-05T06:29:14Z")
hi, anyone knows what will happen if I wrote to a disk on node A, then node A fails, then I move the disk to another node B (new member of the cluster or existing member)... will I be able to recover this data? the 3 n… --- ## [How to output the .kibana index all docs with curl? es 5.x](https://discuss.elastic.co/t/how-to-output-the-kibana-index-all-docs-with-curl-es-5-x/138619)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 2\ **Last updated:** [July 5, 2018, 5:39am UTC](https://discuss.elastic.co/t/how-to-output-the-kibana-index-all-docs-with-curl-es-5-x/138619 "2018-07-05T05:39:42Z")
curl -XGET http://xxx.xxx:9200/.kibana/????? i need see all the data in .kibana like with elasticsearch-head plugins output. --- ## [Elasticsearch down after server backup](https://discuss.elastic.co/t/elasticsearch-down-after-server-backup/138614)
**Author:** [@sadeshdrj](https://discuss.elastic.co/u/sadeshdrj)\ **Replies:** 4\ **Last updated:** [July 5, 2018, 2:55am UTC](https://discuss.elastic.co/t/elasticsearch-down-after-server-backup/138614 "2018-07-05T02:55:38Z")
later i checked in my elasticsearch logs it shows the service has been shutdown, when i run systemctl start elasticsearch.serivce and later systemctl status elasticsearch.serivice, it shows active failed. Jul 05 10:21:… --- ## [How to format a date as a string in Painless](https://discuss.elastic.co/t/how-to-format-a-date-as-a-string-in-painless/138618)
**Author:** [@Steven\_Ensslen](https://discuss.elastic.co/u/Steven_Ensslen)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 2:45am UTC](https://discuss.elastic.co/t/how-to-format-a-date-as-a-string-in-painless/138618 "2018-07-05T02:45:38Z")
Continuing the discussion from Elasticsearch 6.2 - how to format date (epoch) field in yyyymm string in painless inline scripting: Despite the frequent mentions of switching to java.time in the docs, my cloud.elastic.co … --- ## [X-Pack basic license and security](https://discuss.elastic.co/t/x-pack-basic-license-and-security/135331)
**Author:** [@guy.braeckmans](https://discuss.elastic.co/u/guy.braeckmans)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 10:27am UTC](https://discuss.elastic.co/t/x-pack-basic-license-and-security/135331 "2018-06-11T10:27:43Z")
Hi, I installed Elastic, Kibana en filebeats + winlogbeats successfully on a Windows server. All is working fine. It was clear that we needed some security for this. I therefore installed x-pack. The install worked fine… --- ## [Choosing the right heap size for warm data nodes](https://discuss.elastic.co/t/choosing-the-right-heap-size-for-warm-data-nodes/136341)
**Author:** [@iamredlus](https://discuss.elastic.co/u/iamredlus)\ **Replies:** 10\ **Last updated:** [July 4, 2018, 11:35pm UTC](https://discuss.elastic.co/t/choosing-the-right-heap-size-for-warm-data-nodes/136341 "2018-07-04T23:35:50Z")
Hi One of our production clusters has 40 hot data nodes (8 cores, 64gb ram out of which 30.5gb heap and 1.5tb of local SSD storage), and 30 warm data nodes (8 cores, 32gb ram out of which 16gb heap and 5tb of attached H… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1627) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1629)