# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1427 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1428 --- ## [Elasticsearch nodes doing young generation gc very frequently](https://discuss.elastic.co/t/elasticsearch-nodes-doing-young-generation-gc-very-frequently/163661)
**Author:** [@Nitish\_Goyal](https://discuss.elastic.co/u/Nitish_Goyal)\ **Replies:** 6\ **Last updated:** [January 14, 2019, 4:37am UTC](https://discuss.elastic.co/t/elasticsearch-nodes-doing-young-generation-gc-very-frequently/163661 "2019-01-14T04:37:43Z")
Cluster details : Elasticsearch version : 6.3.0 Java version : 1.8.0\_191 54 data nodes Each BM is split into 2 VMs. Each VM has configuration : 128 GB RAM, 31 GB Heap, 18 cores 3 master nodes Jvm options -Xms31744… --- ## [Search a PDF file using its content](https://discuss.elastic.co/t/search-a-pdf-file-using-its-content/164024)
**Author:** [@Mani\_Sundharram](https://discuss.elastic.co/u/Mani_Sundharram)\ **Replies:** 8\ **Last updated:** [January 14, 2019, 3:48am UTC](https://discuss.elastic.co/t/search-a-pdf-file-using-its-content/164024 "2019-01-14T03:48:21Z")
Hi, I have installed and setup elastic search and ingest-attachment plugin. I need to search through a list of pdf files (20000) given in a file path how would I do. How do you push the data to the elastic search , i… --- ## [ES Template versioning](https://discuss.elastic.co/t/es-template-versioning/163962)
**Author:** [@oxygen300](https://discuss.elastic.co/u/oxygen300)\ **Replies:** 1\ **Last updated:** [January 14, 2019, 1:49am UTC](https://discuss.elastic.co/t/es-template-versioning/163962 "2019-01-14T01:49:58Z")
Hi Team, I have a question about the use case of version field in the index templates. https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html#versioning-templates I have observed that I… --- ## [Help Required with query](https://discuss.elastic.co/t/help-required-with-query/163980)
**Author:** [@Ramachandra\_Bhaskar](https://discuss.elastic.co/u/Ramachandra_Bhaskar)\ **Replies:** 5\ **Last updated:** [January 13, 2019, 9:52pm UTC](https://discuss.elastic.co/t/help-required-with-query/163980 "2019-01-13T21:52:35Z")
We have our logs in Elasticsearch which are normalized. Here is what we store in nginx logs. “\_source”: { “remote”: “influx-dev-cluster01”, “host”: “10.88.14.126”, “ident”: “-”, “user”: “user”, “method”: “POST”, “path”:… --- ## [Show/Display indexes that have stopped receiving data](https://discuss.elastic.co/t/show-display-indexes-that-have-stopped-receiving-data/164047)
**Author:** [@bryan\_stuhlsatz](https://discuss.elastic.co/u/bryan_stuhlsatz)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 5:50pm UTC](https://discuss.elastic.co/t/show-display-indexes-that-have-stopped-receiving-data/164047 "2019-01-13T17:50:32Z")
We have a diverse multi-purpose cluster. It has over 60 different interfaces that receive data (snmp, syslog, filebeat, logstash, pipelines, perl/powershell/python scripts, ... Each of these are important and we need … --- ## [Elasticsearch probability depends on document weight](https://discuss.elastic.co/t/elasticsearch-probability-depends-on-document-weight/164048)
**Author:** [@Dmitry\_Ermichev](https://discuss.elastic.co/u/Dmitry_Ermichev)\ **Replies:** 0\ **Last updated:** [January 13, 2019, 5:24pm UTC](https://discuss.elastic.co/t/elasticsearch-probability-depends-on-document-weight/164048 "2019-01-13T17:24:01Z")
I want to make query, that returns random documents with probability of each document depends on it weight. In my index each document has weight from 1 to N. So element with weight 1 must appears in result 2 times less, … --- ## [Elasticsearch how to prevent duplicate data?](https://discuss.elastic.co/t/elasticsearch-how-to-prevent-duplicate-data/164039)
**Author:** [@m\_5amy](https://discuss.elastic.co/u/m_5amy)\ **Replies:** 2\ **Last updated:** [January 13, 2019, 2:39pm UTC](https://discuss.elastic.co/t/elasticsearch-how-to-prevent-duplicate-data/164039 "2019-01-13T14:39:01Z")
Hi all, I need to prevent documents to be inserted into single index if it has the same data EX: { "\_score":1.0, "\_source":{ "value":{ "user":"test user", "age":"20" } }… --- ## [Elasticsearch - Snapshot having unassigned shards and Repository verification exception](https://discuss.elastic.co/t/elasticsearch-snapshot-having-unassigned-shards-and-repository-verification-exception/163749)
**Author:** [@Nikesh](https://discuss.elastic.co/u/Nikesh)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 12:31pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-having-unassigned-shards-and-repository-verification-exception/163749 "2019-01-13T12:31:36Z")
Hi, I am working on backing up my indexes present in my cluster. I have a dedicated master node and two data nodes. My configuration file in all nodes contains " path.repo: \["/u01/backup"\] ". Firstly, I created a … --- ## [How to detect and log 400 bad requests?](https://discuss.elastic.co/t/how-to-detect-and-log-400-bad-requests/163778)
**Author:** [@gcsabbagh](https://discuss.elastic.co/u/gcsabbagh)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 12:21pm UTC](https://discuss.elastic.co/t/how-to-detect-and-log-400-bad-requests/163778 "2019-01-13T12:21:20Z")
Hello, My apologies if asked before but I've been scouting the forums and the docs for hours trying to figure this out.. My setup: Fluentd daemonset on Kubernetes collecting logs from containers and sending them to AW… --- ## [Elasticsearch 2.1.1 -DFS\_QUERY\_THEN\_FETCH - how to get most recent doc](https://discuss.elastic.co/t/elasticsearch-2-1-1-dfs-query-then-fetch-how-to-get-most-recent-doc/163947)
**Author:** [@Neni](https://discuss.elastic.co/u/Neni)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 11:38am UTC](https://discuss.elastic.co/t/elasticsearch-2-1-1-dfs-query-then-fetch-how-to-get-most-recent-doc/163947 "2019-01-13T11:38:42Z")
We are using ES version 2.1.1 and below code not returning the latest document created. Please suggest and let me know if the DFS\_QUERY\_THEN\_FETCH is good idea to use in prod ? we lookup the given input in ES by using… --- ## [Elasticsearch not creating the Index for new pipeline via logstash](https://discuss.elastic.co/t/elasticsearch-not-creating-the-index-for-new-pipeline-via-logstash/164004)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 11:33am UTC](https://discuss.elastic.co/t/elasticsearch-not-creating-the-index-for-new-pipeline-via-logstash/164004 "2019-01-13T11:33:04Z")
I have set-up a ELK but I see elasticsearch not creating the Index and unable to upload the data, Service Elasticsearch and Logstash both are running.. Below is the details.. However I do not see anything on he logs. E… --- ## [Using save in elastic\_search dsl results in 504 gateway timeout error](https://discuss.elastic.co/t/using-save-in-elastic-search-dsl-results-in-504-gateway-timeout-error/164026)
**Author:** [@rehabreda](https://discuss.elastic.co/u/rehabreda)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 11:27am UTC](https://discuss.elastic.co/t/using-save-in-elastic-search-dsl-results-in-504-gateway-timeout-error/164026 "2019-01-13T11:27:34Z")
hi i am using elasticsearch dsl to save data it may succeed in saving 10 records for example and in the record #11 it fails and it show 504 gateway timeout error i don't know how to solve this ! --- ## [Long Duration IndexOrDocValuesQuery](https://discuss.elastic.co/t/long-duration-indexordocvaluesquery/164022)
**Author:** [@wakawaka54](https://discuss.elastic.co/u/wakawaka54)\ **Replies:** 0\ **Last updated:** [January 13, 2019, 7:33am UTC](https://discuss.elastic.co/t/long-duration-indexordocvaluesquery/164022 "2019-01-13T07:33:50Z")
Hey guys! I am using a 5 node elasticsearch cluster with a ~120GB index. We are having issues with slow queries that have several filters. After profiling our queries, I see that IndexOrDocValuesQuery is taking about 3… --- ## [Server specification](https://discuss.elastic.co/t/server-specification/164019)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 6:35am UTC](https://discuss.elastic.co/t/server-specification/164019 "2019-01-13T06:35:52Z")
hi all, I have 10G log per day, could you please advise me about the specification of servers? (suppose there are 3 nodes to have high availability and each node is in a separate server (VM), also just there is an index… --- ## [Recommender Query With Rules](https://discuss.elastic.co/t/recommender-query-with-rules/164009)
**Author:** [@pferrel](https://discuss.elastic.co/u/pferrel)\ **Replies:** 0\ **Last updated:** [January 12, 2019, 11:07pm UTC](https://discuss.elastic.co/t/recommender-query-with-rules/164009 "2019-01-12T23:07:06Z")
I have a recommender that stores it's model in Elasticsearch. I want a query that return items that match "terms" in the model, but that also has business rules to include, exclude, and boost hits based on metadata store… --- ## [Rejecting mapping update to \[.kibana\] as the final mapping would have more than 1 type: \[visualization, doc\]](https://discuss.elastic.co/t/rejecting-mapping-update-to-kibana-as-the-final-mapping-would-have-more-than-1-type-visualization-doc/162491)
**Author:** [@kanagasathish](https://discuss.elastic.co/u/kanagasathish)\ **Replies:** 1\ **Last updated:** [January 12, 2019, 7:23pm UTC](https://discuss.elastic.co/t/rejecting-mapping-update-to-kibana-as-the-final-mapping-would-have-more-than-1-type-visualization-doc/162491 "2019-01-12T19:23:35Z")
In postman i posted below data Url: localhost:9200/.kibana/visualization/scanreport header : content-type/application/json Method:Put Requestbody: {"title":"PieSampleChart","visState":"{"aggs":\[{"id":"1","params":{}… --- ## [Clear text password in Elasticsearch, logstash, kibana and Beats](https://discuss.elastic.co/t/clear-text-password-in-elasticsearch-logstash-kibana-and-beats/163487)
**Author:** [@DineshTripathi](https://discuss.elastic.co/u/DineshTripathi)\ **Replies:** 4\ **Last updated:** [January 12, 2019, 3:29pm UTC](https://discuss.elastic.co/t/clear-text-password-in-elasticsearch-logstash-kibana-and-beats/163487 "2019-01-12T15:29:16Z")
How we can encrypt the clear text password in different components config files? I see few doc talking about adding those setting in keystore but not very clear. Can someone guide? --- ## [How to do the elastic search bulk get operation with multiple document Id's using python](https://discuss.elastic.co/t/how-to-do-the-elastic-search-bulk-get-operation-with-multiple-document-ids-using-python/163986)
**Author:** [@vinodanumarla](https://discuss.elastic.co/u/vinodanumarla)\ **Replies:** 1\ **Last updated:** [January 12, 2019, 1:26pm UTC](https://discuss.elastic.co/t/how-to-do-the-elastic-search-bulk-get-operation-with-multiple-document-ids-using-python/163986 "2019-01-12T13:26:18Z")
I would like to do elastic search bulk get operation with multiple document Id's using python. But I could able to find proper solution. Code which I'm using to get the single document based on document Id. import elas… --- ## [Complex query - possible with ElasticSearch?](https://discuss.elastic.co/t/complex-query-possible-with-elasticsearch/163994)
**Author:** [@Clogg](https://discuss.elastic.co/u/Clogg)\ **Replies:** 0\ **Last updated:** [January 12, 2019, 12:10pm UTC](https://discuss.elastic.co/t/complex-query-possible-with-elasticsearch/163994 "2019-01-12T12:10:26Z")
Hello, I am just starting by adventure with ElasticSearch and I am thinking to use it in my upcoming project, as it seems to fit there. The ElasticSearch part of the project would be to store and search through hundred… --- ## [How to use Nest Library (C#) for msearch/template](https://discuss.elastic.co/t/how-to-use-nest-library-c-for-msearch-template/163575)
**Author:** [@Nico92](https://discuss.elastic.co/u/Nico92)\ **Replies:** 1\ **Last updated:** [January 11, 2019, 10:43pm UTC](https://discuss.elastic.co/t/how-to-use-nest-library-c-for-msearch-template/163575 "2019-01-11T22:43:26Z")
Hi guys, I have a saved template for msearch thats look like: POST \_scripts/t1 { "script": { "lang": "mustache", "source": { "query": { "match\_phrase": { "{{field}}": "{{value}}" … --- ## [Missing prediction values with moving average aggregation or moving function in Elasticsearch](https://discuss.elastic.co/t/missing-prediction-values-with-moving-average-aggregation-or-moving-function-in-elasticsearch/163914)
**Author:** [@jmou](https://discuss.elastic.co/u/jmou)\ **Replies:** 0\ **Last updated:** [January 11, 2019, 2:50pm UTC](https://discuss.elastic.co/t/missing-prediction-values-with-moving-average-aggregation-or-moving-function-in-elasticsearch/163914 "2019-01-11T14:50:18Z")
Hello there, I'm getting 2 issues when trying to use the moving average aggregation functions in Elasticsearch to generate Holtwinters predictions: getting missing parts of prediction values (getting "null" instead) t… --- ## [X-pack security fallback realm configuration?](https://discuss.elastic.co/t/x-pack-security-fallback-realm-configuration/163934)
**Author:** [@jharrisonpnnl](https://discuss.elastic.co/u/jharrisonpnnl)\ **Replies:** 1\ **Last updated:** [January 11, 2019, 7:56pm UTC](https://discuss.elastic.co/t/x-pack-security-fallback-realm-configuration/163934 "2019-01-11T19:56:20Z")
Hi all, I'm trying to set up tiered realms in my elasticsearch.yml. I'd think that the following realm config: xpack.security.authc.realms: native1: type: native order: 1 pki1: type: pki … --- ## [Elasticsearch YUM Repositories Are Down](https://discuss.elastic.co/t/elasticsearch-yum-repositories-are-down/163405)
**Author:** [@vanjohnson](https://discuss.elastic.co/u/vanjohnson)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 7:43pm UTC](https://discuss.elastic.co/t/elasticsearch-yum-repositories-are-down/163405 "2019-01-11T19:43:45Z")
Elasticsearch Community, The documentation for installing Elasticsearch as an RPM is out of date. Not sure on the status of the YUM repos. I noticed there was ticket(155198) open in Nov, yet was closed with no resolut… --- ## [ES - cluster not green - not sure if this is due to JVM memory](https://discuss.elastic.co/t/es-cluster-not-green-not-sure-if-this-is-due-to-jvm-memory/163948)
**Author:** [@Neni](https://discuss.elastic.co/u/Neni)\ **Replies:** 0\ **Last updated:** [January 11, 2019, 6:50pm UTC](https://discuss.elastic.co/t/es-cluster-not-green-not-sure-if-this-is-due-to-jvm-memory/163948 "2019-01-11T18:50:32Z")
Continuing the discussion from ElasticSearch -Unable to lock JVM Memory: --- ## [ElasticSearch version 2.1.1 -Unable to lock JVM Memory](https://discuss.elastic.co/t/elasticsearch-version-2-1-1-unable-to-lock-jvm-memory/163274)
**Author:** [@Neni](https://discuss.elastic.co/u/Neni)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 6:43pm UTC](https://discuss.elastic.co/t/elasticsearch-version-2-1-1-unable-to-lock-jvm-memory/163274 "2019-01-11T18:43:10Z")
Could someone please look into this error and provide solution. We have 3 aws instances of type m4.2xlarge being used for elasticsearch with 1 master node and 2 shards; total of 3 indices , one of the index is holding 1… --- ## [BulkProcessor does not retry on failures](https://discuss.elastic.co/t/bulkprocessor-does-not-retry-on-failures/163945)
**Author:** [@Brian\_Clozel](https://discuss.elastic.co/u/Brian_Clozel)\ **Replies:** 0\ **Last updated:** [January 11, 2019, 6:11pm UTC](https://discuss.elastic.co/t/bulkprocessor-does-not-retry-on-failures/163945 "2019-01-11T18:11:29Z")
As of https://github.com/elastic/elasticsearch/issues/14620, the BulkProcessor retries individual request failures. In case the Elasticsearch instance fails to respond or in case of a network error, the RetryHandler list… --- ## [More\_like\_this query returns no results unless min\_doc\_freq increased](https://discuss.elastic.co/t/more-like-this-query-returns-no-results-unless-min-doc-freq-increased/163824)
**Author:** [@pgodzin](https://discuss.elastic.co/u/pgodzin)\ **Replies:** 1\ **Last updated:** [January 11, 2019, 4:17pm UTC](https://discuss.elastic.co/t/more-like-this-query-returns-no-results-unless-min-doc-freq-increased/163824 "2019-01-11T16:17:28Z")
I am posting the following query against the \_explain endpoint of a document with "three" in the title field { "query": { "bool": { "must": \[ { "bool": { "shou… --- ## [How to retrieve xpack watch\_id](https://discuss.elastic.co/t/how-to-retrieve-xpack-watch-id/162888)
**Author:** [@thinkdffrnt](https://discuss.elastic.co/u/thinkdffrnt)\ **Replies:** 1\ **Last updated:** [January 11, 2019, 3:29pm UTC](https://discuss.elastic.co/t/how-to-retrieve-xpack-watch-id/162888 "2019-01-11T15:29:43Z")
Currently I replaced all data nodes and my cluster is red. There are unassigned shards in .watches index which I'm unable to delete. Based on this documentation I need to add watch\_id in delete query https://www.elasti… --- ## [Why aggregation result return max results only 10](https://discuss.elastic.co/t/why-aggregation-result-return-max-results-only-10/163892)
**Author:** [@examin](https://discuss.elastic.co/u/examin)\ **Replies:** 4\ **Last updated:** [January 11, 2019, 2:59pm UTC](https://discuss.elastic.co/t/why-aggregation-result-return-max-results-only-10/163892 "2019-01-11T14:59:08Z")
i have a query on which i am bucketing year wise. Result return only 10 results --- ## [Elastic Search future Versions?](https://discuss.elastic.co/t/elastic-search-future-versions/163904)
**Author:** [@rog\_NB](https://discuss.elastic.co/u/rog_NB)\ **Replies:** 1\ **Last updated:** [January 11, 2019, 2:40pm UTC](https://discuss.elastic.co/t/elastic-search-future-versions/163904 "2019-01-11T14:40:23Z")
Hello I would like to know, if there is any information for upcomming versions of Elasticsearch. There are a lot of informations around, when which version came out, but I didn't found any for upcomming versions and wh… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1426) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1428)