# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1602 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1603 --- ## [How to remove duplicate values in ealstic search](https://discuss.elastic.co/t/how-to-remove-duplicate-values-in-ealstic-search/112226)
**Author:** [@narasiman1986](https://discuss.elastic.co/u/narasiman1986)\ **Replies:** 26\ **Last updated:** [July 26, 2018, 8:00pm UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-values-in-ealstic-search/112226 "2018-07-26T20:00:13Z")
i am using elastic search 5.6.4 GET /clinicaldata/\_search { "query": { "match" : { "mrdno" : "112627" } } } in that mrdno 112627 have duplicate of 75 records. when run in the above shows ouput as follows { "t… --- ## [Assigning higher scores to specific terms in BM25](https://discuss.elastic.co/t/assigning-higher-scores-to-specific-terms-in-bm25/141839)
**Author:** [@nuketro0p3r](https://discuss.elastic.co/u/nuketro0p3r)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 7:37pm UTC](https://discuss.elastic.co/t/assigning-higher-scores-to-specific-terms-in-bm25/141839 "2018-07-26T19:37:55Z")
Is it possible to tell ElasticSearch to boost the score of a set of words when computing BM25 scores for a document match? The context is that the document contains some words/symbols that are more important to match th… --- ## [Question regarding Sorts](https://discuss.elastic.co/t/question-regarding-sorts/138018)
**Author:** [@XiaoConstantine](https://discuss.elastic.co/u/XiaoConstantine)\ **Replies:** 10\ **Last updated:** [July 26, 2018, 6:18pm UTC](https://discuss.elastic.co/t/question-regarding-sorts/138018 "2018-07-26T18:18:27Z")
Hello guys! I have a question regarding to sorts, the scenario i have is following: I m creating a list of sort on one field with different custom sorting scripts, so eventually it looks like: sorts: \[ script: {}, #… --- ## [Jar hell when running integration test](https://discuss.elastic.co/t/jar-hell-when-running-integration-test/141819)
**Author:** [@svangeti](https://discuss.elastic.co/u/svangeti)\ **Replies:** 2\ **Last updated:** [July 26, 2018, 6:07pm UTC](https://discuss.elastic.co/t/jar-hell-when-running-integration-test/141819 "2018-07-26T18:07:37Z")
getting this error when running integration test. using ES 6.3.2 \[INFO\] --- junit4-maven-plugin:2.3.3:junit4 (unit-tests) @ ingest-bano --- \[INFO\] says ciao! Master seed: 417785527A133182 Executing 1 suite with 1 JV… --- ## [Elasticsearch Processors](https://discuss.elastic.co/t/elasticsearch-processors/141435)
**Author:** [@amandrell](https://discuss.elastic.co/u/amandrell)\ **Replies:** 1\ **Last updated:** [July 24, 2018, 9:43pm UTC](https://discuss.elastic.co/t/elasticsearch-processors/141435 "2018-07-24T21:43:37Z")
Hello, I am going through the list of Processors to use to pre-process documents before they get indexed into ES. For example, let's talk about the Convert Processor. I understand that it is necessary to provide the "fi… --- ## [Accessing Data in Pipelines](https://discuss.elastic.co/t/accessing-data-in-pipelines/141824)
**Author:** [@amandrell](https://discuss.elastic.co/u/amandrell)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 5:27pm UTC](https://discuss.elastic.co/t/accessing-data-in-pipelines/141824 "2018-07-26T17:27:49Z")
Hello, I am having trouble accessing data that gets ingested into my data pipeline, that should transform data within each document that gets indexed into ES. I understand that this data can be accessed via dot notatio… --- ## [ScrollAll varying record counts and speed](https://discuss.elastic.co/t/scrollall-varying-record-counts-and-speed/141629)
**Author:** [@Robert\_McGirr](https://discuss.elastic.co/u/Robert_McGirr)\ **Replies:** 0\ **Last updated:** [July 25, 2018, 4:55pm UTC](https://discuss.elastic.co/t/scrollall-varying-record-counts-and-speed/141629 "2018-07-25T16:55:27Z")
I have a daily need to retrieve all documents from a particular index. I am using ScrollAll (below) but have 2 issues. The number of docs in this index is currently static, but I receive varying doc counts when I run … --- ## [\[DEBUG\]\[action.admin.cluster.health\] when i try to create a cluster on 2.4.0 ElasticSearch](https://discuss.elastic.co/t/debug-action-admin-cluster-health-when-i-try-to-create-a-cluster-on-2-4-0-elasticsearch/141672)
**Author:** [@Rodrigo\_Mtz](https://discuss.elastic.co/u/Rodrigo_Mtz)\ **Replies:** 14\ **Last updated:** [July 26, 2018, 5:11pm UTC](https://discuss.elastic.co/t/debug-action-admin-cluster-health-when-i-try-to-create-a-cluster-on-2-4-0-elasticsearch/141672 "2018-07-26T17:11:48Z")
Hi i have an issue when i try to create a cluster with 2 nodes here is the first configuration file: ---------------------------------- Cluster ----------------------------------- Use a descriptive name for your clust… --- ## [Elastic search installation problem - \\Common cannot be processed syntactically at this point](https://discuss.elastic.co/t/elastic-search-installation-problem-common-cannot-be-processed-syntactically-at-this-point/141810)
**Author:** [@satish6955](https://discuss.elastic.co/u/satish6955)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 3:11pm UTC](https://discuss.elastic.co/t/elastic-search-installation-problem-common-cannot-be-processed-syntactically-at-this-point/141810 "2018-07-26T15:11:05Z")
Hi everyone, Pls help me out here. While trying to run the "elasticsearch.bat" file in command prompt for windows, i get the following error. \\Common cannot be processed syntactically at this point Pls help me out her… --- ## [Convert a string field to date in order to sort it](https://discuss.elastic.co/t/convert-a-string-field-to-date-in-order-to-sort-it/141791)
**Author:** [@Angelos](https://discuss.elastic.co/u/Angelos)\ **Replies:** 3\ **Last updated:** [July 26, 2018, 3:06pm UTC](https://discuss.elastic.co/t/convert-a-string-field-to-date-in-order-to-sort-it/141791 "2018-07-26T15:06:43Z")
Hello! I want to sort a field that happens to be a string. but the actual value of that string is date. Is there any way to convert this field to date? GET /index/\_search { "query": { "bool": { "must": \[ { "matc… --- ## [Script quering with Painless scripts](https://discuss.elastic.co/t/script-quering-with-painless-scripts/141792)
**Author:** [@Jevgenij](https://discuss.elastic.co/u/Jevgenij)\ **Replies:** 1\ **Last updated:** [July 26, 2018, 2:54pm UTC](https://discuss.elastic.co/t/script-quering-with-painless-scripts/141792 "2018-07-26T14:54:21Z")
Hi! Have tried a lot of things, like Java Instant.now().toEpochMilli() or joda Days().toStandardDuration() with no success. Found out what while Java Instant class isn't accepted, Java System.currentTimeMillis() is. M… --- ## [Search similar phrases](https://discuss.elastic.co/t/search-similar-phrases/141796)
**Author:** [@Gosforth](https://discuss.elastic.co/u/Gosforth)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 2:35pm UTC](https://discuss.elastic.co/t/search-similar-phrases/141796 "2018-07-26T14:35:48Z")
Hi, I'm new to Elastic. I think about using this product to build product searching engine (currently works with MySQL). Suppose I have products "HP DeskJet 710c", "HP DeskJet 712c", "HP DeskJet 820cXi", "Oki C5650" T… --- ## [Poor performance on brand new cluster](https://discuss.elastic.co/t/poor-performance-on-brand-new-cluster/139919)
**Author:** [@1fe60245fe160bc0f8ab](https://discuss.elastic.co/u/1fe60245fe160bc0f8ab)\ **Replies:** 12\ **Last updated:** [July 26, 2018, 2:12pm UTC](https://discuss.elastic.co/t/poor-performance-on-brand-new-cluster/139919 "2018-07-26T14:12:53Z")
Hi all, We've set a new elasticsearch cluster to store netflow data. Settings are as following: 4 data nodes, each 32GB RAM (around 40% heap size), 16 cores, 240GB NFS-accessed SSD 3 master nodes 8GB RAM, 2 cores, 20G… --- ## [Delete by query took more time than expected](https://discuss.elastic.co/t/delete-by-query-took-more-time-than-expected/141608)
**Author:** [@kish](https://discuss.elastic.co/u/kish)\ **Replies:** 12\ **Last updated:** [July 26, 2018, 1:31pm UTC](https://discuss.elastic.co/t/delete-by-query-took-more-time-than-expected/141608 "2018-07-26T13:31:42Z")
Hello - I have a made query to delete an entire from index at 13:49:48 but ES took almost 40-50sec to delete the query. But in /elasticsearch/log/\*.log file i am unable to find the any details regarding it. From avail… --- ## [High load on ES 1.7.6 single node](https://discuss.elastic.co/t/high-load-on-es-1-7-6-single-node/141774)
**Author:** [@emot](https://discuss.elastic.co/u/emot)\ **Replies:** 1\ **Last updated:** [July 26, 2018, 1:31pm UTC](https://discuss.elastic.co/t/high-load-on-es-1-7-6-single-node/141774 "2018-07-26T13:31:42Z")
Hi, we have an application that doesn't require a cluster (at least for now) and was running OK for a while with only one node. Today we started to notice high load on the machine with no clear information in the logs. … --- ## [Primary shards not balanced across nodes for elasticsearch 5.2](https://discuss.elastic.co/t/primary-shards-not-balanced-across-nodes-for-elasticsearch-5-2/141746)
**Author:** [@Jayakrishnan\_Nair](https://discuss.elastic.co/u/Jayakrishnan_Nair)\ **Replies:** 1\ **Last updated:** [July 26, 2018, 1:18pm UTC](https://discuss.elastic.co/t/primary-shards-not-balanced-across-nodes-for-elasticsearch-5-2/141746 "2018-07-26T13:18:14Z")
Hi, I am using elasticsearch version 5.2, we have setup 4 data nodes and 3 masters cluster. In our cluster we have 20 indexes with 5 primary and 1 replica. We have seen 2 data node on same physical server has 89 primary… --- ## [Log4j2 configuration file not found](https://discuss.elastic.co/t/log4j2-configuration-file-not-found/141771)
**Author:** [@dikshash](https://discuss.elastic.co/u/dikshash)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 12:56pm UTC](https://discuss.elastic.co/t/log4j2-configuration-file-not-found/141771 "2018-07-26T12:56:08Z")
Hi all I am new to elasticsearch and have to perform some basic operation using elasticsearch and when i run elasticsearch from command prompt I am getting following mentioned error. Kindly help. Exception in thread "m… --- ## [Cannot instantiate the type TransportClient](https://discuss.elastic.co/t/cannot-instantiate-the-type-transportclient/141725)
**Author:** [@BleedCode](https://discuss.elastic.co/u/BleedCode)\ **Replies:** 7\ **Last updated:** [July 26, 2018, 12:03pm UTC](https://discuss.elastic.co/t/cannot-instantiate-the-type-transportclient/141725 "2018-07-26T12:03:03Z")
For fetching data from multiple indices, i am trying to use SearchResponse client. But while creating transportClient i get instantiate error. What wrong i am doing? Following is my code snippet: public void setup() t… --- ## [Elasticsearch Data Directory size anomaly](https://discuss.elastic.co/t/elasticsearch-data-directory-size-anomaly/141742)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 4\ **Last updated:** [July 26, 2018, 11:30am UTC](https://discuss.elastic.co/t/elasticsearch-data-directory-size-anomaly/141742 "2018-07-26T11:30:37Z")
Hi All , It is a theoretical question. I am indexed almost 2 GB of data in elasticsearch with default Shards & Replicas . Logs indexed from Feb 1 to July 15 year . Index name is with respect to date . ie in log… --- ## [Nested filter and parents in Aggregations](https://discuss.elastic.co/t/nested-filter-and-parents-in-aggregations/141741)
**Author:** [@Stephan\_Daratha](https://discuss.elastic.co/u/Stephan_Daratha)\ **Replies:** 1\ **Last updated:** [July 26, 2018, 11:28am UTC](https://discuss.elastic.co/t/nested-filter-and-parents-in-aggregations/141741 "2018-07-26T11:28:27Z")
Hi, I have a little problem with aggregations. Is it possible to filter parents in aggregations. If i try \> "aggregations": { \> "sleeping": { \> "filter": { \> "bool": { \> "must": \[ \> … --- ## [Syslogs from fortigate](https://discuss.elastic.co/t/syslogs-from-fortigate/141405)
**Author:** [@Marcos\_Felix](https://discuss.elastic.co/u/Marcos_Felix)\ **Replies:** 15\ **Last updated:** [July 26, 2018, 11:10am UTC](https://discuss.elastic.co/t/syslogs-from-fortigate/141405 "2018-07-26T11:10:13Z")
Hello, I just configured fortigate to send all the syslogs to logstash This is my logstash config file: filter { udp { port =\> 514 { if \[type\] == "SYSLOG" { grok { patterns\_dir =\> \["/etc/logstash… --- ## [ES data persistence?](https://discuss.elastic.co/t/es-data-persistence/141745)
**Author:** [@bjg2](https://discuss.elastic.co/u/bjg2)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 11:03am UTC](https://discuss.elastic.co/t/es-data-persistence/141745 "2018-07-26T11:03:16Z")
Hi! I have kubernetes with elasticsearch in it, for logs. I have a single-node elastic search cluster (to keep things simple as I think I don't need bigger at the moment). /data folder is on persisted storage. In this … --- ## [After the 6.3.2 Update I get many Warnings in the Log](https://discuss.elastic.co/t/after-the-6-3-2-update-i-get-many-warnings-in-the-log/141737)
**Author:** [@kyuubi](https://discuss.elastic.co/u/kyuubi)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 10:52am UTC](https://discuss.elastic.co/t/after-the-6-3-2-update-i-get-many-warnings-in-the-log/141737 "2018-07-26T10:52:21Z")
I did just updated my Elastic Stack and after that I see many Warnings in the elasticsearch.log. Does anyone know where they come from and how I can eliminate them? \[2018-07-26T11:31:27,040\]\[INFO \]\[o.e.n.Node … --- ## [Curator crontab not running](https://discuss.elastic.co/t/curator-crontab-not-running/141530)
**Author:** [@babydoll](https://discuss.elastic.co/u/babydoll)\ **Replies:** 5\ **Last updated:** [July 26, 2018, 10:41am UTC](https://discuss.elastic.co/t/curator-crontab-not-running/141530 "2018-07-26T10:41:49Z")
Hello. I'm running elasticsearch curator on Ubuntu. So my curator works: curator --config /home/xen/curator/curator.yml /home/xen/curator/rollover.yml and executing the curator creates a new index as I expected. I want… --- ## [Nginx-status page](https://discuss.elastic.co/t/nginx-status-page/141734)
**Author:** [@sicute](https://discuss.elastic.co/u/sicute)\ **Replies:** 0\ **Last updated:** [July 26, 2018, 10:34am UTC](https://discuss.elastic.co/t/nginx-status-page/141734 "2018-07-26T10:34:11Z")
Hi , I try grab nginx-status page for metricbeat . here my nginx config \[spoiler\]cat /etc/nginx/conf.d/xxxxxxxx.conf server { listen 80 default\_server; server\_name xxxxxx; if ($http\_x\_forwarded\_proto = 'http') { r… --- ## [Is Elasticsearch SQL going to be in XPack?](https://discuss.elastic.co/t/is-elasticsearch-sql-going-to-be-in-xpack/141723)
**Author:** [@Drammy](https://discuss.elastic.co/u/Drammy)\ **Replies:** 4\ **Last updated:** [July 26, 2018, 10:33am UTC](https://discuss.elastic.co/t/is-elasticsearch-sql-going-to-be-in-xpack/141723 "2018-07-26T10:33:38Z")
Hi all, I'd love to be able to use the beta Elasticsearch SQL functionality but I don't want to waste time developing around something that we won't be able to afford should it only be released post-beta in the XPack. I… --- ## [Elasticsearch won't rebalance (v. 2.0.1); Need to upgrade!](https://discuss.elastic.co/t/elasticsearch-wont-rebalance-v-2-0-1-need-to-upgrade/141245)
**Author:** [@eschie](https://discuss.elastic.co/u/eschie)\ **Replies:** 6\ **Last updated:** [July 26, 2018, 10:13am UTC](https://discuss.elastic.co/t/elasticsearch-wont-rebalance-v-2-0-1-need-to-upgrade/141245 "2018-07-26T10:13:56Z")
I inherited a 1 node cluster running on 2.0.1 that is crashing due to too many open file descriptors! (1258 indices, 6280 shards, 5 each w/no replicas). To mitigate the load, I added a new node to the cluster but the sh… --- ## [\[Ergent and confusing\]HDFS snapshot in elasticsearch 6.3.1](https://discuss.elastic.co/t/ergent-and-confusing-hdfs-snapshot-in-elasticsearch-6-3-1/141727)
**Author:** [@Zhengda\_Wu](https://discuss.elastic.co/u/Zhengda_Wu)\ **Replies:** 1\ **Last updated:** [July 26, 2018, 9:55am UTC](https://discuss.elastic.co/t/ergent-and-confusing-hdfs-snapshot-in-elasticsearch-6-3-1/141727 "2018-07-26T09:55:35Z")
Hi, I've worked on making a three node cluster elasticsearch 6.3.1 to make an HDFS snapshot, I've done everything I need to do and is encountering the issue above. What really bothers me is what is the thing that follo… --- ## [Hello World ES Plugin](https://discuss.elastic.co/t/hello-world-es-plugin/141617)
**Author:** [@Marco\_Costantini](https://discuss.elastic.co/u/Marco_Costantini)\ **Replies:** 2\ **Last updated:** [July 26, 2018, 9:48am UTC](https://discuss.elastic.co/t/hello-world-es-plugin/141617 "2018-07-26T09:48:08Z")
My goal is to test the invocation of some in-house software from within an ES plugin. My approach was to make a simple, Hello World style plugin that would print to the ES log/console. If I can see my log output, then I … --- ## [Transform + action](https://discuss.elastic.co/t/transform-action/141589)
**Author:** [@chrillebile](https://discuss.elastic.co/u/chrillebile)\ **Replies:** 3\ **Last updated:** [July 26, 2018, 9:36am UTC](https://discuss.elastic.co/t/transform-action/141589 "2018-07-26T09:36:26Z")
Hey, Is it possible to use the values you get from transform in the action log message? In my transform, I calculate the derivative and want to display it in my log/slack message. [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1601) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1603)