# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1821 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1822 --- ## [Error while creating spark dataframe with elasticsearch query](https://discuss.elastic.co/t/error-while-creating-spark-dataframe-with-elasticsearch-query/115293)
**Author:** [@pavan1](https://discuss.elastic.co/u/pavan1)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 5:59pm UTC](https://discuss.elastic.co/t/error-while-creating-spark-dataframe-with-elasticsearch-query/115293 "2018-01-19T17:59:46Z")
Hi, I am trying to read from elasticsearch and create a spark dataframe. I am getting following error. ERROR 2018-01-12 17:59:00,603 Executor task launch worker-1 DefaultWriterContainer:96 - Aborting task. org.elastic… --- ## [MISP Integration with ELK](https://discuss.elastic.co/t/misp-integration-with-elk/110625)
**Author:** [@anji](https://discuss.elastic.co/u/anji)\ **Replies:** 0\ **Last updated:** [December 7, 2017, 7:44am UTC](https://discuss.elastic.co/t/misp-integration-with-elk/110625 "2017-12-07T07:44:17Z")
Hello , can we integrate MISP (Malware Information Sharing Platform, a Threat Sharing Platform.) with ELK ?? if it's possible then how can we integrate MISP with ELK? Thnks & Regards, Anji --- ## [Docker Pull Failing](https://discuss.elastic.co/t/docker-pull-failing/116266)
**Author:** [@jdanielcook](https://discuss.elastic.co/u/jdanielcook)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 4:52pm UTC](https://discuss.elastic.co/t/docker-pull-failing/116266 "2018-01-19T16:52:25Z")
I am attempting to pull and receiving the error below: docker pull docker.elastic.co/elasticsearch/elasticsearch:6.1.2 Error response from daemon: Get https://docker.elastic.co/v2/: net/http: request canceled while wai… --- ## [Running Elasticsearch on AWS](https://discuss.elastic.co/t/running-elasticsearch-on-aws/113461)
**Author:** [@rudolphk](https://discuss.elastic.co/u/rudolphk)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 4:35pm UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws/113461 "2018-01-19T16:35:04Z")
I am planning on setting up a self-managed Elasticsearch cluster on AWS using i3.xlarge instance types. My question is regarding the ephemeral NVMe storage which comes with the instance type. I know information stored … --- ## [Finding parts of user query that were not matched](https://discuss.elastic.co/t/finding-parts-of-user-query-that-were-not-matched/115383)
**Author:** [@svoss](https://discuss.elastic.co/u/svoss)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 4:28pm UTC](https://discuss.elastic.co/t/finding-parts-of-user-query-that-were-not-matched/115383 "2018-01-19T16:28:09Z")
I am working on a project where a user is provided with Suggest as you type functionality. The suggestions are provided using a separate suggest index which is build from query history and predefined product categories. … --- ## [X-Pack 6.1.2 - Watcher Firing twice for the same event](https://discuss.elastic.co/t/x-pack-6-1-2-watcher-firing-twice-for-the-same-event/116267)
**Author:** [@jamesspi](https://discuss.elastic.co/u/jamesspi)\ **Replies:** 5\ **Last updated:** [January 19, 2018, 4:28pm UTC](https://discuss.elastic.co/t/x-pack-6-1-2-watcher-firing-twice-for-the-same-event/116267 "2018-01-19T16:28:06Z")
Hi, For most of our watches, watcher is firing twice for the same results since 6.1.2. Additionally, sometimes it doesn't fire all together. No issues whatsoever on 6.1.1. Anyone know if this is a known bug? Thanks! … --- ## [Reindex data with a new Field](https://discuss.elastic.co/t/reindex-data-with-a-new-field/115073)
**Author:** [@mikelD](https://discuss.elastic.co/u/mikelD)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 3:43pm UTC](https://discuss.elastic.co/t/reindex-data-with-a-new-field/115073 "2018-01-19T15:43:48Z")
Hi, Is there anyway to reindex the data and create a new field, from the data in the documents. For example, reindex an entire index and create a new field Hour(number) from a date type field(YYYY-mm-dd, HH-mm-ss). I K… --- ## [Unknown setting \[xpack.notification.email\]](https://discuss.elastic.co/t/unknown-setting-xpack-notification-email/115218)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 3\ **Last updated:** [January 19, 2018, 3:40pm UTC](https://discuss.elastic.co/t/unknown-setting-xpack-notification-email/115218 "2018-01-19T15:40:22Z")
I'm trying to follow Email Action | X-Pack for the Elastic Stack \[6.1\] | Elastic and getting following error while trying to start elasticsearch: elasticsearch | \[2018-01-12T05:48:39,956\]\[WARN \]\[o.e.b.ElasticsearchUncau… --- ## [Scaling Elastic on Azure](https://discuss.elastic.co/t/scaling-elastic-on-azure/115801)
**Author:** [@jdm1983](https://discuss.elastic.co/u/jdm1983)\ **Replies:** 4\ **Last updated:** [January 19, 2018, 3:21pm UTC](https://discuss.elastic.co/t/scaling-elastic-on-azure/115801 "2018-01-19T15:21:22Z")
I recently deployed an Elastic cluster to Azure using the template in the marketplace, but we need to add a couple more data nodes to the cluster. Is there an easy way to do this? I had attempted it on a previous cluste… --- ## [X-pack license node count](https://discuss.elastic.co/t/x-pack-license-node-count/116254)
**Author:** [@elasticus](https://discuss.elastic.co/u/elasticus)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 3:20pm UTC](https://discuss.elastic.co/t/x-pack-license-node-count/116254 "2018-01-19T15:20:45Z")
Hi does anyone know if the x-pack license node count includes ONLY elasticsearch cluster nodes or must I also count kibana instances and logstash instances? --- ## [Filtered index alias not working](https://discuss.elastic.co/t/filtered-index-alias-not-working/116244)
**Author:** [@mheck](https://discuss.elastic.co/u/mheck)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 3:17pm UTC](https://discuss.elastic.co/t/filtered-index-alias-not-working/116244 "2018-01-19T15:17:07Z")
I want to use an Index alias with a filter to have an index which only contains documents with a certain property set to a specific value. I added the following filter: { "filebeat-2018.01": { "aliases": { … --- ## [Significant Terms Aggregation to many fields](https://discuss.elastic.co/t/significant-terms-aggregation-to-many-fields/116116)
**Author:** [@66f41c7883cae82367a4](https://discuss.elastic.co/u/66f41c7883cae82367a4)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 3:02pm UTC](https://discuss.elastic.co/t/significant-terms-aggregation-to-many-fields/116116 "2018-01-19T15:02:06Z")
HI I would like to how to make Significant Terms Aggregation on many fields i found in the docs, that I can make aggregation over aggergation (parent) , but im afraid its only "affect" the doc\_count value and not the b… --- ## [Discover tab missing after xpack default user password change](https://discuss.elastic.co/t/discover-tab-missing-after-xpack-default-user-password-change/115931)
**Author:** [@elasticus](https://discuss.elastic.co/u/elasticus)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 2:27pm UTC](https://discuss.elastic.co/t/discover-tab-missing-after-xpack-default-user-password-change/115931 "2018-01-19T14:27:08Z")
Ouch! - just changed the default passwords for elastic, logstash\_system & kibana as per documentation guides (5.4.1) and after restarting my cluster and my kibana - the Discover tab is no longer present. Can anyone help… --- ## [Wildcard query](https://discuss.elastic.co/t/wildcard-query/116230)
**Author:** [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 2:17pm UTC](https://discuss.elastic.co/t/wildcard-query/116230 "2018-01-19T14:17:47Z")
I'm trying a wildcard query to match ip addresses. I have two src\_ip documents of starting "5." This query: "query" : { "wildcard" : { "src\_ip" : "5\*"} } returns one hit. What am I doing wrong? --- ## [Changing lat/lon from float to geo\_point](https://discuss.elastic.co/t/changing-lat-lon-from-float-to-geo-point/116067)
**Author:** [@jgwentworth](https://discuss.elastic.co/u/jgwentworth)\ **Replies:** 8\ **Last updated:** [January 19, 2018, 2:01pm UTC](https://discuss.elastic.co/t/changing-lat-lon-from-float-to-geo-point/116067 "2018-01-19T14:01:23Z")
When running GET myindex/\_mapping, I'm trying to use that same output to replace the mapping of lat/lon from datatype float to geo\_point with a put request. However, I'm getting the error: { "error": { "root\_cause": \[ … --- ## [Elasticsearch 6.1.0 Java API search issues](https://discuss.elastic.co/t/elasticsearch-6-1-0-java-api-search-issues/113581)
**Author:** [@Ajit\_Kumar](https://discuss.elastic.co/u/Ajit_Kumar)\ **Replies:** 5\ **Last updated:** [January 19, 2018, 1:53pm UTC](https://discuss.elastic.co/t/elasticsearch-6-1-0-java-api-search-issues/113581 "2018-01-19T13:53:21Z")
I am using ES - 6.1 and corresponding java version of the api. I was able to index the data using the java API. I validated this via kibana. All the mappings were correctly saved but an error comes up when I try to searc… --- ## [Removing last letter of a word - but only some words?](https://discuss.elastic.co/t/removing-last-letter-of-a-word-but-only-some-words/115499)
**Author:** [@smhoeks](https://discuss.elastic.co/u/smhoeks)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:37pm UTC](https://discuss.elastic.co/t/removing-last-letter-of-a-word-but-only-some-words/115499 "2018-01-19T13:37:32Z")
Hi there, elastic-people :slight\_smile: Im new to Elastic, basically working on already implemented settings, where i try to adapt index-setting to my needs. Right now im working specific with an elasticsearch-extensio… --- ## [Boost, but don't require?](https://discuss.elastic.co/t/boost-but-dont-require/115845)
**Author:** [@SparkyRih](https://discuss.elastic.co/u/SparkyRih)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 1:35pm UTC](https://discuss.elastic.co/t/boost-but-dont-require/115845 "2018-01-19T13:35:41Z")
I'm looking for a way to boost the score certain document which contain an array of fuzzy terms... Documents not containing any of the terms should still appear (but with a lower rating)... However, when I apply a filter… --- ## [Concatenate a value of three fields in the a new filed](https://discuss.elastic.co/t/concatenate-a-value-of-three-fields-in-the-a-new-filed/115725)
**Author:** [@marouane](https://discuss.elastic.co/u/marouane)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:27pm UTC](https://discuss.elastic.co/t/concatenate-a-value-of-three-fields-in-the-a-new-filed/115725 "2018-01-19T13:27:36Z")
Hello every one, I want to replace 3 fields that contains string(date) , string(time) and string(tz) by a new field with date type that contains (date+time+tz) converted. { "query":{ "bool":{ "must\_not":{ "exists":{ "f… --- ## [Doubts about Encrypting data at rest](https://discuss.elastic.co/t/doubts-about-encrypting-data-at-rest/115757)
**Author:** [@agonex](https://discuss.elastic.co/u/agonex)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:25pm UTC](https://discuss.elastic.co/t/doubts-about-encrypting-data-at-rest/115757 "2018-01-19T13:25:41Z")
Hi, I need to know if exist any guide or case to know how configure encrypt in rest. Its necessary for me make sure data at rest and not suffering changes in data visualization in kibana. Some forums talking about dm-cr… --- ## [Where to check indices.store.throttle.max\_bytes\_per\_sec when it is statically set](https://discuss.elastic.co/t/where-to-check-indices-store-throttle-max-bytes-per-sec-when-it-is-statically-set/115790)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:22pm UTC](https://discuss.elastic.co/t/where-to-check-indices-store-throttle-max-bytes-per-sec-when-it-is-statically-set/115790 "2018-01-19T13:22:28Z")
I have updated indices.store.throttle.max\_bytes\_per\_sec value in the yml file but I am unable to see the value when I make a GET call to the path /\_cluster/settings. { "persistent" : { }, "transient" : { } } Howeve… --- ## [How to redistribute the existing data](https://discuss.elastic.co/t/how-to-redistribute-the-existing-data/115972)
**Author:** [@huajing](https://discuss.elastic.co/u/huajing)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:14pm UTC](https://discuss.elastic.co/t/how-to-redistribute-the-existing-data/115972 "2018-01-19T13:14:50Z")
My elasticsearch server is 1.7.5 ,it has 4 nodes, node1 and node2 are masters ,node2 have 170G data and node4 only have 41G data . how to redistribute the existing data? --- ## [Elasticsearch can't handle multiple requests without dramatically decrease its performance](https://discuss.elastic.co/t/elasticsearch-cant-handle-multiple-requests-without-dramatically-decrease-its-performance/115938)
**Author:** [@montenegrodr](https://discuss.elastic.co/u/montenegrodr)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:13pm UTC](https://discuss.elastic.co/t/elasticsearch-cant-handle-multiple-requests-without-dramatically-decrease-its-performance/115938 "2018-01-19T13:13:32Z")
I have a two node cluster hosted in ElasticCloud. Host Elastic Cloud Platform Google Cloud Region US Central 1 (Iowa) Memory 8 GB Storage 192 GB SSD Yes HA Yes Each node has: Allocated Processors … --- ## [Added new nodes to the existing prod cluster](https://discuss.elastic.co/t/added-new-nodes-to-the-existing-prod-cluster/115971)
**Author:** [@srikanth\_nagineni](https://discuss.elastic.co/u/srikanth_nagineni)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 1:09pm UTC](https://discuss.elastic.co/t/added-new-nodes-to-the-existing-prod-cluster/115971 "2018-01-19T13:09:08Z")
I am having the 8 nodes existing es cluster. I need to double the size of the cluster. I added 8 more new nodes to the existing cluster. But the data syncing is not completed yet. I added 8 nodes in 24 hours back. Is the… --- ## [Does fuzzy query enumerate all terms?](https://discuss.elastic.co/t/does-fuzzy-query-enumerate-all-terms/115906)
**Author:** [@hubo3085632](https://discuss.elastic.co/u/hubo3085632)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 1:08pm UTC](https://discuss.elastic.co/t/does-fuzzy-query-enumerate-all-terms/115906 "2018-01-19T13:08:47Z")
The fuzzy query is pretty fast though it use Levenshtein distance,does it enumerate all terms? --- ## [Query question using should and must](https://discuss.elastic.co/t/query-question-using-should-and-must/115785)
**Author:** [@iitgrad](https://discuss.elastic.co/u/iitgrad)\ **Replies:** 5\ **Last updated:** [January 19, 2018, 12:32pm UTC](https://discuss.elastic.co/t/query-question-using-should-and-must/115785 "2018-01-19T12:32:07Z")
I have documents which have a "category" : \["cat1", "cat2", "cat3"\] etc. Some documents may have 20 or so different categories in the array, some may only have 1. I am trying to construct a query that does the following… --- ## [Aggregration through Java High Level Rest Client](https://discuss.elastic.co/t/aggregration-through-java-high-level-rest-client/116184)
**Author:** [@devil\_srj7](https://discuss.elastic.co/u/devil_srj7)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 11:50am UTC](https://discuss.elastic.co/t/aggregration-through-java-high-level-rest-client/116184 "2018-01-19T11:50:56Z")
I need to perform aggregations on multiple index through Java. RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost",9200,"http"))); SearchRequest searchRequest = new Sear… --- ## [The test gives different result from the real installation](https://discuss.elastic.co/t/the-test-gives-different-result-from-the-real-installation/113275)
**Author:** [@vgindin](https://discuss.elastic.co/u/vgindin)\ **Replies:** 4\ **Last updated:** [January 19, 2018, 11:55am UTC](https://discuss.elastic.co/t/the-test-gives-different-result-from-the-real-installation/113275 "2018-01-19T11:55:22Z")
Hi all. I've written a simple plugin, that implements custom scoring logic and extending Explanation. I have some real index data that looks like this: "\_source": { "price": 10000, "params\_model": "Дверь за… --- ## [When 5.6.6 version will be available in maven central](https://discuss.elastic.co/t/when-5-6-6-version-will-be-available-in-maven-central/116186)
**Author:** [@vsevolodk](https://discuss.elastic.co/u/vsevolodk)\ **Replies:** 4\ **Last updated:** [January 19, 2018, 11:11am UTC](https://discuss.elastic.co/t/when-5-6-6-version-will-be-available-in-maven-central/116186 "2018-01-19T11:11:02Z")
Hello, please tell when 5.6.6 version will be available in maven. Example for trasport client --- ## [How to aggregate a term before it's used to filter results](https://discuss.elastic.co/t/how-to-aggregate-a-term-before-its-used-to-filter-results/116081)
**Author:** [@Guy\_Bowden](https://discuss.elastic.co/u/Guy_Bowden)\ **Replies:** 2\ **Last updated:** [January 19, 2018, 11:05am UTC](https://discuss.elastic.co/t/how-to-aggregate-a-term-before-its-used-to-filter-results/116081 "2018-01-19T11:05:17Z")
I have a field containing simple tags for example. I'd like to be able to use those tags to perform a faceted search but have the tag counters showing the counts of matches before the tag is used as a filter. e.g. Foo … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1820) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1822)