# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1567 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1568 --- ## [Snapshot failing for 1 shard](https://discuss.elastic.co/t/snapshot-failing-for-1-shard/263446)
**Author:** [@ParashB](https://discuss.elastic.co/u/ParashB)\ **Replies:** 3\ **Last updated:** [February 8, 2021, 11:44am UTC](https://discuss.elastic.co/t/snapshot-failing-for-1-shard/263446 "2021-02-08T11:44:46Z")
Hi Guys, I am using SLM to create snapshots but its failing to snapshot 1 of the shards consistently. The shard is 120GB in size but I have been able to snapshot such large shards for other indices successfully. I can't… --- ## [Kibana Bulk Create API Fails To Overwrite on 7.10 returning 409](https://discuss.elastic.co/t/kibana-bulk-create-api-fails-to-overwrite-on-7-10-returning-409/262123)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 5\ **Last updated:** [February 8, 2021, 11:21am UTC](https://discuss.elastic.co/t/kibana-bulk-create-api-fails-to-overwrite-on-7-10-returning-409/262123 "2021-02-08T11:21:20Z")
I have been using various scripts to create dashboards and visualizations from a template successfully on 7.5. When I try the same script on 7.10 it fails, and every visualization returns a 409 conflict error. I'm post… --- ## [Harvester not harvesting Autodiscover](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392)
**Author:** [@Alexis\_Oviedo](https://discuss.elastic.co/u/Alexis_Oviedo)\ **Replies:** 4\ **Last updated:** [February 8, 2021, 11:10am UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392 "2021-02-08T11:10:24Z")
I'm trying to use filebeat docker but it's not harvesting files. Autodiscover seems to work because found the logs I want but it's not harvesting them. This is my filebeat.yml: filebeat.autodiscover: providers: - t… --- ## [Limit in fetching documents through rest API](https://discuss.elastic.co/t/limit-in-fetching-documents-through-rest-api/263551)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 3\ **Last updated:** [February 8, 2021, 10:32am UTC](https://discuss.elastic.co/t/limit-in-fetching-documents-through-rest-api/263551 "2021-02-08T10:32:10Z")
Hi All, I'm trying to fetch documents from elasticsearch using rest api but it looks like i can pull down only limited set of documents, below is the query i'm using and i'm getting very limited documents eventhough my … --- ## [Beats Developer Guide \[master\] is out of date, creating new beat fails](https://discuss.elastic.co/t/beats-developer-guide-master-is-out-of-date-creating-new-beat-fails/263156)
**Author:** [@jackemuk-es](https://discuss.elastic.co/u/jackemuk-es)\ **Replies:** 5\ **Last updated:** [February 8, 2021, 10:10am UTC](https://discuss.elastic.co/t/beats-developer-guide-master-is-out-of-date-creating-new-beat-fails/263156 "2021-02-08T10:10:23Z")
There are errors in the documentation to get started with creating your own beats. If you search in the history, they have been around for some time and haven't been updated. This makes it extremely difficult to create a… --- ## [Logstash doesn't auto reconnect to another node ES cluster](https://discuss.elastic.co/t/logstash-doesnt-auto-reconnect-to-another-node-es-cluster/263568)
**Author:** [@lenjeti](https://discuss.elastic.co/u/lenjeti)\ **Replies:** 0\ **Last updated:** [February 8, 2021, 9:41am UTC](https://discuss.elastic.co/t/logstash-doesnt-auto-reconnect-to-another-node-es-cluster/263568 "2021-02-08T09:41:19Z")
We have a 3 node elasticsearch cluster set up. The logstash ouput config looks like this: output { elasticsearch { hosts =\> \["node1:9200", "node2:9200", "node3:9200" … --- ## [Elasticsearch join query](https://discuss.elastic.co/t/elasticsearch-join-query/263567)
**Author:** [@diya5](https://discuss.elastic.co/u/diya5)\ **Replies:** 0\ **Last updated:** [February 8, 2021, 9:36am UTC](https://discuss.elastic.co/t/elasticsearch-join-query/263567 "2021-02-08T09:36:11Z")
I have two indices. One indices "indications" which have some set of values. Other is "projects". In this indices i will add indications value like " indication = oncology". Now i want to show all indications. Which i… --- ## [Search for fields with dot in name](https://discuss.elastic.co/t/search-for-fields-with-dot-in-name/260692)
**Author:** [@sudpaw](https://discuss.elastic.co/u/sudpaw)\ **Replies:** 6\ **Last updated:** [February 8, 2021, 9:35am UTC](https://discuss.elastic.co/t/search-for-fields-with-dot-in-name/260692 "2021-02-08T09:35:19Z")
Hi, We are migrating from Es6.8 to latest ES7. We are using Re-index. While testing we came across a doc with a malformed fieldname. (the mapping does not allow for the field, but it is there). Example: { "id": "1… --- ## [Filebeat only operating at 10%-15% of expected Performance](https://discuss.elastic.co/t/filebeat-only-operating-at-10-15-of-expected-performance/262865)
**Author:** [@svbernem](https://discuss.elastic.co/u/svbernem)\ **Replies:** 3\ **Last updated:** [February 8, 2021, 6:56am UTC](https://discuss.elastic.co/t/filebeat-only-operating-at-10-15-of-expected-performance/262865 "2021-02-08T06:56:34Z")
Hi, I have invested quite some time profiling configurations of filebeat (7.6.2) in the past few days, since it is currently running very slow. My test dataset consists of ~2GB ndjson files, at 10 MB per file, with ~6 m… --- ## [Custom UI Changes on local machine(server machine) is not reflecting on external IP](https://discuss.elastic.co/t/custom-ui-changes-on-local-machine-server-machine-is-not-reflecting-on-external-ip/262458)
**Author:** [@Rakesh\_Rayabarapu](https://discuss.elastic.co/u/Rakesh_Rayabarapu)\ **Replies:** 14\ **Last updated:** [February 8, 2021, 9:15am UTC](https://discuss.elastic.co/t/custom-ui-changes-on-local-machine-server-machine-is-not-reflecting-on-external-ip/262458 "2021-02-08T09:15:14Z")
Hi Team, I made custom logo changes in these files "template.js", security.chunk.4.js" and "core.entry.js" and generated the file (gz and br) with npx. \[Steps from this post - https://discuss.elastic.co/t/custom-logo-k… --- ## [Elastic Cloud Integration with FusionAuth](https://discuss.elastic.co/t/elastic-cloud-integration-with-fusionauth/263323)
**Author:** [@vexana](https://discuss.elastic.co/u/vexana)\ **Replies:** 7\ **Last updated:** [February 8, 2021, 8:51am UTC](https://discuss.elastic.co/t/elastic-cloud-integration-with-fusionauth/263323 "2021-02-08T08:51:26Z")
Hi Everyone, I am trying to using FusionAuth (https://fusionauth.io/) as IdP (OpenID or SAML) to my elastic stack running on Elastic Cloud. Here is my elasticsearch.yaml configuration: xpack: security: authc: … --- ## [Match\_phrase is not working with keep\_words\_filter in percolator index](https://discuss.elastic.co/t/match-phrase-is-not-working-with-keep-words-filter-in-percolator-index/263557)
**Author:** [@Dahea\_Moon](https://discuss.elastic.co/u/Dahea_Moon)\ **Replies:** 0\ **Last updated:** [February 8, 2021, 8:27am UTC](https://discuss.elastic.co/t/match-phrase-is-not-working-with-keep-words-filter-in-percolator-index/263557 "2021-02-08T08:27:55Z")
Hi, elastic team. I'm working on percolator index with match\_phrase query and keep\_words\_filter. I found that match\_phrase query is not working properly with keep\_words\_filter. I've tried both match\_phrase query and s… --- ## [Kibana Query DSL Sorting and size does not work](https://discuss.elastic.co/t/kibana-query-dsl-sorting-and-size-does-not-work/263078)
**Author:** [@Sudarshan\_Reddy](https://discuss.elastic.co/u/Sudarshan_Reddy)\ **Replies:** 4\ **Last updated:** [February 8, 2021, 7:14am UTC](https://discuss.elastic.co/t/kibana-query-dsl-sorting-and-size-does-not-work/263078 "2021-02-08T07:14:27Z")
Hi , how to make sorting works in kibana Query DSL . i am using the following script. --- ## [Whether to allow the profile to be opened on every query request](https://discuss.elastic.co/t/whether-to-allow-the-profile-to-be-opened-on-every-query-request/263540)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 12\ **Last updated:** [February 8, 2021, 7:05am UTC](https://discuss.elastic.co/t/whether-to-allow-the-profile-to-be-opened-on-every-query-request/263540 "2021-02-08T07:05:29Z")
I want to turn on profile for each elasticsearch query request in my system, so that I can get the profile results to print the logs, combined with traceId, I can get what the reason for each slow request is, but I am wo… --- ## [How can use 'org.elasticsearch.spark.sql' with kerberos to read es data](https://discuss.elastic.co/t/how-can-use-org-elasticsearch-spark-sql-with-kerberos-to-read-es-data/263548)
**Author:** [@wenyangchou](https://discuss.elastic.co/u/wenyangchou)\ **Replies:** 0\ **Last updated:** [February 8, 2021, 6:39am UTC](https://discuss.elastic.co/t/how-can-use-org-elasticsearch-spark-sql-with-kerberos-to-read-es-data/263548 "2021-02-08T06:39:22Z")
I have tried use es-hadoop to read a elasticsearch which had configured kerberos. I have a reference to this https://www.elastic.co/guide/en/elasticsearch/hadoop/7.x/kerberos.html but not success. Is there any example I … --- ## [How to add nested datatype using Java api?](https://discuss.elastic.co/t/how-to-add-nested-datatype-using-java-api/263542)
**Author:** [@rohit.singh3](https://discuss.elastic.co/u/rohit.singh3)\ **Replies:** 0\ **Last updated:** [February 8, 2021, 5:20am UTC](https://discuss.elastic.co/t/how-to-add-nested-datatype-using-java-api/263542 "2021-02-08T05:20:44Z")
Below is the sample snippet: "price":{ "type":"nested", --\> how to add this nested type? "properties":{ "activity\_price":{ "type":"double" }, … --- ## [How calculate session duration script field](https://discuss.elastic.co/t/how-calculate-session-duration-script-field/263313)
**Author:** [@ITchandj](https://discuss.elastic.co/u/ITchandj)\ **Replies:** 1\ **Last updated:** [February 8, 2021, 5:24am UTC](https://discuss.elastic.co/t/how-calculate-session-duration-script-field/263313 "2021-02-08T05:24:06Z")
Hi @Havana i am new to kibana and i need to create exactly one table that gives me the min and max of a session. please can you tell me how you did it. i would be very happy and thank you in advance --- ## [Help please.. to acchive json format](https://discuss.elastic.co/t/help-please-to-acchive-json-format/263330)
**Author:** [@onkarborade](https://discuss.elastic.co/u/onkarborade)\ **Replies:** 2\ **Last updated:** [February 8, 2021, 5:06am UTC](https://discuss.elastic.co/t/help-please-to-acchive-json-format/263330 "2021-02-08T05:06:05Z")
I have rosterentry field as bellow rosterentry = "100,01-JUL-20,01-AUG-22|100,02-JUL-20,02-AUG-22|101,03-JUL-19,03-AUG-19" I have converted to : rosterentry = \[ "100,01-JUL-20,01-AUG-22", … --- ## [X-axis is not visible in vega kibana while calling through index](https://discuss.elastic.co/t/x-axis-is-not-visible-in-vega-kibana-while-calling-through-index/263088)
**Author:** [@saumya1](https://discuss.elastic.co/u/saumya1)\ **Replies:** 2\ **Last updated:** [February 8, 2021, 4:59am UTC](https://discuss.elastic.co/t/x-axis-is-not-visible-in-vega-kibana-while-calling-through-index/263088 "2021-02-08T04:59:47Z")
{ "$schema": "https://vega.github.io/schema/vega/v5.json", "autosize":"none", "height":"350", "width":"300", "padding":"30" "title": { "text": "Sample incident data", "subtitle": "timeunit test", "subtitleFontSt… --- ## [Shrink shards not respecting node settings 7.10.0](https://discuss.elastic.co/t/shrink-shards-not-respecting-node-settings-7-10-0/262883)
**Author:** [@nz-1](https://discuss.elastic.co/u/nz-1)\ **Replies:** 1\ **Last updated:** [February 8, 2021, 4:49am UTC](https://discuss.elastic.co/t/shrink-shards-not-respecting-node-settings-7-10-0/262883 "2021-02-08T04:49:44Z")
I have a 5 node cluster with 4 nodes as the data\_warm, data\_hot and 1 node as the cold storage. When ES goes to shrink the node I see on some of the indexes it chooses the cold storage node (dx6MirFmT2y9AdXEF02pDw) whic… --- ## [Transforming into JSON-Documents](https://discuss.elastic.co/t/transforming-into-json-documents/262884)
**Author:** [@Werner](https://discuss.elastic.co/u/Werner)\ **Replies:** 9\ **Last updated:** [February 8, 2021, 4:48am UTC](https://discuss.elastic.co/t/transforming-into-json-documents/262884 "2021-02-08T04:48:33Z")
Hello ! I am an absolute beginner in elasticsearch and after some hours of reading an watching tutorial-videos I got the impression, that it needs a lot of knowledge and preparation, before I can make a fast search. In… --- ## [Questions on getting started with ELK](https://discuss.elastic.co/t/questions-on-getting-started-with-elk/263194)
**Author:** [@Anita2](https://discuss.elastic.co/u/Anita2)\ **Replies:** 3\ **Last updated:** [February 8, 2021, 4:46am UTC](https://discuss.elastic.co/t/questions-on-getting-started-with-elk/263194 "2021-02-08T04:46:28Z")
For reference, I have followed this guide to get me started: Hello, I'm extremely new to ELK, as in I have just now installed everything, as I will be needing it for a research project. My current goal is to be able t… --- ## [What does this error mean, and would it cause a data node to crash](https://discuss.elastic.co/t/what-does-this-error-mean-and-would-it-cause-a-data-node-to-crash/263173)
**Author:** [@cis4life](https://discuss.elastic.co/u/cis4life)\ **Replies:** 7\ **Last updated:** [February 8, 2021, 4:45am UTC](https://discuss.elastic.co/t/what-does-this-error-mean-and-would-it-cause-a-data-node-to-crash/263173 "2021-02-08T04:45:21Z")
I have a cluster on ES 5.x and suddenly some of the data nodes simply crash randomly. I see this error and want to know what it is and is this something I'm overlooking to fix it??? Any insight would be GREATLY appreci… --- ## [How to pass a variable to elasticsearch query javascript](https://discuss.elastic.co/t/how-to-pass-a-variable-to-elasticsearch-query-javascript/263536)
**Author:** [@adonnini](https://discuss.elastic.co/u/adonnini)\ **Replies:** 0\ **Last updated:** [February 8, 2021, 3:07am UTC](https://discuss.elastic.co/t/how-to-pass-a-variable-to-elasticsearch-query-javascript/263536 "2021-02-08T03:07:13Z")
Hi, I am a nebie to ElasticSearch. I have searched for an answer to this question before making this post. I am trying to run an ElasticSearch query from a JavaScript app using a variable for the search string. I have … --- ## [Format metricbeat field value in watcher alert](https://discuss.elastic.co/t/format-metricbeat-field-value-in-watcher-alert/263306)
**Author:** [@twilight](https://discuss.elastic.co/u/twilight)\ **Replies:** 6\ **Last updated:** [February 8, 2021, 3:05am UTC](https://discuss.elastic.co/t/format-metricbeat-field-value-in-watcher-alert/263306 "2021-02-08T03:05:23Z")
Hello, I am using a number of watcher with various conditions and alert to be sent via email and slack. I am receiving some values in the alert which are not easily readable e.g. disk free space which is sent by metric… --- ## [If else condition with logstash filter](https://discuss.elastic.co/t/if-else-condition-with-logstash-filter/263517)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [February 7, 2021, 3:24pm UTC](https://discuss.elastic.co/t/if-else-condition-with-logstash-filter/263517 "2021-02-07T15:24:13Z")
Hi Folks, I am trying to achive certain conditions with logstash and wondering \\if - if-else statement or validation is possible in Logstash filter? I have below messages in json and parsing DMARC logs with logstash. I… --- ## [Get subset of values that exist for a given search](https://discuss.elastic.co/t/get-subset-of-values-that-exist-for-a-given-search/263523)
**Author:** [@cozzbp](https://discuss.elastic.co/u/cozzbp)\ **Replies:** 0\ **Last updated:** [February 7, 2021, 2:25pm UTC](https://discuss.elastic.co/t/get-subset-of-values-that-exist-for-a-given-search/263523 "2021-02-07T14:25:20Z")
Is it possible to query for which values of a given set are represented/not represented in the data set? If I have documents that look like: { "category": "fiction", "tags": \["horror", "thriller"\] }… --- ## [Force Read few files from start when filebeat restarts](https://discuss.elastic.co/t/force-read-few-files-from-start-when-filebeat-restarts/263362)
**Author:** [@rsgupta0110](https://discuss.elastic.co/u/rsgupta0110)\ **Replies:** 2\ **Last updated:** [February 7, 2021, 1:49pm UTC](https://discuss.elastic.co/t/force-read-few-files-from-start-when-filebeat-restarts/263362 "2021-02-07T13:49:48Z")
Filebeat maintains the registry and just uploads new logs between filebeat restarts. Is there a way to specify to read few log files from beginning everytime filebeat restarts while maintaining the registry for other fil… --- ## [How to monitor a file for its version, code signature etc](https://discuss.elastic.co/t/how-to-monitor-a-file-for-its-version-code-signature-etc/263178)
**Author:** [@rsgupta0110](https://discuss.elastic.co/u/rsgupta0110)\ **Replies:** 2\ **Last updated:** [February 7, 2021, 1:29pm UTC](https://discuss.elastic.co/t/how-to-monitor-a-file-for-its-version-code-signature-etc/263178 "2021-02-07T13:29:15Z")
I would like to monitor files inside a directory for the file description like version, code signature etc. without uploading the files to Elastic Search. I found this File fields but not sure how to add these fields in… --- ## [Syslog filter for IP](https://discuss.elastic.co/t/syslog-filter-for-ip/263512)
**Author:** [@dandotwalker](https://discuss.elastic.co/u/dandotwalker)\ **Replies:** 3\ **Last updated:** [February 7, 2021, 9:57am UTC](https://discuss.elastic.co/t/syslog-filter-for-ip/263512 "2021-02-07T09:57:01Z")
Hi, I am learning Elastic so I have set it up in my home to prepare for when we start designing it at work. I have a simple set up where logstash is listening for syslog messages. It works great with most hosts but my … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1566) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1568)