# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=895 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 896 --- ## [X-PACK DLS and caching](https://discuss.elastic.co/t/x-pack-dls-and-caching/236367)
**Author:** [@Aharon](https://discuss.elastic.co/u/Aharon)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 2:47pm UTC](https://discuss.elastic.co/t/x-pack-dls-and-caching/236367 "2020-06-09T14:47:34Z")
Sometime around ES 7.3 or 7.4 a fix was applied that stopped DLS caching in Fixed Bitsets from growing out of control for large DLS deployments (https://github.com/elastic/elasticsearch/commit/bb130f554e8908be2d8fbb9bb11… --- ## [Cluster has already maximum shards open](https://discuss.elastic.co/t/cluster-has-already-maximum-shards-open/236345)
**Author:** [@anto\_gian](https://discuss.elastic.co/u/anto_gian)\ **Replies:** 1\ **Last updated:** [June 9, 2020, 2:22pm UTC](https://discuss.elastic.co/t/cluster-has-already-maximum-shards-open/236345 "2020-06-09T14:22:36Z")
I'm using Windows 10 and I'm getting Elasticsearch exception \[type=validation\_exception, reason=Validation Failed: 1: this action would add \[2\] total shards, but this cluster currently has \[1000\]/\[1000\] maximum shards … --- ## [Extracting data without saved \_source](https://discuss.elastic.co/t/extracting-data-without-saved-source/236342)
**Author:** [@kosmakoff](https://discuss.elastic.co/u/kosmakoff)\ **Replies:** 1\ **Last updated:** [June 9, 2020, 2:07pm UTC](https://discuss.elastic.co/t/extracting-data-without-saved-source/236342 "2020-06-09T14:07:45Z")
Say I need to index documents containing sensitive information, which I would only like to search by, but not return in search results. I learned that I can disable the \_source in index, so that nothing in document is re… --- ## [Hide a document from the searching](https://discuss.elastic.co/t/hide-a-document-from-the-searching/236313)
**Author:** [@gitterhh](https://discuss.elastic.co/u/gitterhh)\ **Replies:** 4\ **Last updated:** [June 9, 2020, 1:03pm UTC](https://discuss.elastic.co/t/hide-a-document-from-the-searching/236313 "2020-06-09T13:03:41Z")
We want to use Elasticsearch as primary data source, and we also want to keep track of changes of document, so we want to store all the versions of a document. As a solution will be to hide previous document versions fr… --- ## [Installation on a single Windows Server](https://discuss.elastic.co/t/installation-on-a-single-windows-server/236335)
**Author:** [@nodes](https://discuss.elastic.co/u/nodes)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 12:47pm UTC](https://discuss.elastic.co/t/installation-on-a-single-windows-server/236335 "2020-06-09T12:47:39Z")
Hey guys - I was recently put on to the elastic search for log collection and monitoring. Seeing as though I'm the only sysadmin in the company, I'd like to be able to cut down on the time it takes me to run through lo… --- ## [Importance of shard sizing for search performance](https://discuss.elastic.co/t/importance-of-shard-sizing-for-search-performance/236034)
**Author:** [@magnalite](https://discuss.elastic.co/u/magnalite)\ **Replies:** 2\ **Last updated:** [June 9, 2020, 11:44am UTC](https://discuss.elastic.co/t/importance-of-shard-sizing-for-search-performance/236034 "2020-06-09T11:44:16Z")
I'm currently in the process of designing/deploying a cluster which needs to handle \>2B docs/day and keep these docs for at least 3 months. This means an overall capacity of ~180B docs which can be searched at any moment… --- ## [Help writing a test for a custom plugin in ES](https://discuss.elastic.co/t/help-writing-a-test-for-a-custom-plugin-in-es/236316)
**Author:** [@rahulkarajgikar](https://discuss.elastic.co/u/rahulkarajgikar)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 10:52am UTC](https://discuss.elastic.co/t/help-writing-a-test-for-a-custom-plugin-in-es/236316 "2020-06-09T10:52:45Z")
Hi everyone, I'm trying to create a test for a plugin that should be able to do these 3 things: Start a cluster(can have one node) with my plugin installed on each node. Customize the elasticsearch.yml for the nodes i… --- ## [Ingest Processor Conditional](https://discuss.elastic.co/t/ingest-processor-conditional/236311)
**Author:** [@Atul\_Gunjal](https://discuss.elastic.co/u/Atul_Gunjal)\ **Replies:** 1\ **Last updated:** [June 9, 2020, 10:53am UTC](https://discuss.elastic.co/t/ingest-processor-conditional/236311 "2020-06-09T10:53:25Z")
I want to process data in the message field from a source if fields.log\_type is combinedlogs...but it's not working. Can someone help me there? Thanks in Advance. Ingest Pipeline: PUT \_ingest/pipeline/pipeline\_combine… --- ## [How Set Setting.Builder in RestHighLevelClient Elastic 7.6 like in TransportClient and preBuildTransportClient](https://discuss.elastic.co/t/how-set-setting-builder-in-resthighlevelclient-elastic-7-6-like-in-transportclient-and-prebuildtransportclient/236315)
**Author:** [@saba\_safavi](https://discuss.elastic.co/u/saba_safavi)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 10:50am UTC](https://discuss.elastic.co/t/how-set-setting-builder-in-resthighlevelclient-elastic-7-6-like-in-transportclient-and-prebuildtransportclient/236315 "2020-06-09T10:50:21Z")
I want to migrate from Elastic 5.6 to 7.6 we can set some settings in Settings.Builder setting = Settings.builder(); like node.name , cluster.name and some SSL setting but I can find neither in Java Rest Client api nor i… --- ## [Mapper parser exception when creating an index with mapping](https://discuss.elastic.co/t/mapper-parser-exception-when-creating-an-index-with-mapping/235833)
**Author:** [@Sesha\_Sai\_Suhash\_Des](https://discuss.elastic.co/u/Sesha_Sai_Suhash_Des)\ **Replies:** 3\ **Last updated:** [June 9, 2020, 10:03am UTC](https://discuss.elastic.co/t/mapper-parser-exception-when-creating-an-index-with-mapping/235833 "2020-06-09T10:03:26Z")
I am getting error when running the following command. I am a noob please help me. command : PUT /companydatabase?pretty { "mappings": { "employees": { "properties": { "FirstName… --- ## [Timeout when creating S3 repository (but success)](https://discuss.elastic.co/t/timeout-when-creating-s3-repository-but-success/235905)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 1\ **Last updated:** [June 8, 2020, 8:26am UTC](https://discuss.elastic.co/t/timeout-when-creating-s3-repository-but-success/235905 "2020-06-08T08:26:54Z")
Using elasticsearch (with basic licence) v7.7.0, on kubernetes (3 master / 2 data nodes). Other side, we have a minio server (S3 like). When creating a snapshot repository: PUT \_snapshot/unsecure\_minio { "type": "s3… --- ## [Indexing word, pdf documents?](https://discuss.elastic.co/t/indexing-word-pdf-documents/235758)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 11\ **Last updated:** [June 9, 2020, 7:56am UTC](https://discuss.elastic.co/t/indexing-word-pdf-documents/235758 "2020-06-09T07:56:20Z")
Can someone please guide me to a step-by-step documentation to index a word or pdf document in elasticsearch ?? I have gone through couple of posts on this and came across FS crawler etc. I would like to know if there … --- ## [CreateSnapshotRequest in High Level Rest Client has a problem](https://discuss.elastic.co/t/createsnapshotrequest-in-high-level-rest-client-has-a-problem/236273)
**Author:** [@TheDevLuffy](https://discuss.elastic.co/u/TheDevLuffy)\ **Replies:** 1\ **Last updated:** [June 9, 2020, 7:22am UTC](https://discuss.elastic.co/t/createsnapshotrequest-in-high-level-rest-client-has-a-problem/236273 "2020-06-09T07:22:58Z")
Hello, Now, I'm using Amazon Elasticsearch Service with High Level Rest Client. Firstable, I tried to create snapshot with CreateSnapshotRequest. This trial has request body (following code block) according to http lo… --- ## [JAVA\_HOME error at ES start, Ubuntu](https://discuss.elastic.co/t/java-home-error-at-es-start-ubuntu/236266)
**Author:** [@bajg](https://discuss.elastic.co/u/bajg)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 5:31am UTC](https://discuss.elastic.co/t/java-home-error-at-es-start-ubuntu/236266 "2020-06-09T05:31:53Z")
I know that there is some topic about it already, but still can't fix it. I used this tutorial to install ES6 I got "JAVA\_HOME" error so before exported JAVA\_HOME . bitnami@bitnami-magento-b4b5:/$ export JAVA\_HOM… --- ## [No result found in dashbaord](https://discuss.elastic.co/t/no-result-found-in-dashbaord/236258)
**Author:** [@chairxiao](https://discuss.elastic.co/u/chairxiao)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 3:42am UTC](https://discuss.elastic.co/t/no-result-found-in-dashbaord/236258 "2020-06-09T03:42:51Z")
Hello, I use filebeat to collect IIS logs and import them into elk. Error logs have been displayed in discover over the past two hours, but no data is displayed in the dashboard charts. --- ## [Querying shingles](https://discuss.elastic.co/t/querying-shingles/236233)
**Author:** [@chari](https://discuss.elastic.co/u/chari)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 8:58pm UTC](https://discuss.elastic.co/t/querying-shingles/236233 "2020-06-08T20:58:59Z")
I have used the following settings to index content and generate bigrams. es.indices.create( index= "shingles", body= { "settings": { "analysis": { "analyzer": { "shingle\_analyzer": { … --- ## [Complex JSON structure - Mapping](https://discuss.elastic.co/t/complex-json-structure-mapping/236229)
**Author:** [@Lemonlemmings](https://discuss.elastic.co/u/Lemonlemmings)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 8:27pm UTC](https://discuss.elastic.co/t/complex-json-structure-mapping/236229 "2020-06-08T20:27:31Z")
Hi, I'm looking at using elasticsearch for a project I'm working on. I think it'll be a good fit because I want to use elasticsearch as a place to aggregate lots of different data sources to help centralise my data. I'm… --- ## [ElasticSearch docker container fails (FileRun documentation followed)](https://discuss.elastic.co/t/elasticsearch-docker-container-fails-filerun-documentation-followed/232633)
**Author:** [@zilexa](https://discuss.elastic.co/u/zilexa)\ **Replies:** 2\ **Last updated:** [May 14, 2020, 12:57pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-fails-filerun-documentation-followed/232633 "2020-05-14T12:57:39Z")
Following the documentation of FileRun: https://docs.filerun.com/docker-tika This is my docker-compose.yml, identical to the documentation. I am using Traefik as SSL reverse proxy but I have removed the labels from the … --- ## [Find a document with query and modify a string in the message field](https://discuss.elastic.co/t/find-a-document-with-query-and-modify-a-string-in-the-message-field/236222)
**Author:** [@cotjoey](https://discuss.elastic.co/u/cotjoey)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 7:47pm UTC](https://discuss.elastic.co/t/find-a-document-with-query-and-modify-a-string-in-the-message-field/236222 "2020-06-08T19:47:57Z")
Hello, I have some filebeat events which were processed by Logstash and output in an Elasticsearch index. The documents in the index have a "message" field which contain a string that I need to modify. Is there a comma… --- ## [Elastic search azure](https://discuss.elastic.co/t/elastic-search-azure/236221)
**Author:** [@geethu\_priya](https://discuss.elastic.co/u/geethu_priya)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 7:20pm UTC](https://discuss.elastic.co/t/elastic-search-azure/236221 "2020-06-08T19:20:30Z")
HI, Am trying to snapshot the index data from on prem to azure am getting the same exception { "error": { "root\_cause": \[ { "type": "repository\_verification\_exception", "reason": "\[qarepository\] path is not acces… --- ## [Trouble getting an new node added to a cluster](https://discuss.elastic.co/t/trouble-getting-an-new-node-added-to-a-cluster/234814)
**Author:** [@Andreas\_Falk](https://discuss.elastic.co/u/Andreas_Falk)\ **Replies:** 2\ **Last updated:** [June 8, 2020, 7:16pm UTC](https://discuss.elastic.co/t/trouble-getting-an-new-node-added-to-a-cluster/234814 "2020-06-08T19:16:00Z")
Hi, When trying to add the new node to the cluster it fails on bootstrapcheck. \[2020-05-28T20:05:45,054\]\[INFO \]\[o.e.t.TransportService \] \[log04\] publish\_address {192.168.2.194:9300}, bound\_addresses {192.168.2.194:93… --- ## [Developers would like to see some log files from they applications](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814)
**Author:** [@jacobzh](https://discuss.elastic.co/u/jacobzh)\ **Replies:** 8\ **Last updated:** [June 8, 2020, 5:26pm UTC](https://discuss.elastic.co/t/developers-would-like-to-see-some-log-files-from-they-applications/235814 "2020-06-08T17:26:26Z")
I created elk project(elasticsearch, logstash, kibana and filebeat) for production environment. It’s contained 1 server and 5 filebeat clients. Our developers would like to see some log files from they applications. I us… --- ## [Rescore from Page 2 only](https://discuss.elastic.co/t/rescore-from-page-2-only/236203)
**Author:** [@Nagach](https://discuss.elastic.co/u/Nagach)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 4:20pm UTC](https://discuss.elastic.co/t/rescore-from-page-2-only/236203 "2020-06-08T16:20:27Z")
Can re-scoring be done from page2 only. Use case is we are trying to learn user interests in page 1 and want to apply re-scoring basing on user interest and behavior based off of page 1 clicks and apply to page 2..n wit… --- ## [Using shingle and stop filters](https://discuss.elastic.co/t/using-shingle-and-stop-filters/236067)
**Author:** [@chari](https://discuss.elastic.co/u/chari)\ **Replies:** 1\ **Last updated:** [June 8, 2020, 4:17pm UTC](https://discuss.elastic.co/t/using-shingle-and-stop-filters/236067 "2020-06-08T16:17:48Z")
I am using shingle and stop filters, trim and filler\_token="". When i run analyze API on a string : "Word1 Word2 StopWord1 StopWord2 Word3 Word4", I get the following SHINGLES which seem to be correct. i) "Word1 Word2… --- ## [Doubts on Huge Elastic Clustering and HA](https://discuss.elastic.co/t/doubts-on-huge-elastic-clustering-and-ha/235539)
**Author:** [@Adrian\_Varona](https://discuss.elastic.co/u/Adrian_Varona)\ **Replies:** 3\ **Last updated:** [June 8, 2020, 4:11pm UTC](https://discuss.elastic.co/t/doubts-on-huge-elastic-clustering-and-ha/235539 "2020-06-08T16:11:22Z")
Hello mates. Today I got involved in a project in which we are supposed to provide high availability on an elasticsearch cluster. This is going to be a huge project, meaning this that resources are not a problem (We are… --- ## [Aggregate over bucket result](https://discuss.elastic.co/t/aggregate-over-bucket-result/236201)
**Author:** [@Johnny2Chen](https://discuss.elastic.co/u/Johnny2Chen)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 3:35pm UTC](https://discuss.elastic.co/t/aggregate-over-bucket-result/236201 "2020-06-08T15:35:26Z")
I want to aggregate or count the result in the bucket. for example: { ID: 1, customer\_name: a, age: 21, other\_field: x }, { ID: 2, customer\_name: a, age: 25, other\_fie… --- ## [Generate alerts using Elastalert for logs & write the alerts to a New Index name "abc" in elasticsearch](https://discuss.elastic.co/t/generate-alerts-using-elastalert-for-logs-write-the-alerts-to-a-new-index-name-abc-in-elasticsearch/236176)
**Author:** [@Dragon9](https://discuss.elastic.co/u/Dragon9)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 12:51pm UTC](https://discuss.elastic.co/t/generate-alerts-using-elastalert-for-logs-write-the-alerts-to-a-new-index-name-abc-in-elasticsearch/236176 "2020-06-08T12:51:01Z")
my usecase is to send alerts in email and also write the alerts in elasticsearch index. --- ## [Custom weights for fields, custom score](https://discuss.elastic.co/t/custom-weights-for-fields-custom-score/236191)
**Author:** [@Alex\_D](https://discuss.elastic.co/u/Alex_D)\ **Replies:** 0\ **Last updated:** [June 8, 2020, 2:40pm UTC](https://discuss.elastic.co/t/custom-weights-for-fields-custom-score/236191 "2020-06-08T14:40:38Z")
Hi all ! I'm trying to make a query with a custom weights for fields and custom result score. Suppose we have 3 fields: field1 : Text, field2 : Text, phrase : Text. Each field must have a different weight in the res… --- ## [Advice on Using Joins](https://discuss.elastic.co/t/advice-on-using-joins/235966)
**Author:** [@denmcktr](https://discuss.elastic.co/u/denmcktr)\ **Replies:** 2\ **Last updated:** [June 8, 2020, 1:34pm UTC](https://discuss.elastic.co/t/advice-on-using-joins/235966 "2020-06-08T13:34:13Z")
Starting a new project (on 7.7) and trying to decide if the join framework (parent/child) makes sense as an approach or not. A simplified model of what I'm building is as follows: A house (the parent type) House membe… --- ## [Ingest pipeline for kv split value](https://discuss.elastic.co/t/ingest-pipeline-for-kv-split-value/235215)
**Author:** [@inf2ravikumar](https://discuss.elastic.co/u/inf2ravikumar)\ **Replies:** 4\ **Last updated:** [June 8, 2020, 12:45pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-kv-split-value/235215 "2020-06-08T12:45:22Z")
Hi There I'm trying to create ingest pipeline for KV Ingest Processor splitting filed parsing not sure where I'm wrong any help would appreciate it POST /\_ingest/pipeline/\_simulate { "pipeline": { "description"… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=894) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=896)