# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1375 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1376 --- ## [Don't return whole BASE64 encoded files (ingest plugin)](https://discuss.elastic.co/t/dont-return-whole-base64-encoded-files-ingest-plugin/170949)
**Author:** [@Oscar\_Sanchez\_Sanche](https://discuss.elastic.co/u/Oscar_Sanchez_Sanche)\ **Replies:** 1\ **Last updated:** [March 5, 2019, 4:50pm UTC](https://discuss.elastic.co/t/dont-return-whole-base64-encoded-files-ingest-plugin/170949 "2019-03-05T16:50:37Z")
Hi, I've been reading a couple about how the ingest plugin works. So far this works correctly for me when searching for words inside the PDF files, however, I noticed a problem with large files. Let's say I search for … --- ## [Watcher alert for an scripted field](https://discuss.elastic.co/t/watcher-alert-for-an-scripted-field/169518)
**Author:** [@Alvaro\_Garrido](https://discuss.elastic.co/u/Alvaro_Garrido)\ **Replies:** 2\ **Last updated:** [March 5, 2019, 4:38pm UTC](https://discuss.elastic.co/t/watcher-alert-for-an-scripted-field/169518 "2019-03-05T16:38:12Z")
Is it possible to set up a Watcher alert for an scripted field overcoming a threshold value? For example, when latency (scripted field for endTime - startTime) overcomes 500 ms, send me an alert --- ## [How to get list of indices which are not updating from past 1 month](https://discuss.elastic.co/t/how-to-get-list-of-indices-which-are-not-updating-from-past-1-month/170950)
**Author:** [@prabhakar\_talari](https://discuss.elastic.co/u/prabhakar_talari)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 4:26pm UTC](https://discuss.elastic.co/t/how-to-get-list-of-indices-which-are-not-updating-from-past-1-month/170950 "2019-03-05T16:26:42Z")
Hi All, Could any one provide query to get list of indices for which the maxtime stamp is olderthan 30 days from now. Thanks --- ## [OR search queries using Watchers](https://discuss.elastic.co/t/or-search-queries-using-watchers/169817)
**Author:** [@Bill.S.Preston](https://discuss.elastic.co/u/Bill.S.Preston)\ **Replies:** 1\ **Last updated:** [March 5, 2019, 4:18pm UTC](https://discuss.elastic.co/t/or-search-queries-using-watchers/169817 "2019-03-05T16:18:01Z")
I'm looking for a specific type of event. I have a query that I input into the discovery search that looks similar to below: (event1 OR event2) AND (message1 OR message2 OR message3) AND message10 It works in Discovery… --- ## [Problems with getting/checking map value by given key in painless script](https://discuss.elastic.co/t/problems-with-getting-checking-map-value-by-given-key-in-painless-script/170901)
**Author:** [@Olga\_Gorun](https://discuss.elastic.co/u/Olga_Gorun)\ **Replies:** 1\ **Last updated:** [March 5, 2019, 3:22pm UTC](https://discuss.elastic.co/t/problems-with-getting-checking-map-value-by-given-key-in-painless-script/170901 "2019-03-05T15:22:27Z")
I experience problems with getting/checking map value by given key in painless script. Examples: GET tt/clip/\_search { "query": { "function\_score": { "script\_score": { "script": { "lang": … --- ## [Elastic Function\_Score not applying properly](https://discuss.elastic.co/t/elastic-function-score-not-applying-properly/170660)
**Author:** [@anveshdd](https://discuss.elastic.co/u/anveshdd)\ **Replies:** 3\ **Last updated:** [March 5, 2019, 3:21pm UTC](https://discuss.elastic.co/t/elastic-function-score-not-applying-properly/170660 "2019-03-05T15:21:31Z")
Working on elastic search 6.5.3. I am using function\_score to change the weights of the particular results. I am expecting the output as the results with the highest weight comes first and then remaining follows. Below… --- ## [Why close/reopen index will reset index stats to zero](https://discuss.elastic.co/t/why-close-reopen-index-will-reset-index-stats-to-zero/170830)
**Author:** [@LoadingZhang](https://discuss.elastic.co/u/LoadingZhang)\ **Replies:** 3\ **Last updated:** [March 5, 2019, 3:15pm UTC](https://discuss.elastic.co/t/why-close-reopen-index-will-reset-index-stats-to-zero/170830 "2019-03-05T15:15:30Z")
Now we calculate index useage by index stats api, if close and reopen index, the index stats will reset to 0, such indexing/search/merge metrics. it will bring us misunderstanding. I'm wondering why this operation will … --- ## [Create an alias from index template](https://discuss.elastic.co/t/create-an-alias-from-index-template/170940)
**Author:** [@WassimDhib](https://discuss.elastic.co/u/WassimDhib)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 3:14pm UTC](https://discuss.elastic.co/t/create-an-alias-from-index-template/170940 "2019-03-05T15:14:37Z")
Hi, I have a cluster that receives data from multiple applications Data contains fields like "service.name", for example service.app1 and service.app2 Data for application app1 must go in index named data-app1-yyy… --- ## [Elasticsearch getting very slow](https://discuss.elastic.co/t/elasticsearch-getting-very-slow/170828)
**Author:** [@shikhar\_singh](https://discuss.elastic.co/u/shikhar_singh)\ **Replies:** 1\ **Last updated:** [March 5, 2019, 3:08pm UTC](https://discuss.elastic.co/t/elasticsearch-getting-very-slow/170828 "2019-03-05T15:08:41Z")
what i observed: The kibana dashboards were not getting refreshed. Navigating the kibana UI was very slow. what i observed on the server side . what i did as a work around after clearing the cache my ES/KIbana c… --- ## [SQL “where not exists” equivalent for Elastic](https://discuss.elastic.co/t/sql-where-not-exists-equivalent-for-elastic/170934)
**Author:** [@CraigMc](https://discuss.elastic.co/u/CraigMc)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 2:57pm UTC](https://discuss.elastic.co/t/sql-where-not-exists-equivalent-for-elastic/170934 "2019-03-05T14:57:45Z")
Hello, I have a set of stages I am logging. A GUID is on each record to tie the stage together. Example: Guid Stage 1 Stage 1 1 Stage 2 1 Stage 3 1 Stage 4 1 Stage 5 2 Stage 1 2 Stage 2 2 Stage 3 2 Stage 4 2… --- ## [Add additional Master Nodes to exisiting runnning matsers](https://discuss.elastic.co/t/add-additional-master-nodes-to-exisiting-runnning-matsers/167203)
**Author:** [@jagadeeswar\_Reddy](https://discuss.elastic.co/u/jagadeeswar_Reddy)\ **Replies:** 1\ **Last updated:** [March 5, 2019, 2:37pm UTC](https://discuss.elastic.co/t/add-additional-master-nodes-to-exisiting-runnning-matsers/167203 "2019-03-05T14:37:40Z")
Hi , can you please help us to how to add the additionale master nodes to existing masters , please share the instalation and configuration set up . Thanks Jagadish --- ## [Problem with stopword filter, SimpleQueryStringQuery and default operator AND](https://discuss.elastic.co/t/problem-with-stopword-filter-simplequerystringquery-and-default-operator-and/170930)
**Author:** [@xrammit](https://discuss.elastic.co/u/xrammit)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 2:35pm UTC](https://discuss.elastic.co/t/problem-with-stopword-filter-simplequerystringquery-and-default-operator-and/170930 "2019-03-05T14:35:16Z")
Hi everybody, I have a problem with a Simple Query String Query with default operator AND and language-specific stopword filters. I configured the index to apply the stopword filters on the indexed documents (analyzer c… --- ## [EMAILLOCALPART grok pattern](https://discuss.elastic.co/t/emaillocalpart-grok-pattern/170927)
**Author:** [@kmousikos](https://discuss.elastic.co/u/kmousikos)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 2:14pm UTC](https://discuss.elastic.co/t/emaillocalpart-grok-pattern/170927 "2019-03-05T14:14:35Z")
I'm checking https://github.com/elastic/elasticsearch/blob/6.6/libs/grok/src/main/resources/patterns/grok-patterns and I see that EMAILLOCALPART supposes that the local part of an email must be at least 2 characters long… --- ## [Elasticsearch - Need to improve cluster performance, (sock hang up) happening a lot](https://discuss.elastic.co/t/elasticsearch-need-to-improve-cluster-performance-sock-hang-up-happening-a-lot/170924)
**Author:** [@jatinseth2007](https://discuss.elastic.co/u/jatinseth2007)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 2:02pm UTC](https://discuss.elastic.co/t/elasticsearch-need-to-improve-cluster-performance-sock-hang-up-happening-a-lot/170924 "2019-03-05T14:02:27Z")
We have Elasticsearch cluster of 3 nodes where 1 node is dedicated master and 2 nodes are dedicated data nodes. we have around 200GB of data on server and heavy \_bulk and \_search operations. The nodes configuration is M… --- ## [Active Kibana session error with SAML implemented when trying to login (Solved)](https://discuss.elastic.co/t/active-kibana-session-error-with-saml-implemented-when-trying-to-login-solved/170923)
**Author:** [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 2:02pm UTC](https://discuss.elastic.co/t/active-kibana-session-error-with-saml-implemented-when-trying-to-login-solved/170923 "2019-03-05T14:02:25Z")
Ran across a minor issue logging in as I was getting the dreaded "all shards failed" error when accessing our QA environment via SAML. I tried to login directly by going to each node specific url where I ran into a diff… --- ## [Elasticsearch and logstash performance drop](https://discuss.elastic.co/t/elasticsearch-and-logstash-performance-drop/170918)
**Author:** [@fedwe](https://discuss.elastic.co/u/fedwe)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 1:49pm UTC](https://discuss.elastic.co/t/elasticsearch-and-logstash-performance-drop/170918 "2019-03-05T13:49:21Z")
Hello all im using logstash to index to elasticsearch. I am using batch size 2000 and 16 workers because i have 4 cpus and 4 cores each. I have a problem with performance in logstash .Its not utilizing the full cpu capa… --- ## [Nested fields vs. flat document](https://discuss.elastic.co/t/nested-fields-vs-flat-document/167191)
**Author:** [@SpeedDaemon](https://discuss.elastic.co/u/SpeedDaemon)\ **Replies:** 1\ **Last updated:** [March 5, 2019, 1:36pm UTC](https://discuss.elastic.co/t/nested-fields-vs-flat-document/167191 "2019-03-05T13:36:50Z")
I'm currently creating an index template for network logs. I currently have something like this: "mappings" : { "doc" : { "properties" : { ... "ip\_src" : { "type" : "ip" … --- ## [Linear growth of index size](https://discuss.elastic.co/t/linear-growth-of-index-size/170723)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 10\ **Last updated:** [March 5, 2019, 1:30pm UTC](https://discuss.elastic.co/t/linear-growth-of-index-size/170723 "2019-03-05T13:30:20Z")
To test the growth of index compared to ingested data, we made 1 million calls to insert the same document in an index. We observed linear growth in index size. The query to load the docs was: curl -X POST “http://… --- ## [Querying and Parsing Elasticsearch API in Python](https://discuss.elastic.co/t/querying-and-parsing-elasticsearch-api-in-python/170913)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 1:27pm UTC](https://discuss.elastic.co/t/querying-and-parsing-elasticsearch-api-in-python/170913 "2019-03-05T13:27:33Z")
Hi, Can anyone show me an example of querying the elasticsearch api, such as cat and stats, in python please? All of the examples out there focus on searching indexes... Regards, D --- ## [Trying to use json codec on stdin - switching to json\_lines?](https://discuss.elastic.co/t/trying-to-use-json-codec-on-stdin-switching-to-json-lines/170910)
**Author:** [@djmcgreal-cc](https://discuss.elastic.co/u/djmcgreal-cc)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 1:18pm UTC](https://discuss.elastic.co/t/trying-to-use-json-codec-on-stdin-switching-to-json-lines/170910 "2019-03-05T13:18:10Z")
My interpretation of https://www.elastic.co/guide/en/logstash/6.6/plugins-codecs-json.html is that if I have input { stdin { codec =\> json } } I should be able to pipe in e.g. \[ {"event": 1}, {"event": 2} \] An… --- ## [X Pack license tariff](https://discuss.elastic.co/t/x-pack-license-tariff/170874)
**Author:** [@vasudevan](https://discuss.elastic.co/u/vasudevan)\ **Replies:** 2\ **Last updated:** [March 5, 2019, 12:49pm UTC](https://discuss.elastic.co/t/x-pack-license-tariff/170874 "2019-03-05T12:49:52Z")
We are using 3 nodes cluster setup in our production, I requested x Pack licensing cost but not yet get replied for the same. If some one has info regarding this. Please share ASAP --- ## [Can I upload pdf file directly instead of Base64 data](https://discuss.elastic.co/t/can-i-upload-pdf-file-directly-instead-of-base64-data/170703)
**Author:** [@johnsonjesumani](https://discuss.elastic.co/u/johnsonjesumani)\ **Replies:** 7\ **Last updated:** [March 5, 2019, 12:39pm UTC](https://discuss.elastic.co/t/can-i-upload-pdf-file-directly-instead-of-base64-data/170703 "2019-03-05T12:39:22Z")
Hi, I would like to upload pdf file directly instead of converting the content which inside the pdf to Base64 as the below. PUT \_ingest/pipeline/attachment { "description" : "Extract attachment information", "proces… --- ## [Enabling/Disabling \_source field query](https://discuss.elastic.co/t/enabling-disabling-source-field-query/170105)
**Author:** [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Replies:** 5\ **Last updated:** [March 5, 2019, 12:37pm UTC](https://discuss.elastic.co/t/enabling-disabling-source-field-query/170105 "2019-03-05T12:37:36Z")
I have gone through the merits and problems or disabling the \_source field. But my main query is that if I disable the \_source field then what am I supposed to expect as search result query. When I tried this then all I … --- ## [Boosting fields for multi match without specifying complete field list in query](https://discuss.elastic.co/t/boosting-fields-for-multi-match-without-specifying-complete-field-list-in-query/170768)
**Author:** [@xxviktor](https://discuss.elastic.co/u/xxviktor)\ **Replies:** 4\ **Last updated:** [March 5, 2019, 12:06pm UTC](https://discuss.elastic.co/t/boosting-fields-for-multi-match-without-specifying-complete-field-list-in-query/170768 "2019-03-05T12:06:28Z")
I am trying to boost fields using multi match query without specifying complete field list but I cannot find out how to do it. I am searching through multiple indices on all fields, which I don't know at the run time, bu… --- ## [Pagination of sorted aggregation](https://discuss.elastic.co/t/pagination-of-sorted-aggregation/170890)
**Author:** [@TSaleem](https://discuss.elastic.co/u/TSaleem)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 11:56am UTC](https://discuss.elastic.co/t/pagination-of-sorted-aggregation/170890 "2019-03-05T11:56:51Z")
I am using packetbeat along with elasticsearch and i need to be able to paginate source.mac sorted according to bandwidth("source.stats.net\_bytes\_total" + "dest.stats.net\_bytes\_total"). -Partition won't work because it … --- ## [How i can upgrade elasticsearch 5x to 6x?](https://discuss.elastic.co/t/how-i-can-upgrade-elasticsearch-5x-to-6x/170112)
**Author:** [@Uporaba](https://discuss.elastic.co/u/Uporaba)\ **Replies:** 8\ **Last updated:** [March 5, 2019, 11:39am UTC](https://discuss.elastic.co/t/how-i-can-upgrade-elasticsearch-5x-to-6x/170112 "2019-03-05T11:39:36Z")
Hi all, I have a graylog server with 1.5 billion records. How can I upgrade elasticsearch from version 5 to version 6 without losing all the data that I have now? The instructions on the site are not clear to me. I dont … --- ## [Elasticsearch - return records having a field falling in the given range](https://discuss.elastic.co/t/elasticsearch-return-records-having-a-field-falling-in-the-given-range/170676)
**Author:** [@prakritidev](https://discuss.elastic.co/u/prakritidev)\ **Replies:** 10\ **Last updated:** [March 5, 2019, 11:37am UTC](https://discuss.elastic.co/t/elasticsearch-return-records-having-a-field-falling-in-the-given-range/170676 "2019-03-05T11:37:22Z")
A subset of my data looks like: { "6220":{ "abstract":"We investigate the two-dimensional $\\\\mathcal{ N }=(2, 2)$ supersymmetric\\nYang-Mills (SYM) theory on … --- ## [Analyzing logs by grouping set up logs in same thread](https://discuss.elastic.co/t/analyzing-logs-by-grouping-set-up-logs-in-same-thread/170873)
**Author:** [@Rajesh2](https://discuss.elastic.co/u/Rajesh2)\ **Replies:** 0\ **Last updated:** [March 5, 2019, 11:11am UTC](https://discuss.elastic.co/t/analyzing-logs-by-grouping-set-up-logs-in-same-thread/170873 "2019-03-05T11:11:43Z")
Hi, I have following use case In one req-response cycle i will have 2 log statements like 1. data is served from cache / or not 2. total time taken for the request : 1234 I would like to analyze in kibana, how much … --- ## [Are Ids queries faster than keyword match queries?](https://discuss.elastic.co/t/are-ids-queries-faster-than-keyword-match-queries/169656)
**Author:** [@Savo\_Pejovic](https://discuss.elastic.co/u/Savo_Pejovic)\ **Replies:** 0\ **Last updated:** [February 23, 2019, 11:39am UTC](https://discuss.elastic.co/t/are-ids-queries-faster-than-keyword-match-queries/169656 "2019-02-23T11:39:32Z")
I am considering different index mappings, and I am wondering: is Ids query faster than basic terms query, on a keyword field, which would have the value of the desired Id. Anyone with any experience with this? Thanks… --- ## [ES snapshot](https://discuss.elastic.co/t/es-snapshot/170355)
**Author:** [@vasudevan](https://discuss.elastic.co/u/vasudevan)\ **Replies:** 4\ **Last updated:** [March 5, 2019, 11:09am UTC](https://discuss.elastic.co/t/es-snapshot/170355 "2019-03-05T11:09:50Z")
I have created a snapshot on my single node ELK server and will be able to restore from that snapshot taken. But I can't see any file/content under my backup directory. Below is the command and it's output FYR, I have c… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1374) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1376)