# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1478 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1479 --- ## [How to put filter inside Vega visualization scatter plot](https://discuss.elastic.co/t/how-to-put-filter-inside-vega-visualization-scatter-plot/267618)
**Author:** [@Shalhevet](https://discuss.elastic.co/u/Shalhevet)\ **Replies:** 2\ **Last updated:** [April 5, 2021, 7:19am UTC](https://discuss.elastic.co/t/how-to-put-filter-inside-vega-visualization-scatter-plot/267618 "2021-04-05T07:19:28Z")
hi, I want to present the plots for documents with "group"==a, and I only want to present documents with the latest date of the time range applied by the dashboard. How can I do this? here is my script: { $schema: … --- ## [Completion Suggester](https://discuss.elastic.co/t/completion-suggester/269002)
**Author:** [@Lamees\_Mahmoud](https://discuss.elastic.co/u/Lamees_Mahmoud)\ **Replies:** 0\ **Last updated:** [April 1, 2021, 9:04am UTC](https://discuss.elastic.co/t/completion-suggester/269002 "2021-04-01T09:04:45Z")
Why elastic search auto completion suggester doesn't return all matching documents? --- ## [Connect elastic search or Kibana with SQL database](https://discuss.elastic.co/t/connect-elastic-search-or-kibana-with-sql-database/269189)
**Author:** [@pooja1](https://discuss.elastic.co/u/pooja1)\ **Replies:** 4\ **Last updated:** [April 5, 2021, 6:03am UTC](https://discuss.elastic.co/t/connect-elastic-search-or-kibana-with-sql-database/269189 "2021-04-05T06:03:22Z")
I have a database that is build in MYSQL. I want to connect it with Kibana so that I can access the data by a query Please let me know how should I do it? --- ## [Authentication of \[elastic\] was terminated by realm \[reserved\]](https://discuss.elastic.co/t/authentication-of-elastic-was-terminated-by-realm-reserved/268307)
**Author:** [@Divyanshu\_Tyagi](https://discuss.elastic.co/u/Divyanshu_Tyagi)\ **Replies:** 3\ **Last updated:** [April 5, 2021, 5:51am UTC](https://discuss.elastic.co/t/authentication-of-elastic-was-terminated-by-realm-reserved/268307 "2021-04-05T05:51:01Z")
My ELK stack is udpated with same version Elasticsearch version 7.11.1 logstash version 7.11.1 filebeat version 7.11.1 And in elasticsearch.log only thing i am seeing is \[INFO \]\[o.e.x.s.a.AuthenticationService\] \[inlxd… --- ## [Processors \> convert not working when the string includes whitespace](https://discuss.elastic.co/t/processors-convert-not-working-when-the-string-includes-whitespace/269201)
**Author:** [@makeajourney](https://discuss.elastic.co/u/makeajourney)\ **Replies:** 17\ **Last updated:** [April 5, 2021, 4:59am UTC](https://discuss.elastic.co/t/processors-convert-not-working-when-the-string-includes-whitespace/269201 "2021-04-05T04:59:07Z")
Hello. I am setting up the filebeat. I want to convert the string to double and it has whitespace. I was trying it with dissect. but it's not fit my situation. because that string has not only have numbers. so I've th… --- ## [Maps with multiple layer appear not to work as expected](https://discuss.elastic.co/t/maps-with-multiple-layer-appear-not-to-work-as-expected/268959)
**Author:** [@Russell\_Fulton](https://discuss.elastic.co/u/Russell_Fulton)\ **Replies:** 14\ **Last updated:** [April 5, 2021, 4:15am UTC](https://discuss.elastic.co/t/maps-with-multiple-layer-appear-not-to-work-as-expected/268959 "2021-04-05T04:15:12Z")
Kibaba 7.10 When I create a map with multiple layers and then hide all bar one layer the map always displays the same data when in a dashboard panel: The major difference between the two layers is that the "all auth" h… --- ## [Slowness in elastic search - optimization of query and score against individual values of the field](https://discuss.elastic.co/t/slowness-in-elastic-search-optimization-of-query-and-score-against-individual-values-of-the-field/269199)
**Author:** [@Nisar\_Ansari](https://discuss.elastic.co/u/Nisar_Ansari)\ **Replies:** 0\ **Last updated:** [April 5, 2021, 12:07am UTC](https://discuss.elastic.co/t/slowness-in-elastic-search-optimization-of-query-and-score-against-individual-values-of-the-field/269199 "2021-04-05T00:07:06Z")
Dear All, We have documents in below format , the number of documents is more than 5 million. {"\_index":"Testinlive","\_type":"\_doc","\_id":"vxYmiHgBWrQelERu5MPg","\_score":1.0,"\_source":{"GID":"433444","aLIASES":"Kishor… --- ## [Geoip - Apache logs](https://discuss.elastic.co/t/geoip-apache-logs/269166)
**Author:** [@gloupe](https://discuss.elastic.co/u/gloupe)\ **Replies:** 5\ **Last updated:** [April 4, 2021, 7:07pm UTC](https://discuss.elastic.co/t/geoip-apache-logs/269166 "2021-04-04T19:07:12Z")
Hi, I'm trying to get geoip info from ipclient from my access logs generated by a apache2 server. I tried this filter in logstash: filter { if \[tags\]\[path\] =~ "/var/log/apache2/xxx-access.log" { … --- ## [How to do ES fault drills](https://discuss.elastic.co/t/how-to-do-es-fault-drills/269186)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 1\ **Last updated:** [April 4, 2021, 6:05pm UTC](https://discuss.elastic.co/t/how-to-do-es-fault-drills/269186 "2021-04-04T18:05:41Z")
We want to do elasticsearch failure exercises, but currently we can only simulate some simple failures, such as network interruption and process death. The remaining deep-level failures are not well simulated. I would li… --- ## [Visualizing json documents](https://discuss.elastic.co/t/visualizing-json-documents/268807)
**Author:** [@Tlm\_Konstantinos](https://discuss.elastic.co/u/Tlm_Konstantinos)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 4:59pm UTC](https://discuss.elastic.co/t/visualizing-json-documents/268807 "2021-04-04T16:59:03Z")
Hi every one. I have enriched all records in an es index with some json data. ex {'key1':{"key11":val11, "key12":val12}, 'key2': val2} I am looking for a nice way to display that json in Table view or maybe in another… --- ## [How to monitor ILM in canvas](https://discuss.elastic.co/t/how-to-monitor-ilm-in-canvas/268879)
**Author:** [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\ **Replies:** 8\ **Last updated:** [April 4, 2021, 4:32pm UTC](https://discuss.elastic.co/t/how-to-monitor-ilm-in-canvas/268879 "2021-04-04T16:32:53Z")
Hello, I would like to create a Canvas where I can see the number of indexs in hot warm and cold and the the total disk used in each phase., and for example the name of index which has been deleted in the last 24 hours … --- ## [Restore from S3 repository\_exception](https://discuss.elastic.co/t/restore-from-s3-repository-exception/268848)
**Author:** [@twilight](https://discuss.elastic.co/u/twilight)\ **Replies:** 9\ **Last updated:** [April 4, 2021, 1:38pm UTC](https://discuss.elastic.co/t/restore-from-s3-repository-exception/268848 "2021-04-04T13:38:27Z")
Hello, This is about restoring on self-managed from a snapshot created by managed stack. I have ES and Kibana 7.12 installed on Amazon Linux AMI. I have a cloud based cluster too which has created a number of snapshot… --- ## [Using grok filter to parse log file](https://discuss.elastic.co/t/using-grok-filter-to-parse-log-file/269138)
**Author:** [@bab](https://discuss.elastic.co/u/bab)\ **Replies:** 3\ **Last updated:** [April 4, 2021, 1:20pm UTC](https://discuss.elastic.co/t/using-grok-filter-to-parse-log-file/269138 "2021-04-04T13:20:02Z")
Hi guys, i'm trying to parse the following log files, i succeed to filter all fields unless the last one, can some one help pls, thank you. Log file: 1;2;06-19-15;start inbound processing;50034744;1;ok;2021-03-18 16:1… --- ## [Give path as an argument](https://discuss.elastic.co/t/give-path-as-an-argument/269173)
**Author:** [@vincedt09](https://discuss.elastic.co/u/vincedt09)\ **Replies:** 1\ **Last updated:** [April 4, 2021, 12:50pm UTC](https://discuss.elastic.co/t/give-path-as-an-argument/269173 "2021-04-04T12:50:52Z")
Hi! Is there a way to give path as an argument? I was thinking at something like this: input { file { path =\> argv sincedb\_path =\> "NUL" start\_position =\> "beginning" } } logsta… --- ## [Dashboard in vega query elasticsearch and a fixed](https://discuss.elastic.co/t/dashboard-in-vega-query-elasticsearch-and-a-fixed/269075)
**Author:** [@francieliton\_araujo](https://discuss.elastic.co/u/francieliton_araujo)\ **Replies:** 6\ **Last updated:** [April 4, 2021, 12:47pm UTC](https://discuss.elastic.co/t/dashboard-in-vega-query-elasticsearch-and-a-fixed/269075 "2021-04-04T12:47:33Z")
how to create a dashboard in vega sound an elasticsearch query and a fixed value. { "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "description": "A simple pie chart with labels.", "data": { "v… --- ## ["Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \\"{\\", \\"}\\"](https://discuss.elastic.co/t/expected-one-of-t-r-n/269151)
**Author:** [@tmrgbox](https://discuss.elastic.co/u/tmrgbox)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 8:14am UTC](https://discuss.elastic.co/t/expected-one-of-t-r-n/269151 "2021-04-04T08:14:35Z")
Hi, I'm trying to parse a log using the following config: input { udp { port =\> 5555 type =\> "esetlog" add\_field =\> { "etl\_input\_port" =\> 5555 "etl\_input\_protocol" =\> "udp" } } } filter { if \[type\] == "eset… --- ## [Logstash filter with grok](https://discuss.elastic.co/t/logstash-filter-with-grok/269056)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 1\ **Last updated:** [April 1, 2021, 4:39pm UTC](https://discuss.elastic.co/t/logstash-filter-with-grok/269056 "2021-04-01T16:39:29Z")
I have used grok debugger and I've identified the pattern i need for getting the filed id.orig\_h My log looks like this: 1295981543.337241 CCBr9k1C1bSaHQbUhh 192.168.3.131 55954 65.55.17.37 80 tcp http 1.776718 1837 24… --- ## [Don't save in ES logs that have in tag \_grokparsefailure or \_dateparsefailure](https://discuss.elastic.co/t/dont-save-in-es-logs-that-have-in-tag-grokparsefailure-or-dateparsefailure/269141)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 5:20am UTC](https://discuss.elastic.co/t/dont-save-in-es-logs-that-have-in-tag-grokparsefailure-or-dateparsefailure/269141 "2021-04-04T05:20:42Z")
Hi, How can i filter the logs that are saved within ES in order to have only valid data that has no grok parse failure or date parse error? My approach is bellow output { if ("\_grokparsefailure" not in \[tags\]) or ("\_… --- ## [Logstash pipeline schedule](https://discuss.elastic.co/t/logstash-pipeline-schedule/269139)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 3\ **Last updated:** [April 4, 2021, 5:18am UTC](https://discuss.elastic.co/t/logstash-pipeline-schedule/269139 "2021-04-04T05:18:45Z")
Hi there I\`ve faced with a problem... I have a running Logstash with multiple pipelines. I just want to run one of these pipelines from 00:00 am until 08:00 am everyday According to Logstash documentation, pipeline co… --- ## [Shrink How to avoid stop writing](https://discuss.elastic.co/t/shrink-how-to-avoid-stop-writing/269154)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 2\ **Last updated:** [April 4, 2021, 2:27am UTC](https://discuss.elastic.co/t/shrink-how-to-avoid-stop-writing/269154 "2021-04-04T02:27:30Z")
I have 1 index with 2 primary shards, I want to narrow it down to 1 shard, I learned through search that it is recommended to achieve through shrink api, but shrink will stop writing, is there any solution to stop writin… --- ## [Count array size in all document](https://discuss.elastic.co/t/count-array-size-in-all-document/269108)
**Author:** [@Giovanni\_Di\_Lembo](https://discuss.elastic.co/u/Giovanni_Di_Lembo)\ **Replies:** 7\ **Last updated:** [April 3, 2021, 8:49pm UTC](https://discuss.elastic.co/t/count-array-size-in-all-document/269108 "2021-04-03T20:49:19Z")
Hi all and sorry for my poor english I have a document like { "uuid": "SkDHvXr9ES", "userRole": "9qvfWg3isp", "devices":\[ {"uuid": "hwb258pc7x", "macAddress":\["9DGSsuAX4M", "ChypSa8tWx", "fIWrArQyDA", "PQjcvOkHYh… --- ## [Date filter does not work](https://discuss.elastic.co/t/date-filter-does-not-work/269148)
**Author:** [@Andrey\_RF](https://discuss.elastic.co/u/Andrey_RF)\ **Replies:** 3\ **Last updated:** [April 3, 2021, 6:32pm UTC](https://discuss.elastic.co/t/date-filter-does-not-work/269148 "2021-04-03T18:32:34Z")
I use grok and date filter to set correct @timestamp. It works in simple case. But when I added if condition @timestamp no longer installed from the message. filter { if 'backend.info.log' in \[log\]\[file\]\[path\] { g… --- ## [Aggregation by host displays hash instead of hostname](https://discuss.elastic.co/t/aggregation-by-host-displays-hash-instead-of-hostname/269158)
**Author:** [@desoul](https://discuss.elastic.co/u/desoul)\ **Replies:** 0\ **Last updated:** [April 3, 2021, 4:21pm UTC](https://discuss.elastic.co/t/aggregation-by-host-displays-hash-instead-of-hostname/269158 "2021-04-03T16:21:19Z")
Hello, i try to get unique agent hostnames, grouped with its logfile paths (i have Filebeat agents with system.yml configuration). Here is a query: curl -u myUser:myPassword-X GET "myELKhostname:9237/filebeat\*/\_search?p… --- ## [Filebeat initializing error for s3](https://discuss.elastic.co/t/filebeat-initializing-error-for-s3/269137)
**Author:** [@paano](https://discuss.elastic.co/u/paano)\ **Replies:** 4\ **Last updated:** [April 3, 2021, 2:48pm UTC](https://discuss.elastic.co/t/filebeat-initializing-error-for-s3/269137 "2021-04-03T14:48:41Z")
I am trying to follow as explained in the elastic blog And getting error: 2021-04-03T01:04:23.845Z ERROR instance/beat.go:971 Exiting: Failed to start crawler: starting input failed: Error while initializing input: Err… --- ## [Ship logs to multiple logstash servers from single filebeat](https://discuss.elastic.co/t/ship-logs-to-multiple-logstash-servers-from-single-filebeat/269147)
**Author:** [@prajwal93](https://discuss.elastic.co/u/prajwal93)\ **Replies:** 1\ **Last updated:** [April 3, 2021, 1:46pm UTC](https://discuss.elastic.co/t/ship-logs-to-multiple-logstash-servers-from-single-filebeat/269147 "2021-04-03T13:46:10Z")
Currently, I have a requirement to ship logs to multiple logstash endpoints from a single filebeat instances. I don't want to loadbalance, but push same logs to different logstash endpoints. I was not able to find the … --- ## [Filebeat netflow module crashes](https://discuss.elastic.co/t/filebeat-netflow-module-crashes/269126)
**Author:** [@S3l3ct3d](https://discuss.elastic.co/u/S3l3ct3d)\ **Replies:** 2\ **Last updated:** [April 3, 2021, 1:09pm UTC](https://discuss.elastic.co/t/filebeat-netflow-module-crashes/269126 "2021-04-03T13:09:24Z")
I am currently receiving netflow logs on port 2055 (default) on my Logstash server. I also have filebeat running on this server, which is also currently using the panw and crowdstrike modules with no issues. So the fileb… --- ## [How to simulate the failure of elasticsearch？](https://discuss.elastic.co/t/how-to-simulate-the-failure-of-elasticsearch/269149)
**Author:** [@asasas234](https://discuss.elastic.co/u/asasas234)\ **Replies:** 0\ **Last updated:** [April 3, 2021, 12:30pm UTC](https://discuss.elastic.co/t/how-to-simulate-the-failure-of-elasticsearch/269149 "2021-04-03T12:30:44Z")
I found that for some simple failures, I can do it easily with the help of tools, such as network partitioning, nodes being killed, but there are still many abnormal cases that are not good to simulate, for example, ther… --- ## [Latency in indexing using client.update() of the high level rest client](https://discuss.elastic.co/t/latency-in-indexing-using-client-update-of-the-high-level-rest-client/269142)
**Author:** [@Mahesh\_H\_Viraktamath](https://discuss.elastic.co/u/Mahesh_H_Viraktamath)\ **Replies:** 2\ **Last updated:** [April 3, 2021, 10:13am UTC](https://discuss.elastic.co/t/latency-in-indexing-using-client-update-of-the-high-level-rest-client/269142 "2021-04-03T10:13:54Z")
We have started using the high level REST client finally, to ease the development of queries from backend engineering perspective. For indexing, we are using the client.update(request, RequestOptions.DEFAULT) so that new… --- ## [Logstash is not processing logs](https://discuss.elastic.co/t/logstash-is-not-processing-logs/269084)
**Author:** [@James\_P](https://discuss.elastic.co/u/James_P)\ **Replies:** 1\ **Last updated:** [April 3, 2021, 7:49am UTC](https://discuss.elastic.co/t/logstash-is-not-processing-logs/269084 "2021-04-03T07:49:10Z")
Hi, Logstash was installed and was receiving logs from filebeat and syslog file stream. Currently logstash starts up successfully but no logs are being processed. \[2021-04-02T08:48:29,269\]\[INFO \]\[logstash.runner … --- ## [Kibana - Pipeline Editor](https://discuss.elastic.co/t/kibana-pipeline-editor/269134)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 1\ **Last updated:** [April 2, 2021, 9:18pm UTC](https://discuss.elastic.co/t/kibana-pipeline-editor/269134 "2021-04-02T21:18:56Z")
Hi, I'm using Kibana 7.11.1 and I edit my pipelines under the Logstash Pipeline UI. I was wondering if there is a document somewhere that lists available shortcuts that can be used inside of the pipeline editor. Specifi… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1477) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1479)