# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1760 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1761 --- ## [Logstash restarting by output ES](https://discuss.elastic.co/t/logstash-restarting-by-output-es/251509)
**Author:** [@dromval90](https://discuss.elastic.co/u/dromval90)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 10:35pm UTC](https://discuss.elastic.co/t/logstash-restarting-by-output-es/251509 "2020-10-08T22:35:05Z")
Hi good day community: I would like to ask if someone that you it has experiment this behavior: We have 4 servers with instances of logstash with version 7.9, into config file into output to Elasticsearch, we have 12 s… --- ## [Help With ILM](https://discuss.elastic.co/t/help-with-ilm/249177)
**Author:** [@Bruceclegg](https://discuss.elastic.co/u/Bruceclegg)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 9:55pm UTC](https://discuss.elastic.co/t/help-with-ilm/249177 "2020-10-08T21:55:49Z")
I still consider myself very much a neophyte with ELK. A few weeks ago I set up Lifecycle Policies for Indices in Kibana. I used the old index names the developers have hard coded into their programs. But when it came… --- ## [Unable to manually specify start/end timestamps in date range selector](https://discuss.elastic.co/t/unable-to-manually-specify-start-end-timestamps-in-date-range-selector/251487)
**Author:** [@daq](https://discuss.elastic.co/u/daq)\ **Replies:** 9\ **Last updated:** [October 8, 2020, 9:29pm UTC](https://discuss.elastic.co/t/unable-to-manually-specify-start-end-timestamps-in-date-range-selector/251487 "2020-10-08T21:29:39Z")
What am I doing wrong? I'm trying to manually adjust start/end timestamps in the date range selector in discover tab, but it just prints expected format in red and doesn't give me any way to apply manually entered timest… --- ## [Which schema to use with Elasticsearch](https://discuss.elastic.co/t/which-schema-to-use-with-elasticsearch/251430)
**Author:** [@chabow](https://discuss.elastic.co/u/chabow)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 9:15pm UTC](https://discuss.elastic.co/t/which-schema-to-use-with-elasticsearch/251430 "2020-10-08T21:15:07Z")
This is hopefully an easy question for the experts. We just started with an instance of Elasticsearch. And we need to add metadata to a big part of our content. Now the question is: which schema to use? We are aware that… --- ## [Diff of 2 terms aggregations](https://discuss.elastic.co/t/diff-of-2-terms-aggregations/251021)
**Author:** [@Florent\_Fauvin](https://discuss.elastic.co/u/Florent_Fauvin)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 7:08pm UTC](https://discuss.elastic.co/t/diff-of-2-terms-aggregations/251021 "2020-10-08T19:08:29Z")
Hello, I want to create 2 terms aggregations : "allUsers" : {"terms": {"field": "User"}} "allPresentUsers" : {"terms": {"field": "PresentUser"}} =\> Then, I want to find the absent users. So, I want to create a thir… --- ## [Unable to read the txt file through Logstash](https://discuss.elastic.co/t/unable-to-read-the-txt-file-through-logstash/251057)
**Author:** [@sukumar.koganti](https://discuss.elastic.co/u/sukumar.koganti)\ **Replies:** 24\ **Last updated:** [October 8, 2020, 6:55pm UTC](https://discuss.elastic.co/t/unable-to-read-the-txt-file-through-logstash/251057 "2020-10-08T18:55:20Z")
Hi Guys, I am trying to read sample EDI log from the below path using logstash .conf file C:\\Users\\skkoganti\\Documents\\log\\EDI Log-2019-05-27.txt this is my sample input file format input { file { path =\> \["C:/Us… --- ## [Need @timestamp value where the maximum aggregation result occurred](https://discuss.elastic.co/t/need-timestamp-value-where-the-maximum-aggregation-result-occurred/249993)
**Author:** [@silentfilm](https://discuss.elastic.co/u/silentfilm)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 6:30pm UTC](https://discuss.elastic.co/t/need-timestamp-value-where-the-maximum-aggregation-result-occurred/249993 "2020-10-08T18:30:30Z")
I'm using an Elasticsearch query (6.7) to find the maximum number of records returned for a particular service. My aggregation query is working great, but I need to also find the time when this maximum occurred. If I s… --- ## [Script\_fields in MultiSearchRequest](https://discuss.elastic.co/t/script-fields-in-multisearchrequest/251495)
**Author:** [@Nick\_Name](https://discuss.elastic.co/u/Nick_Name)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 6:22pm UTC](https://discuss.elastic.co/t/script-fields-in-multisearchrequest/251495 "2020-10-08T18:22:07Z")
Can I use script\_fields from one query in another query script when I create multisearch request? I have this error, can anyone help me? This my request in Postman: {"index" : "core\_event\_usvc\_provided"} {"que… --- ## [Kibana Visualization - Gauge](https://discuss.elastic.co/t/kibana-visualization-gauge/250037)
**Author:** [@rkarthik.kr](https://discuss.elastic.co/u/rkarthik.kr)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 6:20pm UTC](https://discuss.elastic.co/t/kibana-visualization-gauge/250037 "2020-10-08T18:20:36Z")
Hello All I am trying to build a visualization in Kibana using Gauge . Below is the data in my Index App\_Name Service\_Name Run\_Number Coverage Red A 4 90 Red B 4 … --- ## [Logstash pipeline.yml error](https://discuss.elastic.co/t/logstash-pipeline-yml-error/251441)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 8\ **Last updated:** [October 8, 2020, 6:08pm UTC](https://discuss.elastic.co/t/logstash-pipeline-yml-error/251441 "2020-10-08T18:08:46Z")
I've one logstash pipeline.which is given in pipeline.yml. but when i run bin/logstash.bat command in powershell i'm getting error which is look like this ERROR: Failed to read pipelines yaml file. Location: D:/Elast… --- ## [Histogram aggregation over unique IDs: possible with nested aggregations?](https://discuss.elastic.co/t/histogram-aggregation-over-unique-ids-possible-with-nested-aggregations/250539)
**Author:** [@sandbender](https://discuss.elastic.co/u/sandbender)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 6:07pm UTC](https://discuss.elastic.co/t/histogram-aggregation-over-unique-ids-possible-with-nested-aggregations/250539 "2020-10-08T18:07:55Z")
Hello, I am trying to aggregate data for a heatmap visualisation in Vega-Lite. It is supposed to show the volume of communication (color) in volume ranges (y-axis) over time (x-axis). Where I'm stuck at right now is: t… --- ## [How to display total calls minus closed calls](https://discuss.elastic.co/t/how-to-display-total-calls-minus-closed-calls/250493)
**Author:** [@anjilinga](https://discuss.elastic.co/u/anjilinga)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 6:03pm UTC](https://discuss.elastic.co/t/how-to-display-total-calls-minus-closed-calls/250493 "2020-10-08T18:03:12Z")
Hi, We have the records in the index as below. callid status 1 open 1 open 1 open 1 closed 2 open 2 open 2 closed 3 open 3 open here we can… --- ## [Removing parent object names with ingest pipeline](https://discuss.elastic.co/t/removing-parent-object-names-with-ingest-pipeline/251492)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 5:27pm UTC](https://discuss.elastic.co/t/removing-parent-object-names-with-ingest-pipeline/251492 "2020-10-08T17:27:38Z")
I'm trying to figure out how to clean up my fields and remove some of the top level parents in my documents. I want to turn Properties.Status and Properties.Status.StatusName into Status and Status.StatusName, respective… --- ## [Tracking down who created Dashboards and Visualizations](https://discuss.elastic.co/t/tracking-down-who-created-dashboards-and-visualizations/251475)
**Author:** [@Shaw](https://discuss.elastic.co/u/Shaw)\ **Replies:** 6\ **Last updated:** [October 8, 2020, 5:20pm UTC](https://discuss.elastic.co/t/tracking-down-who-created-dashboards-and-visualizations/251475 "2020-10-08T17:20:32Z")
Is there any ways to track down who created a dashboard and visualizaions? I do not see any options or places to indicate who created it, who was last person to modify it, and when. It will be very helpful when audit ton… --- ## [7.9.2 http code convert ? to %3F. Rest API fails. Used to work](https://discuss.elastic.co/t/7-9-2-http-code-convert-to-3f-rest-api-fails-used-to-work/250385)
**Author:** [@pgervais](https://discuss.elastic.co/u/pgervais)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 5:13pm UTC](https://discuss.elastic.co/t/7-9-2-http-code-convert-to-3f-rest-api-fails-used-to-work/250385 "2020-10-08T17:13:15Z")
I'm trying to convert my plugin that works under 7.8.1 to 7.9.2. I use the saved object REST API. In order to get all index patterns already created. The REST API call look like: 'http://localhost:5601/api/saved\_objec… --- ## [ELK SAML with Gsuite](https://discuss.elastic.co/t/elk-saml-with-gsuite/251462)
**Author:** [@mhr](https://discuss.elastic.co/u/mhr)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 5:12pm UTC](https://discuss.elastic.co/t/elk-saml-with-gsuite/251462 "2020-10-08T17:12:35Z")
HI Support, Do we require license version of ELK stack to use SAML with Google Gsuite. --- ## [ES master CPU spike suddenly 100% and fails to apply cluster state](https://discuss.elastic.co/t/es-master-cpu-spike-suddenly-100-and-fails-to-apply-cluster-state/251488)
**Author:** [@UDixit](https://discuss.elastic.co/u/UDixit)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 4:55pm UTC](https://discuss.elastic.co/t/es-master-cpu-spike-suddenly-100-and-fails-to-apply-cluster-state/251488 "2020-10-08T16:55:50Z")
We are currently operating with ES5.6. All nodes in the ES cluster are master and data eligible. The min\_master\_nodes is set to n/2+1. We are recently seeing issues that the matser's CPU usage spikes up very high ~90% … --- ## [Any filter for parsing the URL request from firewalls?](https://discuss.elastic.co/t/any-filter-for-parsing-the-url-request-from-firewalls/251451)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 4:46pm UTC](https://discuss.elastic.co/t/any-filter-for-parsing-the-url-request-from-firewalls/251451 "2020-10-08T16:46:20Z")
regarding payload within firewall logs, the request field has a single line separated by \\r\\n which I've made it into individual line. Any known parsers/filter to make this into key-value pairs? (especially the User-Age… --- ## [Log field from microservice app logs in Kibana on AWS](https://discuss.elastic.co/t/log-field-from-microservice-app-logs-in-kibana-on-aws/249580)
**Author:** [@Praful](https://discuss.elastic.co/u/Praful)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 4:36pm UTC](https://discuss.elastic.co/t/log-field-from-microservice-app-logs-in-kibana-on-aws/249580 "2020-10-08T16:36:09Z")
I am searching for a string in the logs using scripted field in Kibana using the Painless code. My aim is find more details in the error/exception logs. try { if (doc.containsKey('log.keyword') && !doc\['log.keyw… --- ## [BM25 normalizing per field](https://discuss.elastic.co/t/bm25-normalizing-per-field/251485)
**Author:** [@tp-c-c](https://discuss.elastic.co/u/tp-c-c)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 4:29pm UTC](https://discuss.elastic.co/t/bm25-normalizing-per-field/251485 "2020-10-08T16:29:40Z")
Hi, I have a question about the BM25 ranking algorithm, specifically about the dividing by the field length. I understand the general descriptions of the algorithm where they divide by the document length as in this for… --- ## [Access a field in array using Painless scripted field](https://discuss.elastic.co/t/access-a-field-in-array-using-painless-scripted-field/250774)
**Author:** [@WatsonS](https://discuss.elastic.co/u/WatsonS)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 4:27pm UTC](https://discuss.elastic.co/t/access-a-field-in-array-using-painless-scripted-field/250774 "2020-10-08T16:27:10Z")
Hi, Since nested array is not well supported in Kibana, is there any way I could extract the value from the array using scripted field? My data looks like this 1st document: "page\_name": \[ { "name": "New York" }, … --- ## [Graph not showing correct nodes](https://discuss.elastic.co/t/graph-not-showing-correct-nodes/249235)
**Author:** [@emahdij](https://discuss.elastic.co/u/emahdij)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 4:19pm UTC](https://discuss.elastic.co/t/graph-not-showing-correct-nodes/249235 "2020-10-08T16:19:17Z")
Hello When i add a field to a graph it doesn't show the correct nodes As you can see in below image i have five different value for 'DevName' field but when i add that to a graph it doesn't show any of them. --- ## [Email Troubleshooting](https://discuss.elastic.co/t/email-troubleshooting/251480)
**Author:** [@teej](https://discuss.elastic.co/u/teej)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 4:05pm UTC](https://discuss.elastic.co/t/email-troubleshooting/251480 "2020-10-08T16:05:26Z")
Hi, I am trying to setup alerts using email as the notification vehicle. Unfortunately, the alerts are not showing up in the inbox. So I need to trace where the failure is occurring as it may not even be an elastic issu… --- ## [Loading SQL table with JSON data and columnar data](https://discuss.elastic.co/t/loading-sql-table-with-json-data-and-columnar-data/251286)
**Author:** [@sanmitdesai](https://discuss.elastic.co/u/sanmitdesai)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 3:49pm UTC](https://discuss.elastic.co/t/loading-sql-table-with-json-data-and-columnar-data/251286 "2020-10-08T15:49:00Z")
Hey all, I am a bit new to logstash. I have a SQL table that looks like this. As we can see we have columns 'Address' and 'Skills' in this table that are in JSON format. I want to load all the data while maintainin… --- ## [Monitoring tab blank](https://discuss.elastic.co/t/monitoring-tab-blank/245525)
**Author:** [@bevano](https://discuss.elastic.co/u/bevano)\ **Replies:** 18\ **Last updated:** [October 8, 2020, 3:47pm UTC](https://discuss.elastic.co/t/monitoring-tab-blank/245525 "2020-10-08T15:47:10Z")
Updated from 7.8.1 to 7.9 today on my monitoring cluster and after clicking on the monitoring tab, i am presented with a blank page. No error with regards to Kibana but from the web console i can see this. I assume it i… --- ## [Esrally performance benchmark](https://discuss.elastic.co/t/esrally-performance-benchmark/251477)
**Author:** [@kagrahar](https://discuss.elastic.co/u/kagrahar)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 3:41pm UTC](https://discuss.elastic.co/t/esrally-performance-benchmark/251477 "2020-10-08T15:41:08Z")
i am running esrally for the first time. What are the key metrics in the output that determines performance of the elastic search cluster? Any best practices to stress test? Is there a best practice for setting the numbe… --- ## [How to parse mix json logs file](https://discuss.elastic.co/t/how-to-parse-mix-json-logs-file/251470)
**Author:** [@Purvesh\_Jaiswal](https://discuss.elastic.co/u/Purvesh_Jaiswal)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 3:09pm UTC](https://discuss.elastic.co/t/how-to-parse-mix-json-logs-file/251470 "2020-10-08T15:09:07Z")
''''''''' {"tool": {"driver": {"name": "Shell Script Analysis", "version": "1.0.0-scan", "fullName": "Shell Script Analysis"}}, "conversion": {"tool": {"driver": {"name": "@ShiftLeft/sast-scan"}}, "invocation": {"argume… --- ## [Logstash filtering. Extract data between two strings](https://discuss.elastic.co/t/logstash-filtering-extract-data-between-two-strings/251276)
**Author:** [@Igor\_Olikh](https://discuss.elastic.co/u/Igor_Olikh)\ **Replies:** 6\ **Last updated:** [October 8, 2020, 3:04pm UTC](https://discuss.elastic.co/t/logstash-filtering-extract-data-between-two-strings/251276 "2020-10-08T15:04:33Z")
I have a UNIX log looks like: ACTION started Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s finished when an un… --- ## [Logstash CIDR network range](https://discuss.elastic.co/t/logstash-cidr-network-range/251391)
**Author:** [@ParashB](https://discuss.elastic.co/u/ParashB)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 2:55pm UTC](https://discuss.elastic.co/t/logstash-cidr-network-range/251391 "2020-10-08T14:55:48Z")
Hello, How to match IP range which is not full network range? Example: 172.20.23.5 in \[172.16.0.0 -172.31.255.255 \] CIDR filter plugin specifies the full range but I need to search is specific range. --- ## [Logstash dont start if output fail](https://discuss.elastic.co/t/logstash-dont-start-if-output-fail/251467)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 2:53pm UTC](https://discuss.elastic.co/t/logstash-dont-start-if-output-fail/251467 "2020-10-08T14:53:50Z")
Using RabbitMQ as output, Logstash dont start if RabbitMQ is down: \[2020-10-08T14:45:32,102\]\[ERROR\]\[logstash.outputs.rabbitmq\]\[main\] RabbitMQ connection error, will retry. {:error\_message=\>"Connection was refused. Targe… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1759) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1761)