# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1772 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1773 --- ## [How to use logstash parsers in filebeat to parse customer logs](https://discuss.elastic.co/t/how-to-use-logstash-parsers-in-filebeat-to-parse-customer-logs/250856)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 0\ **Last updated:** [October 3, 2020, 3:39am UTC](https://discuss.elastic.co/t/how-to-use-logstash-parsers-in-filebeat-to-parse-customer-logs/250856 "2020-10-03T03:39:17Z")
Hello, I BIND parsers created with logstash and few others as well. Can someone please guide how do I use those parsers with filebeat to create custom module. TIA --- ## [ConfigurationError: Missing node(s)](https://discuss.elastic.co/t/configurationerror-missing-node-s/250841)
**Author:** [@glimmbo](https://discuss.elastic.co/u/glimmbo)\ **Replies:** 1\ **Last updated:** [October 3, 2020, 2:44am UTC](https://discuss.elastic.co/t/configurationerror-missing-node-s/250841 "2020-10-03T02:44:37Z")
I am using the JS client to connect to AWS Elasticsearch Service. With the setup: import { Client, ClientOptions } from "@elastic/elasticsearch" const { ES\_AWS\_ENDPOINT, ES\_AWS\_USER, ES\_AWS\_PASSWORD, } = process.… --- ## [Painless Script with name contain dot](https://discuss.elastic.co/t/painless-script-with-name-contain-dot/249231)
**Author:** [@Narendra\_Jain](https://discuss.elastic.co/u/Narendra_Jain)\ **Replies:** 2\ **Last updated:** [October 3, 2020, 2:21am UTC](https://discuss.elastic.co/t/painless-script-with-name-contain-dot/249231 "2020-10-03T02:21:15Z")
Hi, I have field name with dot. like param.request\_id, this is actual field name and not Actual: { "param.request\_id": 12343 } Not like { "param" : { "request\_id": 12343 } } . when i try scripted filed i… --- ## [Elasticseach filter on logstash not working?](https://discuss.elastic.co/t/elasticseach-filter-on-logstash-not-working/250849)
**Author:** [@sdonkey](https://discuss.elastic.co/u/sdonkey)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 10:38pm UTC](https://discuss.elastic.co/t/elasticseach-filter-on-logstash-not-working/250849 "2020-10-02T22:38:24Z")
Hi, I'm trying to use the elastic search filter on logstash for make some Data Enrichment. I got two indexes, and my goal it's get some data from one of them and add it to the other. I configured a logstash filter who… --- ## [Stopping LogStash after processing the data](https://discuss.elastic.co/t/stopping-logstash-after-processing-the-data/250749)
**Author:** [@Roman\_Kagan](https://discuss.elastic.co/u/Roman_Kagan)\ **Replies:** 4\ **Last updated:** [October 2, 2020, 9:58pm UTC](https://discuss.elastic.co/t/stopping-logstash-after-processing-the-data/250749 "2020-10-02T21:58:36Z")
Hello: I am creating my own conf file that I supply on command line to LogStash - it works like a charm, however it does not stop after processing all the data. I do not expect to generate new files in some folder, nor… --- ## [Running sample text file thorough logstash](https://discuss.elastic.co/t/running-sample-text-file-thorough-logstash/250821)
**Author:** [@sukumar.koganti](https://discuss.elastic.co/u/sukumar.koganti)\ **Replies:** 8\ **Last updated:** [October 2, 2020, 9:33pm UTC](https://discuss.elastic.co/t/running-sample-text-file-thorough-logstash/250821 "2020-10-02T21:33:46Z")
This is my sample file FName,Lname,sal,location Suresh,Koganti,6000,Montreal Denis,Gallagher,7000,Quebec and this is my conf file input { file{ path =\> "C:\\Users\\skkoganti\\Documents\\log\\users.txt" start\_p… --- ## [Filebeat module design](https://discuss.elastic.co/t/filebeat-module-design/250204)
**Author:** [@bernhard.fluehmann](https://discuss.elastic.co/u/bernhard.fluehmann)\ **Replies:** 3\ **Last updated:** [October 2, 2020, 9:23pm UTC](https://discuss.elastic.co/t/filebeat-module-design/250204 "2020-10-02T21:23:10Z")
In the past, most of the filebeat modules were using elasticsearch ingest pipelines to do the enrichment of documents. This means that they are as lightweight as possible, since the work is done centrally and scales well… --- ## [Nested Json Array parse failure](https://discuss.elastic.co/t/nested-json-array-parse-failure/250564)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 9\ **Last updated:** [October 2, 2020, 9:17pm UTC](https://discuss.elastic.co/t/nested-json-array-parse-failure/250564 "2020-10-02T21:17:57Z")
I am having trouble parsing json log file that contains nested array. I am trying to add the values in the array after "docs" as their own searchable field linked to the target docs. Here is sample log below: {"dba\_serv… --- ## [How to share field data between documents of the same file path](https://discuss.elastic.co/t/how-to-share-field-data-between-documents-of-the-same-file-path/250838)
**Author:** [@NomadicCodeGuy](https://discuss.elastic.co/u/NomadicCodeGuy)\ **Replies:** 2\ **Last updated:** [October 2, 2020, 8:23pm UTC](https://discuss.elastic.co/t/how-to-share-field-data-between-documents-of-the-same-file-path/250838 "2020-10-02T20:23:58Z")
I am using logstash to parse test logs. Some test log files have multiple errors per file. Only the last error in each file contains the "scenario" information but I need all errors from the same file path to share the s… --- ## [Parsing PFSense Proxy logs in Syslog](https://discuss.elastic.co/t/parsing-pfsense-proxy-logs-in-syslog/250818)
**Author:** [@Lee\_Archinal](https://discuss.elastic.co/u/Lee_Archinal)\ **Replies:** 8\ **Last updated:** [October 2, 2020, 7:49pm UTC](https://discuss.elastic.co/t/parsing-pfsense-proxy-logs-in-syslog/250818 "2020-10-02T19:49:12Z")
Good morning everyone, I recently deployed a PFSense box and enabled a Squid Proxy. I am shipping those logs to my ELK server to process and display in Kibana. I currently have filebeat running on my stack and have the … --- ## [Updating plugins with the ansible\\elasticsearch playbook](https://discuss.elastic.co/t/updating-plugins-with-the-ansible-elasticsearch-playbook/245952)
**Author:** [@biggtimber](https://discuss.elastic.co/u/biggtimber)\ **Replies:** 2\ **Last updated:** [October 2, 2020, 7:38pm UTC](https://discuss.elastic.co/t/updating-plugins-with-the-ansible-elasticsearch-playbook/245952 "2020-10-02T19:38:52Z")
How do you use the ansible\\elasticsearch playbook to update the es\_plugins? I did an update from 7.8 to 7.9 and it's not updating the plugin. --- ## [Devtools only showing 10,000 hits](https://discuss.elastic.co/t/devtools-only-showing-10-000-hits/250663)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 2\ **Last updated:** [October 2, 2020, 7:25pm UTC](https://discuss.elastic.co/t/devtools-only-showing-10-000-hits/250663 "2020-10-02T19:25:23Z")
I have just installed Elasticsearch 7.8.0 on my laptop as a dev environment. I am using 6.8.10 in production. I have a dataset of around 600,000 documents however when I did a match\_all in devtools, its only showing 10,0… --- ## [Error stacktrace take separate log line in logstash](https://discuss.elastic.co/t/error-stacktrace-take-separate-log-line-in-logstash/250812)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 1\ **Last updated:** [October 2, 2020, 4:51pm UTC](https://discuss.elastic.co/t/error-stacktrace-take-separate-log-line-in-logstash/250812 "2020-10-02T16:51:18Z")
I used multiline pattern in filebeat. the stacktrace of error log taking another line.not be with error log here is my filebeat multiline pattern: multiline.pattern: '^\[\[:space:\]\]' multiline.negate: false multiline.m… --- ## [Elasticsearch crashing Java Runtime Environment](https://discuss.elastic.co/t/elasticsearch-crashing-java-runtime-environment/250835)
**Author:** [@Cain310](https://discuss.elastic.co/u/Cain310)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 6:16pm UTC](https://discuss.elastic.co/t/elasticsearch-crashing-java-runtime-environment/250835 "2020-10-02T18:16:57Z")
Tried debugging this issue myself not sure whats up. Just crashes when not running or doing anything at random time. elastic version 7.9.2 H/W path Device Class Description =============… --- ## [How to do Nested for Multiple objects arrays in Kibana to create the dashboards](https://discuss.elastic.co/t/how-to-do-nested-for-multiple-objects-arrays-in-kibana-to-create-the-dashboards/250709)
**Author:** [@Prabhu.292](https://discuss.elastic.co/u/Prabhu.292)\ **Replies:** 3\ **Last updated:** [October 2, 2020, 6:01pm UTC](https://discuss.elastic.co/t/how-to-do-nested-for-multiple-objects-arrays-in-kibana-to-create-the-dashboards/250709 "2020-10-02T18:01:09Z")
Hi All, Below is the data set in kibana. My requirement is mentioned below for the kibana dashboards. department wise male & female gender ratio 2.department wise subject count 3.department wise age group people in… --- ## [Create histogram with group by count (binned data)](https://discuss.elastic.co/t/create-histogram-with-group-by-count-binned-data/250512)
**Author:** [@Pond](https://discuss.elastic.co/u/Pond)\ **Replies:** 8\ **Last updated:** [October 2, 2020, 5:57pm UTC](https://discuss.elastic.co/t/create-histogram-with-group-by-count-binned-data/250512 "2020-10-02T17:57:26Z")
Hi, I have devices users are using: {"userID":"1", "deviceID": "11"} {"userID":"1", "deviceID": "12"} {"userID":"1", "deviceID": "13"} {"userID":"2", "deviceID": "21"} {"userID":"2", "deviceID": "22"} {"userID":"3… --- ## [XML filter in logstash](https://discuss.elastic.co/t/xml-filter-in-logstash/250291)
**Author:** [@SSSPR](https://discuss.elastic.co/u/SSSPR)\ **Replies:** 11\ **Last updated:** [October 2, 2020, 5:16pm UTC](https://discuss.elastic.co/t/xml-filter-in-logstash/250291 "2020-10-02T17:16:01Z")
Hi All, I want to match the message name and value but both are linked with a variable p , how to extract each of the message name and match with corresponding value using the variable p with respective corresponding ob… --- ## [Could not execute action: PipelineAction::Create](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create/250802)
**Author:** [@Dung\_Tri\_Le](https://discuss.elastic.co/u/Dung_Tri_Le)\ **Replies:** 4\ **Last updated:** [October 2, 2020, 4:12pm UTC](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create/250802 "2020-10-02T16:12:50Z")
Hello, I'm totally new user for ELK stask and I try to make Logstask working with my logfile format on an ubuntu stand alone server with docker. Kibana, Elastic-search, and logstash containers with the default config i… --- ## [Logstash don't take filebeat logs which is i added in filebeat.yml](https://discuss.elastic.co/t/logstash-dont-take-filebeat-logs-which-is-i-added-in-filebeat-yml/250655)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 4\ **Last updated:** [October 2, 2020, 3:01pm UTC](https://discuss.elastic.co/t/logstash-dont-take-filebeat-logs-which-is-i-added-in-filebeat-yml/250655 "2020-10-02T15:01:04Z")
I installed latest version 7.9.2 for both logstash and filebeat. logstash successfully started api endpoint but don't take logs of filebeat. which i added path of my logs in filebeat log input. i added my logs path but… --- ## [Setting Alerting configuration on Kibana 7.9](https://discuss.elastic.co/t/setting-alerting-configuration-on-kibana-7-9/250530)
**Author:** [@ushma.shah](https://discuss.elastic.co/u/ushma.shah)\ **Replies:** 11\ **Last updated:** [October 2, 2020, 2:53pm UTC](https://discuss.elastic.co/t/setting-alerting-configuration-on-kibana-7-9/250530 "2020-10-02T14:53:38Z")
I am using basic license for elastic search with on-premises deployment without security. I want to get started with Alerting and Actions in Kibana 7.9. I have set xpack.encryptedSavedObjects.encryptionKey value as pe… --- ## [Can't delete empty field](https://discuss.elastic.co/t/cant-delete-empty-field/250678)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 6\ **Last updated:** [October 2, 2020, 2:50pm UTC](https://discuss.elastic.co/t/cant-delete-empty-field/250678 "2020-10-02T14:50:30Z")
Hi, i want to parse a json response and load data into elasticsearch via logstash , i try to delete arrDate and depDate fields if it's empty ,,,but i still got this error, \[2020-10-01T13:24:13,646\]\[WARN \]\[logstash.out… --- ## [How to use select bucket aggregation](https://discuss.elastic.co/t/how-to-use-select-bucket-aggregation/250181)
**Author:** [@wifi](https://discuss.elastic.co/u/wifi)\ **Replies:** 4\ **Last updated:** [October 2, 2020, 2:43pm UTC](https://discuss.elastic.co/t/how-to-use-select-bucket-aggregation/250181 "2020-10-02T14:43:36Z")
I want to get a list of processes that utilize more than 1% CPU. So I created a nested bucket aggregation query which ends in an avg aggregation: GET metricbeat-\*/\_search { "aggs": { "host": { "terms": { … --- ## [Profile API/ Debugging Slow queries](https://discuss.elastic.co/t/profile-api-debugging-slow-queries/250800)
**Author:** [@abhinav\_elk](https://discuss.elastic.co/u/abhinav_elk)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 2:20pm UTC](https://discuss.elastic.co/t/profile-api-debugging-slow-queries/250800 "2020-10-02T14:20:23Z")
I am using Elastic Version Number - 5.6.16 Lucene\_version - 6.6.1 I have some slow queries running on production environment. Upon profiling the requests, and adding the total times of individual components on differ… --- ## [Aggregation & max value from doc\_count](https://discuss.elastic.co/t/aggregation-max-value-from-doc-count/250376)
**Author:** [@KRISHNAPRASAD\_HG](https://discuss.elastic.co/u/KRISHNAPRASAD_HG)\ **Replies:** 2\ **Last updated:** [October 2, 2020, 2:04pm UTC](https://discuss.elastic.co/t/aggregation-max-value-from-doc-count/250376 "2020-10-02T14:04:10Z")
Hello, Hello, I need help for aggregation query. Query : GET index-\*/\_search { "aggs": { "genres": { "terms": { "field": "COMPONENT\_NAME", "min\_doc\_count": 5 } } }, "query": { "bool": { "filter": \[ { "r… --- ## [Upgrading Elastic Search from 5.6.3 to 6.8.2](https://discuss.elastic.co/t/upgrading-elastic-search-from-5-6-3-to-6-8-2/250797)
**Author:** [@Philippe1](https://discuss.elastic.co/u/Philippe1)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 1:54pm UTC](https://discuss.elastic.co/t/upgrading-elastic-search-from-5-6-3-to-6-8-2/250797 "2020-10-02T13:54:19Z")
Hi, I'm upgrading my ES node from 5.6.3 to 6.8.2 on Linux CentOS 7.8. After installing ES, it fails to start with this error : Exception in thread "main" java.lang.NoSuchMethodError: org.elasticsearch.common.xcontent.… --- ## [Install and configure elasticsearch 6.8 and 7.9 on a Single Ubuntu server](https://discuss.elastic.co/t/install-and-configure-elasticsearch-6-8-and-7-9-on-a-single-ubuntu-server/250785)
**Author:** [@hemangc](https://discuss.elastic.co/u/hemangc)\ **Replies:** 3\ **Last updated:** [October 2, 2020, 1:52pm UTC](https://discuss.elastic.co/t/install-and-configure-elasticsearch-6-8-and-7-9-on-a-single-ubuntu-server/250785 "2020-10-02T13:52:43Z")
Hi Guys, I am trying to configure Elasticsearch version 6.8 and version 7.9 on one Ubuntu server. Both versions will be communicate with diffrent TCP port. I am having a trouble while configuring this scenarion in one … --- ## [Creating a counter out of log occurrences and scraping the log itself](https://discuss.elastic.co/t/creating-a-counter-out-of-log-occurrences-and-scraping-the-log-itself/250778)
**Author:** [@Jarek1](https://discuss.elastic.co/u/Jarek1)\ **Replies:** 1\ **Last updated:** [October 2, 2020, 1:38pm UTC](https://discuss.elastic.co/t/creating-a-counter-out-of-log-occurrences-and-scraping-the-log-itself/250778 "2020-10-02T13:38:26Z")
Hi Community, can you please advice me how to proceed with the following topic? I am feeding unstructured logs from 3rd party application to the logstash via UDP feed. Logs are categorized by using grok script. For som… --- ## [Overriding logstash.yml in pipelines.yml](https://discuss.elastic.co/t/overriding-logstash-yml-in-pipelines-yml/250757)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 1\ **Last updated:** [October 2, 2020, 1:36pm UTC](https://discuss.elastic.co/t/overriding-logstash-yml-in-pipelines-yml/250757 "2020-10-02T13:36:49Z")
Hi, I am testing a pipelined configuration using the output isolator pattern. While this I am still using pipeline batch/worker settings from logstash.yml and would like to use pipeline.ordered for one of the outputs. W… --- ## [Logstash file output. gzip performance](https://discuss.elastic.co/t/logstash-file-output-gzip-performance/250761)
**Author:** [@Samuel\_Delepiere](https://discuss.elastic.co/u/Samuel_Delepiere)\ **Replies:** 1\ **Last updated:** [October 2, 2020, 1:34pm UTC](https://discuss.elastic.co/t/logstash-file-output-gzip-performance/250761 "2020-10-02T13:34:28Z")
Hi, I have a logstash instance ingesting large amount of log files data from Kafka. I write them back to disk using the file output plugin. From a performance point of view, is it better to get logstash to zip on the f… --- ## [Can not parse valid JSON message in LogStash](https://discuss.elastic.co/t/can-not-parse-valid-json-message-in-logstash/250791)
**Author:** [@dencowboy](https://discuss.elastic.co/u/dencowboy)\ **Replies:** 0\ **Last updated:** [October 2, 2020, 1:27pm UTC](https://discuss.elastic.co/t/can-not-parse-valid-json-message-in-logstash/250791 "2020-10-02T13:27:00Z")
I'm using ElasticSearch 7.7 and logstash 7.9.2. This is my configuration: input { kinesis { kinesis\_stream\_name =\> "xxx-${ENV}" application\_name =\> "xxx-${ENV}" region =\> "eu-west-1" tags =\> \["log"\] … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1771) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1773)