# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=251 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 252 --- ## [Filebeat logstash loadbalance to kafka](https://discuss.elastic.co/t/filebeat-logstash-loadbalance-to-kafka/265695)
**Author:** [@cool](https://discuss.elastic.co/u/cool)\ **Replies:** 1\ **Last updated:** [February 27, 2021, 4:14pm UTC](https://discuss.elastic.co/t/filebeat-logstash-loadbalance-to-kafka/265695 "2021-02-27T16:14:22Z")
How to send logs from filebeat/logstash to kafak to multiple datacenters if one datacenter is down logs to be sent to another datacenter. --- ## [Logstash split message by \\n into multiple message inputs](https://discuss.elastic.co/t/logstash-split-message-by-n-into-multiple-message-inputs/265620)
**Author:** [@Daniel\_Jankech](https://discuss.elastic.co/u/Daniel_Jankech)\ **Replies:** 1\ **Last updated:** [February 27, 2021, 1:08pm UTC](https://discuss.elastic.co/t/logstash-split-message-by-n-into-multiple-message-inputs/265620 "2021-02-27T13:08:40Z")
Hi , what im trying to do is pretty simple and straight forward but cant figure out why its not working. When I was using logstash with file input it parsed my logs correctly meaning that it separated my log file by \\n i… --- ## [Best blueprint/architecture/best practice to monitor Logstash pipelines with alerts in Kibana?](https://discuss.elastic.co/t/best-blueprint-architecture-best-practice-to-monitor-logstash-pipelines-with-alerts-in-kibana/265677)
**Author:** [@byteandbit](https://discuss.elastic.co/u/byteandbit)\ **Replies:** 0\ **Last updated:** [February 27, 2021, 3:57am UTC](https://discuss.elastic.co/t/best-blueprint-architecture-best-practice-to-monitor-logstash-pipelines-with-alerts-in-kibana/265677 "2021-02-27T03:57:31Z")
Hello, We are looking for best practice to monitor our Logstash (several pipelines with various input and output plugins) and alert via Slack or eMail in case of problems. Do we just read Logstash logs? If so again wha… --- ## [Get count of lines in a multiline message field](https://discuss.elastic.co/t/get-count-of-lines-in-a-multiline-message-field/265649)
**Author:** [@Pavani\_Reddy](https://discuss.elastic.co/u/Pavani_Reddy)\ **Replies:** 2\ **Last updated:** [February 26, 2021, 8:58pm UTC](https://discuss.elastic.co/t/get-count-of-lines-in-a-multiline-message-field/265649 "2021-02-26T20:58:16Z")
Hi Team, I have a message field with multiline i.e Exception in thread "main" java.lang.IllegalStateException: A book has a null property\\n\\tat com.example.myproject.Author.getBookIds(Author.java:38) \\n\\tat com.exampl… --- ## [Translate Regex Not Working](https://discuss.elastic.co/t/translate-regex-not-working/265628)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 11\ **Last updated:** [February 26, 2021, 7:02pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628 "2021-02-26T19:02:38Z")
Regular Expression on Translate filter isn't working as expected. All values in the destination field are matching low. Values will range from 0.0 to 10 with no more than 1 digit after the decimal. translate { fie… --- ## [Multiple file paths for the path parameter in Logstash 7.10](https://discuss.elastic.co/t/multiple-file-paths-for-the-path-parameter-in-logstash-7-10/265640)
**Author:** [@lostsoul352](https://discuss.elastic.co/u/lostsoul352)\ **Replies:** 0\ **Last updated:** [February 26, 2021, 5:03pm UTC](https://discuss.elastic.co/t/multiple-file-paths-for-the-path-parameter-in-logstash-7-10/265640 "2021-02-26T17:03:13Z")
According to the Logstash 7.10 documentation the Path parameter for logstash file input is an array. I tried this: file { path =\> \[ "/data/logs/testing/servers/\*\*/server\*.log", "/data/logs/testing/server… --- ## [Querying specific documents by doc \_id via elasticsearch filter plugin in logstash](https://discuss.elastic.co/t/querying-specific-documents-by-doc-id-via-elasticsearch-filter-plugin-in-logstash/265630)
**Author:** [@pratz](https://discuss.elastic.co/u/pratz)\ **Replies:** 0\ **Last updated:** [February 26, 2021, 3:57pm UTC](https://discuss.elastic.co/t/querying-specific-documents-by-doc-id-via-elasticsearch-filter-plugin-in-logstash/265630 "2021-02-26T15:57:25Z")
Using the elasticsearch filter plugin in logstash, can I fetch a particular document via it's \_id (which I obtain from my input data in logstash)? --- ## [Split event into multiples and parse with grok separately](https://discuss.elastic.co/t/split-event-into-multiples-and-parse-with-grok-separately/265322)
**Author:** [@Daniel\_Jankech](https://discuss.elastic.co/u/Daniel_Jankech)\ **Replies:** 2\ **Last updated:** [February 26, 2021, 2:45pm UTC](https://discuss.elastic.co/t/split-event-into-multiples-and-parse-with-grok-separately/265322 "2021-02-26T14:45:27Z")
Hello, I'm back with another question. From my understanding split function is used to split lets say messages into multiple lines of code and then you use grok to parse this into a single list of variables. However what… --- ## [Need some help with Logstash and the XML filter plugin](https://discuss.elastic.co/t/need-some-help-with-logstash-and-the-xml-filter-plugin/265227)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 10\ **Last updated:** [February 26, 2021, 8:47am UTC](https://discuss.elastic.co/t/need-some-help-with-logstash-and-the-xml-filter-plugin/265227 "2021-02-26T08:47:37Z")
Hello Im trying to parse a XML that Im getting from a Windows Event Viewer. Ill try to explain as best I can and any questions or doubt can be further asked. I have this in a field that is winlog.event\_data.param2 (va… --- ## [Failed to parse field \[agent\] of type \[text\]](https://discuss.elastic.co/t/failed-to-parse-field-agent-of-type-text/265361)
**Author:** [@meissen](https://discuss.elastic.co/u/meissen)\ **Replies:** 4\ **Last updated:** [February 26, 2021, 8:55am UTC](https://discuss.elastic.co/t/failed-to-parse-field-agent-of-type-text/265361 "2021-02-26T08:55:56Z")
Hello, ELK stack in version 7.9.2 it has been working fine with filebeat 6.4, but after upgrading filebeat to 7.0 I get the following error in logstash logs : \[2021-02-24T15:24:56,954\]\[WARN \]\[logstash.outputs.elasticse… --- ## [Logstash not receiving events from Kafka](https://discuss.elastic.co/t/logstash-not-receiving-events-from-kafka/263545)
**Author:** [@Boris\_Joseph](https://discuss.elastic.co/u/Boris_Joseph)\ **Replies:** 1\ **Last updated:** [February 26, 2021, 7:16am UTC](https://discuss.elastic.co/t/logstash-not-receiving-events-from-kafka/263545 "2021-02-26T07:16:07Z")
Hi team, I have a ELK Stack with 2 Logstash Nodes in DataCentre1 and a Kafka Cluster in another DataCentre2 accepting log events from servers in DataCentre2 . SitetoSite VPN connects the DC1 to DC2. Telnet from Kafka … --- ## [Troubles running Logstash on RHEL](https://discuss.elastic.co/t/troubles-running-logstash-on-rhel/265472)
**Author:** [@PawelKosiorek](https://discuss.elastic.co/u/PawelKosiorek)\ **Replies:** 4\ **Last updated:** [February 26, 2021, 7:12am UTC](https://discuss.elastic.co/t/troubles-running-logstash-on-rhel/265472 "2021-02-26T07:12:31Z")
It used to work fine, but recently it has been giving me this. Please help. Thank you. --- ## [Grook Format For APM Nginx Log](https://discuss.elastic.co/t/grook-format-for-apm-nginx-log/265439)
**Author:** [@reza\_naipospos](https://discuss.elastic.co/u/reza_naipospos)\ **Replies:** 2\ **Last updated:** [February 26, 2021, 4:25am UTC](https://discuss.elastic.co/t/grook-format-for-apm-nginx-log/265439 "2021-02-26T04:25:40Z")
i have format log like this, how logstash can make new field on elasticsearch for every this log, currently all log is in message field. timestamp="25/Feb/2021:13:34:22 +0700" client=192.168.0.21 request="GET /api/Supir… --- ## [Does not match the certificate subject provided by the peer](https://discuss.elastic.co/t/does-not-match-the-certificate-subject-provided-by-the-peer/265484)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 3\ **Last updated:** [February 25, 2021, 10:57pm UTC](https://discuss.elastic.co/t/does-not-match-the-certificate-subject-provided-by-the-peer/265484 "2021-02-25T22:57:35Z")
Host name 'localhost' does not match the certificate subject provided by the peer (CN=elasticsearch) How can I deal with it? Can I just verify certificate? I don't want hardcode any IP etc. In /etc/logstash/logstash.ym… --- ## [HTTP input plugin customize response-header](https://discuss.elastic.co/t/http-input-plugin-customize-response-header/265547)
**Author:** [@stemons](https://discuss.elastic.co/u/stemons)\ **Replies:** 1\ **Last updated:** [February 25, 2021, 9:54pm UTC](https://discuss.elastic.co/t/http-input-plugin-customize-response-header/265547 "2021-02-25T21:54:02Z")
Hello! I'm working with logstash and the http input plugin. I've a datasource that send a data stream to logstash and require as response header content-type =\> application/json. I'm using the following configuration, bu… --- ## [Logstash on EKS and AWS elastic - 401 auth errors](https://discuss.elastic.co/t/logstash-on-eks-and-aws-elastic-401-auth-errors/265507)
**Author:** [@kyleh31](https://discuss.elastic.co/u/kyleh31)\ **Replies:** 1\ **Last updated:** [February 25, 2021, 9:29pm UTC](https://discuss.elastic.co/t/logstash-on-eks-and-aws-elastic-401-auth-errors/265507 "2021-02-25T21:29:35Z")
Hi we are getting 401 auth errors as of mid January. We have reached out to AWS support team and they were unable to help. We have had logstash running in production environment for ~6 months, but when we went to update … --- ## [Logstash warning](https://discuss.elastic.co/t/logstash-warning/265416)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [February 25, 2021, 7:12pm UTC](https://discuss.elastic.co/t/logstash-warning/265416 "2021-02-25T19:12:47Z")
I am have metricbeat running on few systems. and getting this error message on and off. can't put my fingure on about which event is generating this error. is there a way to find out? Basically metricbeat is running f… --- ## [Parse Json Logs using logstash](https://discuss.elastic.co/t/parse-json-logs-using-logstash/265517)
**Author:** [@Yashkumar\_Dabhi](https://discuss.elastic.co/u/Yashkumar_Dabhi)\ **Replies:** 1\ **Last updated:** [February 25, 2021, 5:36pm UTC](https://discuss.elastic.co/t/parse-json-logs-using-logstash/265517 "2021-02-25T17:36:03Z")
Hi, I'm trying to parse logs in logstash, My logs are in Json, {"timestamp":"2021-02-25T15:20:38.490Z","logLevel":"INFO","serviceName":"xxx","transactionId":"0000-0000-000","elapsedTime":820,"endTime":"","url":"","statu… --- ## [Sequentially execute filter plugins on listed file inputs](https://discuss.elastic.co/t/sequentially-execute-filter-plugins-on-listed-file-inputs/265496)
**Author:** [@pratz](https://discuss.elastic.co/u/pratz)\ **Replies:** 1\ **Last updated:** [February 25, 2021, 4:22pm UTC](https://discuss.elastic.co/t/sequentially-execute-filter-plugins-on-listed-file-inputs/265496 "2021-02-25T16:22:48Z")
I have two files that I input in logstash using the file plugin Is there a way that the first file is parsed first? input { file { path =\> "abc.txt" type =\> "first" } file { path =\> … --- ## [How can I check logstash throughput(event per second)?](https://discuss.elastic.co/t/how-can-i-check-logstash-throughput-event-per-second/265340)
**Author:** [@jang](https://discuss.elastic.co/u/jang)\ **Replies:** 1\ **Last updated:** [February 25, 2021, 1:33pm UTC](https://discuss.elastic.co/t/how-can-i-check-logstash-throughput-event-per-second/265340 "2021-02-25T13:33:33Z")
I am sending data from Logstash to Elasticsearch. (filebeat -\> kafka -\> logstash -\> elasticsearch) I want to know the Logstash Event Per Second(EPS) (it is assumed that the environment of Elasticsearch is the same.) P… --- ## [Dynamic data type for one field](https://discuss.elastic.co/t/dynamic-data-type-for-one-field/264844)
**Author:** [@tutpaw](https://discuss.elastic.co/u/tutpaw)\ **Replies:** 4\ **Last updated:** [February 25, 2021, 9:39am UTC](https://discuss.elastic.co/t/dynamic-data-type-for-one-field/264844 "2021-02-25T09:39:27Z")
Hi, I don't know, if this topic has been already discussed, but I didn't find an appropriate one - hence, I'm creating a new one. I've got an nginx log whose partial representation is as follows. First, the record stati… --- ## [Logstash hangs with no error](https://discuss.elastic.co/t/logstash-hangs-with-no-error/265441)
**Author:** [@user2416](https://discuss.elastic.co/u/user2416)\ **Replies:** 0\ **Last updated:** [February 25, 2021, 7:08am UTC](https://discuss.elastic.co/t/logstash-hangs-with-no-error/265441 "2021-02-25T07:08:48Z")
Hi, We are running logstash on Kubernetes (deployed using logstash helm chart from elastic). Logstash periodically hangs and stops processing data with no errors. It uses filebeat input and sends data to elasticsearch. … --- ## [Importing Dynamic named file from logstash to elasticsearch in a single index](https://discuss.elastic.co/t/importing-dynamic-named-file-from-logstash-to-elasticsearch-in-a-single-index/265434)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 0\ **Last updated:** [February 25, 2021, 5:19am UTC](https://discuss.elastic.co/t/importing-dynamic-named-file-from-logstash-to-elasticsearch-in-a-single-index/265434 "2021-02-25T05:19:21Z")
Hi Team, So let's just say I'm receiving some dynamic named files based on today's date like, somefile-25-02-2021.csv somefile-26-02-2021.csv somefile-27-02-2021.csv somefile-28-02-2021.csv and I want to increment t… --- ## [Split JSON array doesn't work with multiple config files](https://discuss.elastic.co/t/split-json-array-doesnt-work-with-multiple-config-files/265415)
**Author:** [@alicango](https://discuss.elastic.co/u/alicango)\ **Replies:** 2\ **Last updated:** [February 25, 2021, 1:09am UTC](https://discuss.elastic.co/t/split-json-array-doesnt-work-with-multiple-config-files/265415 "2021-02-25T01:09:54Z")
Hi All, I have two config files under the /etc/logstash/conf.d directory. Each config is listening to a different HTTP port. When I run the logstash as a daemon(systemctl start logstash) with both config files, I see … --- ## [Missing year set to 1970 by Logstash](https://discuss.elastic.co/t/missing-year-set-to-1970-by-logstash/265373)
**Author:** [@ManuelF](https://discuss.elastic.co/u/ManuelF)\ **Replies:** 16\ **Last updated:** [February 24, 2021, 7:46pm UTC](https://discuss.elastic.co/t/missing-year-set-to-1970-by-logstash/265373 "2021-02-24T19:46:24Z")
Hi, Running ELK 7.10.1 Recently I started to process new type of logs that does not include the year, so they come with format "MMM dd HH:mm:ss". According to the Logstash documentation (and this post), in cases like… --- ## [Elapsed time is not calculating for all requests - elapsed\_end\_without\_start](https://discuss.elastic.co/t/elapsed-time-is-not-calculating-for-all-requests-elapsed-end-without-start/262842)
**Author:** [@Ramesh535](https://discuss.elastic.co/u/Ramesh535)\ **Replies:** 1\ **Last updated:** [February 24, 2021, 6:15pm UTC](https://discuss.elastic.co/t/elapsed-time-is-not-calculating-for-all-requests-elapsed-end-without-start/262842 "2021-02-24T18:15:27Z")
Hi Everyone, Out of 1000 requests, elapsed time is not calculating for atleast 100 requests and it is throwing exception as elapsed\_end\_without\_start in tags. Can any one suggest solution to overcome this issue. Thanks… --- ## [Coverting miliseconds into HH:mm:ss in logstash](https://discuss.elastic.co/t/coverting-miliseconds-into-hhss-in-logstash/263098)
**Author:** [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Replies:** 2\ **Last updated:** [February 24, 2021, 5:58pm UTC](https://discuss.elastic.co/t/coverting-miliseconds-into-hhss-in-logstash/263098 "2021-02-24T17:58:03Z")
The below is my csv output data inserted to elasticsearch through logstash. "build\_end\_time" =\> "2021-01-13 01:29:49", "build\_duration" =\> "6409651", "build\_start\_time" =\> "2021-01-12 23:43:00", "build\_date" =\> "2021-01… --- ## [Logstash extract a nested json object](https://discuss.elastic.co/t/logstash-extract-a-nested-json-object/265307)
**Author:** [@imsystem](https://discuss.elastic.co/u/imsystem)\ **Replies:** 2\ **Last updated:** [February 24, 2021, 5:51pm UTC](https://discuss.elastic.co/t/logstash-extract-a-nested-json-object/265307 "2021-02-24T17:51:13Z")
Hello! I have input json: { "fields":{ "created\_date":1614158456696703782, "value":"\\u001B\[35m-\\u001B\[0m | \\u001B\[32m2021-02-20T12:48:10.367337+0300\\u001B\[0m | \\u001B\[1mINFO \\u001B\[0m | application.ap… --- ## [Logstash error - Failed to decode CEF payload. Generating failure event with payload in message field](https://discuss.elastic.co/t/logstash-error-failed-to-decode-cef-payload-generating-failure-event-with-payload-in-message-field/265365)
**Author:** [@testsemd\_email](https://discuss.elastic.co/u/testsemd_email)\ **Replies:** 3\ **Last updated:** [February 24, 2021, 4:38pm UTC](https://discuss.elastic.co/t/logstash-error-failed-to-decode-cef-payload-generating-failure-event-with-payload-in-message-field/265365 "2021-02-24T16:38:36Z")
I configued the syslog.conf file to get the traffic to elasticsearch(installed in same server). Syslog data traffic is coming as CEF format to logstash. syslog.conf file as bellow: input { tcp { port =\> 5514 type =\>… --- ## [UTC vs EST log events in Kibana](https://discuss.elastic.co/t/utc-vs-est-log-events-in-kibana/264921)
**Author:** [@Yuri\_Sibirski](https://discuss.elastic.co/u/Yuri_Sibirski)\ **Replies:** 5\ **Last updated:** [February 24, 2021, 4:09pm UTC](https://discuss.elastic.co/t/utc-vs-est-log-events-in-kibana/264921 "2021-02-24T16:09:45Z")
Hi there. I was not able to find a solution for my problem that is why I am creating this post. My problem is that we are sending logs to elasticsearch from linux nodes using either UTC or EST as their timezones. For EST… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=250) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=252)