# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=414 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 415 --- ## [Parsing lines that only match a start string](https://discuss.elastic.co/t/parsing-lines-that-only-match-a-start-string/215125)
**Author:** [@whoatemyjam](https://discuss.elastic.co/u/whoatemyjam)\ **Replies:** 3\ **Last updated:** [January 27, 2020, 9:57am UTC](https://discuss.elastic.co/t/parsing-lines-that-only-match-a-start-string/215125 "2020-01-27T09:57:59Z")
hi Please if you can help. How can we get logstash to log only the lines starting with "central-logging": "true" and ignore rest which we dont need. { "central-logging": "true", "log-date": "2020-01-15 11:19:08 UTC",… --- ## [To run elastic search and logstash on different server](https://discuss.elastic.co/t/to-run-elastic-search-and-logstash-on-different-server/216361)
**Author:** [@amanpradhan07](https://discuss.elastic.co/u/amanpradhan07)\ **Replies:** 3\ **Last updated:** [January 27, 2020, 9:42am UTC](https://discuss.elastic.co/t/to-run-elastic-search-and-logstash-on-different-server/216361 "2020-01-27T09:42:36Z")
Hello everyone, I am using logstash on a linux based server . I am collecting the logs on this server and after processing the logs with logstash i want to send these logs to elastic search which is on an amazon ec-2 in… --- ## [Grok for WebSphere log](https://discuss.elastic.co/t/grok-for-websphere-log/216599)
**Author:** [@swapnalichingonda](https://discuss.elastic.co/u/swapnalichingonda)\ **Replies:** 1\ **Last updated:** [January 27, 2020, 9:17am UTC](https://discuss.elastic.co/t/grok-for-websphere-log/216599 "2020-01-27T09:17:46Z")
Hi, please help me to define grok for below log { "sysdate":"\[08/Jun/2019:00:00:12 -0400\]", "site":"cdbeser.net", "host":"lmw.net", "method":"POST", "request":"/services/servicePath", "querystring":"", "port":"4413", "… --- ## [Split json with no seperator](https://discuss.elastic.co/t/split-json-with-no-seperator/216368)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 1\ **Last updated:** [January 27, 2020, 7:37am UTC](https://discuss.elastic.co/t/split-json-with-no-seperator/216368 "2020-01-27T07:37:18Z")
Hi All, I have a nxlog tcp out marked towards Logstash, Here's the input from nxlog {"EventReceivedTime":"2020-01-21T03:01:37.025293-07:00", "Severity":"INFO", "EventTime":"2020-01-21T03:01:37.025296-07:00", "Hos… --- ## [Intermittent connectivity issues between Filebeat and Logstash](https://discuss.elastic.co/t/intermittent-connectivity-issues-between-filebeat-and-logstash/216286)
**Author:** [@Sagar\_Kalburgi](https://discuss.elastic.co/u/Sagar_Kalburgi)\ **Replies:** 2\ **Last updated:** [January 26, 2020, 11:58pm UTC](https://discuss.elastic.co/t/intermittent-connectivity-issues-between-filebeat-and-logstash/216286 "2020-01-26T23:58:51Z")
I have few Filebeat containers and a single logstash container running on a paticular namespaces of a kubernetes cluster. However, I see intermittent connectivity issues between Filebeat and Logstash. (Logs below) Due … --- ## [Elastic-charts](https://discuss.elastic.co/t/elastic-charts/216567)
**Author:** [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Replies:** 0\ **Last updated:** [January 26, 2020, 11:58am UTC](https://discuss.elastic.co/t/elastic-charts/216567 "2020-01-26T11:58:22Z")
hi, I'm not sure if that's the correct forum to ask... I'm checking the elastic-charts lib (https://elastic.github.io/elastic-charts), and couldn't find click event handlers for pie charts thanks Lior --- ## [Huge json data, how to stream records fast to filter and to output](https://discuss.elastic.co/t/huge-json-data-how-to-stream-records-fast-to-filter-and-to-output/216543)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 2\ **Last updated:** [January 26, 2020, 5:23am UTC](https://discuss.elastic.co/t/huge-json-data-how-to-stream-records-fast-to-filter-and-to-output/216543 "2020-01-26T05:23:27Z")
Hi All I have a json data of overall 50, 000 records coming from http input plugin. When i get the overall response i get it in the column/event : message ( all 50 K records) Now before leaving to filter i want to pro… --- ## [I wanna restart logstash, but I get multiple instance error](https://discuss.elastic.co/t/i-wanna-restart-logstash-but-i-get-multiple-instance-error/216468)
**Author:** [@lilyyy](https://discuss.elastic.co/u/lilyyy)\ **Replies:** 2\ **Last updated:** [January 26, 2020, 12:57am UTC](https://discuss.elastic.co/t/i-wanna-restart-logstash-but-i-get-multiple-instance-error/216468 "2020-01-26T00:57:35Z")
Hi. I have a critical problem to me. When I start Logstash at first, It works well. But after Killing the logstash and restarting, I have a error message. When I turn off and on ubuntu again, then It works well again… --- ## [How to map nested json array with http output plugin](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 6\ **Last updated:** [January 25, 2020, 8:43pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950 "2020-01-25T20:43:52Z")
Hi All I have incoming events from input as 2 records Project\_Num, Task Name,Task\_num,Task Date PRJ-001,Task Name1,111,1-Jan-2020 PRJ-001,Task Name2,112,2-Jan-2020 How should i map it with my json payload like bel… --- ## [Logstash 5044 address already in use](https://discuss.elastic.co/t/logstash-5044-address-already-in-use/216416)
**Author:** [@living\_jordi](https://discuss.elastic.co/u/living_jordi)\ **Replies:** 1\ **Last updated:** [January 25, 2020, 6:28pm UTC](https://discuss.elastic.co/t/logstash-5044-address-already-in-use/216416 "2020-01-25T18:28:11Z")
I'm running logstash docker image 7.5.2 using this configuration file: input { beats { port =\> 5044 } } filter { mutate { add\_tag =\> \[ "logstash\_filter\_applied" \] } } output { elasticsearch { hos… --- ## [JUnit 5 migration](https://discuss.elastic.co/t/junit-5-migration/216527)
**Author:** [@benoitdupont](https://discuss.elastic.co/u/benoitdupont)\ **Replies:** 0\ **Last updated:** [January 25, 2020, 3:19pm UTC](https://discuss.elastic.co/t/junit-5-migration/216527 "2020-01-25T15:19:07Z")
Hello Logstash team, when browsing the Java code I saw that JUnit version 4.12 is still used, instead of the latest 5.6. If it's not already on the agenda, here's a proposition of a migration plan: First, update the … --- ## [Logstash parsing data from multiple filebeats](https://discuss.elastic.co/t/logstash-parsing-data-from-multiple-filebeats/216289)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 1\ **Last updated:** [January 23, 2020, 4:07pm UTC](https://discuss.elastic.co/t/logstash-parsing-data-from-multiple-filebeats/216289 "2020-01-23T16:07:26Z")
Hey guys! I'm wondering if there is any solution for following scenario: I have one machine on which I have set Elasticsearch and Logstash and shipping there logs via Filebeat from another machine. I'd like to add a ne… --- ## [All files collected under on index only. Is it possible to have multiple indexes for multiple files?](https://discuss.elastic.co/t/all-files-collected-under-on-index-only-is-it-possible-to-have-multiple-indexes-for-multiple-files/212806)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 75\ **Last updated:** [January 25, 2020, 1:34am UTC](https://discuss.elastic.co/t/all-files-collected-under-on-index-only-is-it-possible-to-have-multiple-indexes-for-multiple-files/212806 "2020-01-25T01:34:10Z")
EVery time I search GET cat/index? in kibana, I only get one index under all files are collected. I want to get three indexes for three different log files given in filebeat. This is logstash.conf file input { bea… --- ## [How to send logs from server to local machine to VM. where ELK is running?](https://discuss.elastic.co/t/how-to-send-logs-from-server-to-local-machine-to-vm-where-elk-is-running/216120)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 18\ **Last updated:** [January 25, 2020, 1:19am UTC](https://discuss.elastic.co/t/how-to-send-logs-from-server-to-local-machine-to-vm-where-elk-is-running/216120 "2020-01-25T01:19:12Z")
I have filebeat running on server which collects logs and ships them to logstash. But I want to try to send logs to logstash on my machine. Not on localhost, but on my windows machine to VM. So it will basically be a di… --- ## [Logstash and painless: is logstash going to be replaced by ingest nodes?](https://discuss.elastic.co/t/logstash-and-painless-is-logstash-going-to-be-replaced-by-ingest-nodes/216393)
**Author:** [@Flavio\_Pompermaier](https://discuss.elastic.co/u/Flavio_Pompermaier)\ **Replies:** 2\ **Last updated:** [January 24, 2020, 9:21pm UTC](https://discuss.elastic.co/t/logstash-and-painless-is-logstash-going-to-be-replaced-by-ingest-nodes/216393 "2020-01-24T21:21:16Z")
Hello everybody, I've recently update from ES 6.8 to 7.5 and I found that Beats + Elastic pipelines are getting over Logstash (at least for ECS), and this is somehow very weird I think. Beats should provide Logstash pi… --- ## [Logstash monitorin in kibana](https://discuss.elastic.co/t/logstash-monitorin-in-kibana/216070)
**Author:** [@mende](https://discuss.elastic.co/u/mende)\ **Replies:** 4\ **Last updated:** [January 24, 2020, 6:22pm UTC](https://discuss.elastic.co/t/logstash-monitorin-in-kibana/216070 "2020-01-24T18:22:12Z")
Hi Team, I installed logstash version 7.4.2 in through Kubernetes my elastic cluster and kibana running well. If I see the monitoring in kibana dashboard, its showing only elastic cluster and kibana but its not showin… --- ## [Logstash \_split\_type\_failure](https://discuss.elastic.co/t/logstash-split-type-failure/216018)
**Author:** [@ramypala](https://discuss.elastic.co/u/ramypala)\ **Replies:** 7\ **Last updated:** [January 24, 2020, 6:12pm UTC](https://discuss.elastic.co/t/logstash-split-type-failure/216018 "2020-01-24T18:12:32Z")
My Raw Data {"records":\[{"parent":"","made\_sla":"true","caused\_by":"","watch\_list":"","upon\_reject":"cancel","sys\_updated\_on":"2020-01-22 05:47:33","child\_incidents":"0","u\_incident\_age\_in\_days":"1970-07-21 00:13:56","ho… --- ## [Create documents from array field](https://discuss.elastic.co/t/create-documents-from-array-field/216320)
**Author:** [@nino](https://discuss.elastic.co/u/nino)\ **Replies:** 6\ **Last updated:** [January 24, 2020, 5:11pm UTC](https://discuss.elastic.co/t/create-documents-from-array-field/216320 "2020-01-24T17:11:15Z")
Hello all, this is the output of my pipeline that process a file. I've succesfully extract the lines i want from the file, each file per day and n number of lines each day and split the lines into and array. { "host… --- ## [JVM error in JDBC input logstash](https://discuss.elastic.co/t/jvm-error-in-jdbc-input-logstash/216466)
**Author:** [@rguptarg](https://discuss.elastic.co/u/rguptarg)\ **Replies:** 1\ **Last updated:** [January 24, 2020, 5:08pm UTC](https://discuss.elastic.co/t/jvm-error-in-jdbc-input-logstash/216466 "2020-01-24T17:08:53Z")
Hi Team, We are using JDBC input for oracle connectivity and executing multiple queries. Now I am getting JVM error in logs. Can you suggest RAM for JDBC input, to avoid this error. Current RAM size in 15GB and logsta… --- ## [Null Field Name](https://discuss.elastic.co/t/null-field-name/216457)
**Author:** [@kdp494](https://discuss.elastic.co/u/kdp494)\ **Replies:** 0\ **Last updated:** [January 24, 2020, 3:36pm UTC](https://discuss.elastic.co/t/null-field-name/216457 "2020-01-24T15:36:31Z")
Hello, I am getting an error message "field name cannot be an empty string" for a number of documents coming into logstash. I am attempting to remove the null fields with a mutate filter: mutate { remove\_field =\> \[ "\[… --- ## [Grok regex for a few words](https://discuss.elastic.co/t/grok-regex-for-a-few-words/216435)
**Author:** [@111249](https://discuss.elastic.co/u/111249)\ **Replies:** 2\ **Last updated:** [January 24, 2020, 3:16pm UTC](https://discuss.elastic.co/t/grok-regex-for-a-few-words/216435 "2020-01-24T15:16:04Z")
My line from filebeat looks like this: 2020-01-09T15: 19: 09.939859Z message = \\"Some 'message' here \\", ourProductID = 'AS6167527'. What should the regular expression look like to get the message field? I currently ha… --- ## [Grok crash if line ending by backslash](https://discuss.elastic.co/t/grok-crash-if-line-ending-by-backslash/216441)
**Author:** [@djgreg13](https://discuss.elastic.co/u/djgreg13)\ **Replies:** 1\ **Last updated:** [January 24, 2020, 3:00pm UTC](https://discuss.elastic.co/t/grok-crash-if-line-ending-by-backslash/216441 "2020-01-24T15:00:42Z")
Hello, i have a field pushed by a beat with this content "D:\\Logs\\IIS\\test.domain.com\\20191219.log So i write the following grok for extract test.domain.com grok { match =\> { "\[log\]… --- ## [Help with Logstash xml filter plug in](https://discuss.elastic.co/t/help-with-logstash-xml-filter-plug-in/216131)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 8\ **Last updated:** [January 24, 2020, 2:16pm UTC](https://discuss.elastic.co/t/help-with-logstash-xml-filter-plug-in/216131 "2020-01-24T14:16:24Z")
I'm having trouble parsing out fields from my xml file using the logstash xml filter plugin. Here is my xml file data-- 733812 JVM|Garbage Collection|Number of Minor Collections Per Min Application Infrastructure… --- ## [Centralized Syslog management for Cisco Switches](https://discuss.elastic.co/t/centralized-syslog-management-for-cisco-switches/216253)
**Author:** [@xkazuuma](https://discuss.elastic.co/u/xkazuuma)\ **Replies:** 5\ **Last updated:** [January 24, 2020, 1:48pm UTC](https://discuss.elastic.co/t/centralized-syslog-management-for-cisco-switches/216253 "2020-01-24T13:48:35Z")
Hello everyone! I set up an ELK-Stack which is already running, receiving some logs and also showing everything in kibana. But I have problems with the filtering and properly use of the data because everything is just i… --- ## [Using replace in Logstash](https://discuss.elastic.co/t/using-replace-in-logstash/216227)
**Author:** [@saravana\_hariharan](https://discuss.elastic.co/u/saravana_hariharan)\ **Replies:** 4\ **Last updated:** [January 24, 2020, 12:09pm UTC](https://discuss.elastic.co/t/using-replace-in-logstash/216227 "2020-01-24T12:09:08Z")
if \[score\] \>= "35" mutate replace =\> \[ "score","Success" \] my error is if score Greater than equal to 35 value considered as success so im displaying graph in success, but it only shows 35 to 99 values in label nam… --- ## [Unable to send full event to syslog](https://discuss.elastic.co/t/unable-to-send-full-event-to-syslog/215339)
**Author:** [@Necus](https://discuss.elastic.co/u/Necus)\ **Replies:** 3\ **Last updated:** [January 24, 2020, 11:38am UTC](https://discuss.elastic.co/t/unable-to-send-full-event-to-syslog/215339 "2020-01-24T11:38:17Z")
Hi, I'm trying to send Windows logs through Windows Event Forwarder, through Winlogbeat and Logstash to syslog. (System1-\>System2 with Winlogbeat-\>Logstash-\>syslog). For some reason event which appear in syslog is cropp… --- ## [Customized Index pattern](https://discuss.elastic.co/t/customized-index-pattern/216247)
**Author:** [@Yashwant\_Shettigar](https://discuss.elastic.co/u/Yashwant_Shettigar)\ **Replies:** 2\ **Last updated:** [January 24, 2020, 11:32am UTC](https://discuss.elastic.co/t/customized-index-pattern/216247 "2020-01-24T11:32:06Z")
Hello, I have a centralized ELK server, where data is getting shipped via winlogbeat and filebeat installed on client machines. Now I am trying to route cisco-asa devices logs to this server. Upto some extent I am bit s… --- ## [Logstash java.lang.IllegalStateException](https://discuss.elastic.co/t/logstash-java-lang-illegalstateexception/216407)
**Author:** [@Nusko](https://discuss.elastic.co/u/Nusko)\ **Replies:** 0\ **Last updated:** [January 24, 2020, 11:14am UTC](https://discuss.elastic.co/t/logstash-java-lang-illegalstateexception/216407 "2020-01-24T11:14:05Z")
Hello, I am new to logstash (and so ELK in general). I was trying to set ELK for netflow and I am constantly facing following error: \[ERROR\]\[logstash.agent \] An exception happened when converging configuratio… --- ## [Create new field from existing message match grok filter](https://discuss.elastic.co/t/create-new-field-from-existing-message-match-grok-filter/216387)
**Author:** [@dhody\_rhmd](https://discuss.elastic.co/u/dhody_rhmd)\ **Replies:** 1\ **Last updated:** [January 24, 2020, 8:13am UTC](https://discuss.elastic.co/t/create-new-field-from-existing-message-match-grok-filter/216387 "2020-01-24T08:13:28Z")
I have a log looks like this 2020-01-24 06:49:05\] local.ERROR: syntax error, unexpected ''/home);' (T\_ENCAPSED\_AND\_WHITESPACE) {"exception":"\[object\] (Symfony\\Component\\Debug\\Exception\\FatalThrowableError(code: 0): syn… --- ## [Logstash not closing threads](https://discuss.elastic.co/t/logstash-not-closing-threads/213156)
**Author:** [@sderungs](https://discuss.elastic.co/u/sderungs)\ **Replies:** 0\ **Last updated:** [December 27, 2019, 8:12am UTC](https://discuss.elastic.co/t/logstash-not-closing-threads/213156 "2019-12-27T08:12:27Z")
Hi We're running an Elastic stack on version 7.3.0 with 2 Logstash instances on dedicated VMs each. The problem we're facing is that Logstash's thread count has grown over 4000 threads over the past 3-4 weeks or so. It… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=413) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=415)