# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=311 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 312 --- ## [Convert Different timezones syslog time to datetime format](https://discuss.elastic.co/t/convert-different-timezones-syslog-time-to-datetime-format/246352)
**Author:** [@Pavani\_Reddy](https://discuss.elastic.co/u/Pavani_Reddy)\ **Replies:** 3\ **Last updated:** [August 26, 2020, 4:17pm UTC](https://discuss.elastic.co/t/convert-different-timezones-syslog-time-to-datetime-format/246352 "2020-08-26T16:17:41Z")
I have following timestamp with different timezones in the log messages. Aug 11 03:31:12.942 UTC Aug 11 04:31:12.819 GMT Aug 10 22:31:51.207 CST Aug 10 04:31:38.897 PAC Aug 10 22:31:13.210 CDT Aug 11 03:31:13.048 P… --- ## [Logstash parsing input filter beat csv file](https://discuss.elastic.co/t/logstash-parsing-input-filter-beat-csv-file/246314)
**Author:** [@Carlos\_Xavier](https://discuss.elastic.co/u/Carlos_Xavier)\ **Replies:** 0\ **Last updated:** [August 25, 2020, 3:25pm UTC](https://discuss.elastic.co/t/logstash-parsing-input-filter-beat-csv-file/246314 "2020-08-25T15:25:50Z")
Hi everoene, I getting an issue between filebeat and logstash that I can not uderstand what is gonig on, I need some help with that. Let me tell you what is going on... The filebeat setting is the following.... // f… --- ## [Want to match against single grok pattern and multiple patterns in same filter](https://discuss.elastic.co/t/want-to-match-against-single-grok-pattern-and-multiple-patterns-in-same-filter/246470)
**Author:** [@saurabhc](https://discuss.elastic.co/u/saurabhc)\ **Replies:** 0\ **Last updated:** [August 26, 2020, 2:35pm UTC](https://discuss.elastic.co/t/want-to-match-against-single-grok-pattern-and-multiple-patterns-in-same-filter/246470 "2020-08-26T14:35:13Z")
I want to have below kinds of match patterns in my logstash configuration file to match different lines. (a) match =\> { "message" =\> "\\\[%{TIMESTAMP\_ISO8601:bip\_timestamp}\\\] \\\[%{WORD:app\_server}\] \\\[%{WORD:log\_level}\] \\\[… --- ## [Use part of the log files name for the index?](https://discuss.elastic.co/t/use-part-of-the-log-files-name-for-the-index/246366)
**Author:** [@hcker2000](https://discuss.elastic.co/u/hcker2000)\ **Replies:** 2\ **Last updated:** [August 26, 2020, 2:04pm UTC](https://discuss.elastic.co/t/use-part-of-the-log-files-name-for-the-index/246366 "2020-08-26T14:04:14Z")
I have my files all in the same directory names like such: /var/log/apache2/testsite1.com\_error.log /var/log/apache2/testsite1.com\_access.log /var/log/apache2/subdomain.testsite1.com\_error.log /var/log/apache2/subdomain… --- ## [Scaling with many custom log patterns](https://discuss.elastic.co/t/scaling-with-many-custom-log-patterns/246450)
**Author:** [@pcosic](https://discuss.elastic.co/u/pcosic)\ **Replies:** 0\ **Last updated:** [August 26, 2020, 12:34pm UTC](https://discuss.elastic.co/t/scaling-with-many-custom-log-patterns/246450 "2020-08-26T12:34:26Z")
Hi Community, I'm wondering how would we write a filter with logs from a firewall like cisco asa with thousands of custom logs? Currently in the filebeat module for cisco there are supported 150 cisco asa log id's. If… --- ## [Custom Logstash input plugin](https://discuss.elastic.co/t/custom-logstash-input-plugin/246437)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 0\ **Last updated:** [August 26, 2020, 11:11am UTC](https://discuss.elastic.co/t/custom-logstash-input-plugin/246437 "2020-08-26T11:11:56Z")
Hi all, Since the http\_poller has no support for paginated requests, I want to create a custom input plugin for logstash to solve this problem. is there a clear example to follow! --- ## [How to handle empty result in jdbc\_streaming filter plugin in Logstash](https://discuss.elastic.co/t/how-to-handle-empty-result-in-jdbc-streaming-filter-plugin-in-logstash/246347)
**Author:** [@naveen007](https://discuss.elastic.co/u/naveen007)\ **Replies:** 4\ **Last updated:** [August 26, 2020, 7:42am UTC](https://discuss.elastic.co/t/how-to-handle-empty-result-in-jdbc-streaming-filter-plugin-in-logstash/246347 "2020-08-26T07:42:36Z")
Hi, I'm using the jdbc\_streaming filter plugin to get data from query. But in some case query return empty result or null value. parameters =\> { "firstName" =\> "firstName" } statement =\> "SELECT DISTINCT… --- ## [Getting "\_groktimeout" error for a particular filter sometimes. Need help optimizing the filter](https://discuss.elastic.co/t/getting-groktimeout-error-for-a-particular-filter-sometimes-need-help-optimizing-the-filter/246342)
**Author:** [@saurabhc](https://discuss.elastic.co/u/saurabhc)\ **Replies:** 2\ **Last updated:** [August 26, 2020, 5:36am UTC](https://discuss.elastic.co/t/getting-groktimeout-error-for-a-particular-filter-sometimes-need-help-optimizing-the-filter/246342 "2020-08-26T05:36:04Z")
The particular line is a "multiline". Hence using "codec =\> multiline" to get the line and then applying grok filter to it. But sometimes when the particular line is having large content the parsing fails with "\_groktim… --- ## [Logstash status is active but logs showing error as "\[2020-08-25T02:48:59,350\]\[ERROR\]\[org.logstash.Logstash \] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit"](https://discuss.elastic.co/t/logstash-status-is-active-but-logs-showing-error-as-2020-08-25t0259-350-error-org-logstash-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/246245)
**Author:** [@divya\_lakshmi](https://discuss.elastic.co/u/divya_lakshmi)\ **Replies:** 1\ **Last updated:** [August 26, 2020, 4:39am UTC](https://discuss.elastic.co/t/logstash-status-is-active-but-logs-showing-error-as-2020-08-25t0259-350-error-org-logstash-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/246245 "2020-08-26T04:39:45Z")
Hi, I have been trying to add one more conf file to pipeline.yml and did troubleshooting using sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/new.conf and it worked fine.Later during another troubleshooti… --- ## [ECONNREFUSED: Connection refused - connect (2) for "localhost" port 587](https://discuss.elastic.co/t/econnrefused-connection-refused-connect-2-for-localhost-port-587/246087)
**Author:** [@David03](https://discuss.elastic.co/u/David03)\ **Replies:** 2\ **Last updated:** [August 26, 2020, 2:48am UTC](https://discuss.elastic.co/t/econnrefused-connection-refused-connect-2-for-localhost-port-587/246087 "2020-08-26T02:48:36Z")
Hi, good day. I used email config and i encounter this Connection refused - connect (2) for "localhost" port 587 output { elasticsearch { hosts =\> \["localhost:9200"\] } stdout { codec =\> rubydebug } if "shouldmail… --- ## [Need Help Parsing PS command Output using logstash](https://discuss.elastic.co/t/need-help-parsing-ps-command-output-using-logstash/244145)
**Author:** [@saurabhc](https://discuss.elastic.co/u/saurabhc)\ **Replies:** 7\ **Last updated:** [August 25, 2020, 6:17pm UTC](https://discuss.elastic.co/t/need-help-parsing-ps-command-output-using-logstash/244145 "2020-08-25T18:17:36Z")
Need Help Parsing PS command Output using logstash. My input file looks like below. Linux VM Ubuntu zzz \*\*\*Thu Jul 23 13:00:11 UTC 2020 USER PID PPID PRI %CPU %MEM VSZ RSS WCHAN S STARTED … --- ## [XML key:value Extraction](https://discuss.elastic.co/t/xml-key-value-extraction/246182)
**Author:** [@teej](https://discuss.elastic.co/u/teej)\ **Replies:** 3\ **Last updated:** [August 25, 2020, 4:51pm UTC](https://discuss.elastic.co/t/xml-key-value-extraction/246182 "2020-08-25T16:51:59Z")
Apologies, I have searched but cannot seem to find a good example on how to extract and then index the remaining data. If this is addressed elsewhere and Ive missed it please direct me there. Being new to logstash and i… --- ## [Logstash filter plugin for command line output](https://discuss.elastic.co/t/logstash-filter-plugin-for-command-line-output/246318)
**Author:** [@Arati](https://discuss.elastic.co/u/Arati)\ **Replies:** 0\ **Last updated:** [August 25, 2020, 3:51pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-for-command-line-output/246318 "2020-08-25T15:51:01Z")
Hello everyone, I am new to logstash need help. I want to send commandline output from multiple system to elasticsearch via logstash. which filter plugin suits for this? Below is the sample command line output from 2 s… --- ## [Find logstash compatibility with kafka](https://discuss.elastic.co/t/find-logstash-compatibility-with-kafka/244490)
**Author:** [@satish92](https://discuss.elastic.co/u/satish92)\ **Replies:** 2\ **Last updated:** [August 25, 2020, 3:45pm UTC](https://discuss.elastic.co/t/find-logstash-compatibility-with-kafka/244490 "2020-08-25T15:45:22Z")
Hello, I upgraded logstash from 2.4 to 7.5 and also kafka from 0.10 to 2.5 Im facing few issues here when pushing logs through logstash issue background information: Im using the same logstash config files in /etc/lo… --- ## [Anonymization of sensitive data by Logstash](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 8\ **Last updated:** [August 25, 2020, 1:45pm UTC](https://discuss.elastic.co/t/anonymization-of-sensitive-data-by-logstash/245585 "2020-08-25T13:45:24Z")
Hello, can anyone help me how to anonymize sensitive data from multline log? How to anonymize value of parameter sensitive data: sensitive data:more than one line some part on second and on third line should be repla… --- ## [Logstash not reading keystore file after upgrade to ELK 7.9](https://discuss.elastic.co/t/logstash-not-reading-keystore-file-after-upgrade-to-elk-7-9/245680)
**Author:** [@ManuelF](https://discuss.elastic.co/u/ManuelF)\ **Replies:** 1\ **Last updated:** [August 25, 2020, 12:59pm UTC](https://discuss.elastic.co/t/logstash-not-reading-keystore-file-after-upgrade-to-elk-7-9/245680 "2020-08-25T12:59:31Z")
Hi, Yesterday this was working perfectly fine. I just upgraded the server to ELK 7.9 and after restarted Logstash I am getting this error stating Logstash found the keystore file, but "... but it is not a valid Logstash… --- ## [Filebeat Apache module + Logstash: "Mapping definition for \[source\] has unsupported parameters"](https://discuss.elastic.co/t/filebeat-apache-module-logstash-mapping-definition-for-source-has-unsupported-parameters/246281)
**Author:** [@rubjurgens](https://discuss.elastic.co/u/rubjurgens)\ **Replies:** 0\ **Last updated:** [August 25, 2020, 12:25pm UTC](https://discuss.elastic.co/t/filebeat-apache-module-logstash-mapping-definition-for-source-has-unsupported-parameters/246281 "2020-08-25T12:25:46Z")
Hello, For some time now, we are using ES, Logstash and Kibana with filebeats parsing application logs from different servers. Recently we configured filebeat for the first time using a module. I followed the instructi… --- ## [How do I stop a Logtsash pipeline from also printing every document to STDOUT](https://discuss.elastic.co/t/how-do-i-stop-a-logtsash-pipeline-from-also-printing-every-document-to-stdout/246258)
**Author:** [@c1jt](https://discuss.elastic.co/u/c1jt)\ **Replies:** 0\ **Last updated:** [August 25, 2020, 9:40am UTC](https://discuss.elastic.co/t/how-do-i-stop-a-logtsash-pipeline-from-also-printing-every-document-to-stdout/246258 "2020-08-25T09:40:02Z")
Ayup, I have a Docker setup (E, L & K w/ compose) which has its Logstash parsing custom logs. The "Sophos" pipeline only outputs to an elastic node. However; Logstash It thinks its super funny to also print every singl… --- ## [Logstash keeps crashing,used 7 different ports](https://discuss.elastic.co/t/logstash-keeps-crashing-used-7-different-ports/246247)
**Author:** [@divya\_lakshmi](https://discuss.elastic.co/u/divya_lakshmi)\ **Replies:** 0\ **Last updated:** [August 25, 2020, 8:06am UTC](https://discuss.elastic.co/t/logstash-keeps-crashing-used-7-different-ports/246247 "2020-08-25T08:06:11Z")
logstash keeps crashing,used 7 different ports with 7 different conf files added to pipeline.yml. How many ports in adding multiple pipeline can be logstash 6.5 version support --- ## [Write grok for 1 Aug 14 13:30:36 date time](https://discuss.elastic.co/t/write-grok-for-1-aug-14-1336-date-time/246231)
**Author:** [@tharu85](https://discuss.elastic.co/u/tharu85)\ **Replies:** 0\ **Last updated:** [August 25, 2020, 7:02am UTC](https://discuss.elastic.co/t/write-grok-for-1-aug-14-1336-date-time/246231 "2020-08-25T07:02:17Z")
I need to write a proper date time log parser for "1 Aug 14 13:30:36". I found this log from PAM solution and forwarded to the logstash through CEF format. How ever I have tried several grok patterns to extract, but no … --- ## [Can't find logstash.conf on Elastic Cloud](https://discuss.elastic.co/t/cant-find-logstash-conf-on-elastic-cloud/246218)
**Author:** [@jmaharjan](https://discuss.elastic.co/u/jmaharjan)\ **Replies:** 1\ **Last updated:** [August 25, 2020, 5:51am UTC](https://discuss.elastic.co/t/cant-find-logstash-conf-on-elastic-cloud/246218 "2020-08-25T05:51:54Z")
Could be a really stupid question, please bear with me. I have an instance of ELK stack on the cloud. I am not sure where to find the logstash.conf file or set up the pipeline. The stack management section does not have… --- ## [How to disable all formatting/processing done by logstash](https://discuss.elastic.co/t/how-to-disable-all-formatting-processing-done-by-logstash/245377)
**Author:** [@joel.ng](https://discuss.elastic.co/u/joel.ng)\ **Replies:** 5\ **Last updated:** [August 25, 2020, 5:02am UTC](https://discuss.elastic.co/t/how-to-disable-all-formatting-processing-done-by-logstash/245377 "2020-08-25T05:02:52Z")
I am using logstash purely to upload them to my S3 bucket. My logs are already formatted nicely and I do not want any further formatting done on it. Is there any way to disable all the processing done by logstash? It's … --- ## [Multiple Match Array](https://discuss.elastic.co/t/multiple-match-array/245817)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 6\ **Last updated:** [August 25, 2020, 1:29am UTC](https://discuss.elastic.co/t/multiple-match-array/245817 "2020-08-25T01:29:48Z")
I am trying to parse a message that will have multiple items that I am trying to capture into an array. For example : message: test message @word blah blah @word test ... In this case, the number of captures can and w… --- ## [Logstash - Is it Possible To Handle Dissect Failures Without Filling Up Syslog](https://discuss.elastic.co/t/logstash-is-it-possible-to-handle-dissect-failures-without-filling-up-syslog/246044)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 3\ **Last updated:** [August 25, 2020, 1:23am UTC](https://discuss.elastic.co/t/logstash-is-it-possible-to-handle-dissect-failures-without-filling-up-syslog/246044 "2020-08-25T01:23:23Z")
Hello ELK Community, I have a pipeline that processes a certain type of log file, depending on where the log file was generated the fields may be different, this causes my dissect to fail. With the following code block… --- ## [Logstash to panos pipeline - CircuitBreakerException script.max\_compilations\_rate](https://discuss.elastic.co/t/logstash-to-panos-pipeline-circuitbreakerexception-script-max-compilations-rate/246187)
**Author:** [@nverrill](https://discuss.elastic.co/u/nverrill)\ **Replies:** 0\ **Last updated:** [August 24, 2020, 9:09pm UTC](https://discuss.elastic.co/t/logstash-to-panos-pipeline-circuitbreakerexception-script-max-compilations-rate/246187 "2020-08-24T21:09:18Z")
I am using the filebeat panw module to ingest palo-alto logs. Due to some performance testing the current path is a little convoluted, but filebeat listens on syslog port with panw module and the output sends to Redis. T… --- ## [Forwarding Rsyslog to Logstash](https://discuss.elastic.co/t/forwarding-rsyslog-to-logstash/246162)
**Author:** [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\ **Replies:** 1\ **Last updated:** [August 24, 2020, 6:31pm UTC](https://discuss.elastic.co/t/forwarding-rsyslog-to-logstash/246162 "2020-08-24T18:31:04Z")
Hello, I am forwarding my rsyslog to Logstash to which I can see traffic coming on 5001 within logstash using tcpdump command: tcpdump -Xni ens192 port 5001 and src \ However, I am not seeing this serv… --- ## [Parsing JSON input with multiple arrays as events](https://discuss.elastic.co/t/parsing-json-input-with-multiple-arrays-as-events/246040)
**Author:** [@InnerJoin](https://discuss.elastic.co/u/InnerJoin)\ **Replies:** 6\ **Last updated:** [August 24, 2020, 5:30pm UTC](https://discuss.elastic.co/t/parsing-json-input-with-multiple-arrays-as-events/246040 "2020-08-24T17:30:25Z")
Hi, I am currently using the http\_poller input to pull from an API which is giving me 4 arrays which hold 4 different kinds of events. I am trying to move them all into another array and tag each event so that I know wh… --- ## [Logstash wildcard](https://discuss.elastic.co/t/logstash-wildcard/246121)
**Author:** [@headtea](https://discuss.elastic.co/u/headtea)\ **Replies:** 3\ **Last updated:** [August 24, 2020, 5:12pm UTC](https://discuss.elastic.co/t/logstash-wildcard/246121 "2020-08-24T17:12:49Z")
Logstash recieves many .log files such as mongod.log and mysqld.log. I deal with it in logstash by having an if statement for each file such as: if \[log\]\[file\]\[path\] =~ "mysqld.log" I've been asked to also ship data fr… --- ## [Logstash http\_poller json parsing error](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990)
**Author:** [@jerin](https://discuss.elastic.co/u/jerin)\ **Replies:** 19\ **Last updated:** [August 24, 2020, 4:27pm UTC](https://discuss.elastic.co/t/logstash-http-poller-json-parsing-error/245990 "2020-08-24T16:27:26Z")
Hello , I am using http\_poller logstash input plugin to poll rest API ( JIRA ) . When i poll it , these are the sample fields maxresults startAt tags issues Fields under issues issues { fields … --- ## [Openam acess log grok filter](https://discuss.elastic.co/t/openam-acess-log-grok-filter/245883)
**Author:** [@nikhilesh](https://discuss.elastic.co/u/nikhilesh)\ **Replies:** 3\ **Last updated:** [August 24, 2020, 3:17pm UTC](https://discuss.elastic.co/t/openam-acess-log-grok-filter/245883 "2020-08-24T15:17:19Z")
Hi, Can any please help on and provide the grok pattern for the below logs. i want to get the uid(uid=12398938) in a separate field or if i can get the complete "uid=12398938,ou=people,dc=abcdef,dc=com" in seperate fie… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=310) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=312)