# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=444 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 445 --- ## [Logstash shutdown](https://discuss.elastic.co/t/logstash-shutdown/208491)
**Author:** [@vatsalnarry2](https://discuss.elastic.co/u/vatsalnarry2)\ **Replies:** 1\ **Last updated:** [November 20, 2019, 4:22am UTC](https://discuss.elastic.co/t/logstash-shutdown/208491 "2019-11-20T04:22:10Z")
I am integrating mysql with elastic search. when i am running below command in command prompt, Logsatsh display query result but after that shutdown.Please advise how to resolve this issue logstash.bat - f D:\\Elasti… --- ## [Logstash has fail safe?](https://discuss.elastic.co/t/logstash-has-fail-safe/208459)
**Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 2\ **Last updated:** [November 20, 2019, 1:55am UTC](https://discuss.elastic.co/t/logstash-has-fail-safe/208459 "2019-11-20T01:55:01Z")
My current setup is the data is from beats(filebeat and winlogbeat) then to logstash for mapping then to elasticsearch. What will happen if elasticsearch is down? Does the data will still be in queue? --- ## [I can't see my pipelines in Kibana](https://discuss.elastic.co/t/i-cant-see-my-pipelines-in-kibana/208586)
**Author:** [@vbryandc](https://discuss.elastic.co/u/vbryandc)\ **Replies:** 1\ **Last updated:** [November 19, 2019, 10:04pm UTC](https://discuss.elastic.co/t/i-cant-see-my-pipelines-in-kibana/208586 "2019-11-19T22:04:13Z")
Hello friends, I want to configure multiple pipes in my pipelines.yml file. He configured the logstash.yml files and added my .conf files. I am also running the logstash service from Windows from a Power Shell console. … --- ## [Unable to connect to azure sql database](https://discuss.elastic.co/t/unable-to-connect-to-azure-sql-database/208565)
**Author:** [@WalterLeff](https://discuss.elastic.co/u/WalterLeff)\ **Replies:** 2\ **Last updated:** [November 19, 2019, 8:19pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-azure-sql-database/208565 "2019-11-19T20:19:41Z")
Hi i'm new to logstash and trying to import data from azure sql db. However i'm getting an error when trying to connect with the database: ELK stack : 7.4.2 Conf file: input { jdbc { jdbc\_connection\_string =\> "jdb… --- ## [Unable to configure Index Lifecycle Management (ILM) with Logstash Elasticsearch output](https://discuss.elastic.co/t/unable-to-configure-index-lifecycle-management-ilm-with-logstash-elasticsearch-output/208564)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 0\ **Last updated:** [November 19, 2019, 7:01pm UTC](https://discuss.elastic.co/t/unable-to-configure-index-lifecycle-management-ilm-with-logstash-elasticsearch-output/208564 "2019-11-19T19:01:37Z")
I'm simply trying to get ILM to work at the moment with Logstash, Elasticsearch, and Filebeat. Or any other beat. This is my elasticsearch output block: output { elasticsearch { hosts =\> \["localhost:9200"\] … --- ## [Logstash Syntax :: Add a new field... which is the product of other fields?](https://discuss.elastic.co/t/logstash-syntax-add-a-new-field-which-is-the-product-of-other-fields/208369)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 2\ **Last updated:** [November 19, 2019, 4:42pm UTC](https://discuss.elastic.co/t/logstash-syntax-add-a-new-field-which-is-the-product-of-other-fields/208369 "2019-11-19T16:42:04Z")
Hello Lords of Logstash, In my data, I need to add a new field that multiplies the integer values of two other fields and a constant. In other words, if I have this: FieldA FieldB ================ 1 5 2… --- ## [GROK pattern to collect whole exception stacktrace](https://discuss.elastic.co/t/grok-pattern-to-collect-whole-exception-stacktrace/208295)
**Author:** [@gyana\_nayak](https://discuss.elastic.co/u/gyana_nayak)\ **Replies:** 1\ **Last updated:** [November 19, 2019, 3:45pm UTC](https://discuss.elastic.co/t/grok-pattern-to-collect-whole-exception-stacktrace/208295 "2019-11-19T15:45:11Z")
Hi All, Below config file not capturing complete stacktrace, it is only collecting single line. Any suggestion how can collect whole stacktrace. filter { #If log line contains tab character followed by 'at' then… --- ## [Use Own Systemd Unit File](https://discuss.elastic.co/t/use-own-systemd-unit-file/208518)
**Author:** [@tomx1](https://discuss.elastic.co/u/tomx1)\ **Replies:** 1\ **Last updated:** [November 19, 2019, 2:17pm UTC](https://discuss.elastic.co/t/use-own-systemd-unit-file/208518 "2019-11-19T14:17:38Z")
I would like to use my own systemd unit file for logstash because I am starting a dedicated logstash process for each configfile via initscript. I noticed that the logstash unit file is generated via /usr/share/logstash… --- ## [File watcher](https://discuss.elastic.co/t/file-watcher/208520)
**Author:** [@Krypton8](https://discuss.elastic.co/u/Krypton8)\ **Replies:** 0\ **Last updated:** [November 19, 2019, 2:04pm UTC](https://discuss.elastic.co/t/file-watcher/208520 "2019-11-19T14:04:56Z")
Hello all, I have a problem regarding file input to get data from several csv files. The goal is pretty simple but i'm struggling. I need to retrieve data from a csv file, easy, I works nicely. The data are stored in … --- ## [How to Parse my JSON stack trace in logstash?](https://discuss.elastic.co/t/how-to-parse-my-json-stack-trace-in-logstash/208451)
**Author:** [@rishab](https://discuss.elastic.co/u/rishab)\ **Replies:** 10\ **Last updated:** [November 19, 2019, 1:39pm UTC](https://discuss.elastic.co/t/how-to-parse-my-json-stack-trace-in-logstash/208451 "2019-11-19T13:39:44Z")
Logstash is not parsing the my full stack trace. The stack trace is parsing but only part is coming of stack trace as full stack trace is delayed. --- ## [Field enclosed in quotes](https://discuss.elastic.co/t/field-enclosed-in-quotes/208499)
**Author:** [@halz](https://discuss.elastic.co/u/halz)\ **Replies:** 0\ **Last updated:** [November 19, 2019, 11:34am UTC](https://discuss.elastic.co/t/field-enclosed-in-quotes/208499 "2019-11-19T11:34:03Z")
Hello I have a csv file where one of the attributes is enclosed in quotes. It contains lyrics of songs and so it can contain the csv separator symbol ("," and ";") and line ends. How do I parse this attribute correctly… --- ## [Stop logstash if elasticsearch is unreachable](https://discuss.elastic.co/t/stop-logstash-if-elasticsearch-is-unreachable/208481)
**Author:** [@kb\_z](https://discuss.elastic.co/u/kb_z)\ **Replies:** 0\ **Last updated:** [November 19, 2019, 10:06am UTC](https://discuss.elastic.co/t/stop-logstash-if-elasticsearch-is-unreachable/208481 "2019-11-19T10:06:56Z")
Hi, I have written a bash script that does the following: for each file in the input folder, feed the file to logstash for processing and send that to elasticsearch. Once the file has been processed it moves it to the… --- ## [Again lot of queuing into logstash](https://discuss.elastic.co/t/again-lot-of-queuing-into-logstash/208455)
**Author:** [@Nishant\_Hurnekar](https://discuss.elastic.co/u/Nishant_Hurnekar)\ **Replies:** 0\ **Last updated:** [November 19, 2019, 8:27am UTC](https://discuss.elastic.co/t/again-lot-of-queuing-into-logstash/208455 "2019-11-19T08:27:36Z")
Hi, Again, I have started facing long queuing into logstash. As per below topic, I had made changes into logstash & issues was resolved that time. Can anyone look & help me out here. Regards --- ## [Logstash Not Running, keep restarting](https://discuss.elastic.co/t/logstash-not-running-keep-restarting/204393)
**Author:** [@skyluke.1987](https://discuss.elastic.co/u/skyluke.1987)\ **Replies:** 7\ **Last updated:** [November 19, 2019, 7:13am UTC](https://discuss.elastic.co/t/logstash-not-running-keep-restarting/204393 "2019-11-19T07:13:15Z")
Hi all, I am facing with this issue that my logstash keep restarting. How do I troubleshoot this? Installing logstash on /usr/share/ is it correct? rather than /etc/? logstash.service - logstash Loaded: loaded (/e… --- ## [Replace filter not working properly after upgrade to logstash 7.1.1](https://discuss.elastic.co/t/replace-filter-not-working-properly-after-upgrade-to-logstash-7-1-1/208209)
**Author:** [@sumitpramanik](https://discuss.elastic.co/u/sumitpramanik)\ **Replies:** 4\ **Last updated:** [November 19, 2019, 3:56am UTC](https://discuss.elastic.co/t/replace-filter-not-working-properly-after-upgrade-to-logstash-7-1-1/208209 "2019-11-19T03:56:39Z")
Hi All, I am trying to replace the timestamp parsed from my message with existing timestamp + year (parsed from the source of the file). This below code was working perfectly in logstash 6.7.2 but after upgrading to 7.… --- ## [Logstash File input plugin not working Windows 10](https://discuss.elastic.co/t/logstash-file-input-plugin-not-working-windows-10/208312)
**Author:** [@Sudheesh\_P](https://discuss.elastic.co/u/Sudheesh_P)\ **Replies:** 2\ **Last updated:** [November 19, 2019, 3:39am UTC](https://discuss.elastic.co/t/logstash-file-input-plugin-not-working-windows-10/208312 "2019-11-19T03:39:25Z")
Hi, I am using Logstash File input plugin for debugging purposes. Currently I migrated to latest Logstash version 7.4.2. But the existing logstash configuration unable to read file from the disk. I am using Windows 10. … --- ## [How to use logstash keystore values in ruby script](https://discuss.elastic.co/t/how-to-use-logstash-keystore-values-in-ruby-script/208329)
**Author:** [@HansPeterSloot](https://discuss.elastic.co/u/HansPeterSloot)\ **Replies:** 5\ **Last updated:** [November 18, 2019, 11:39pm UTC](https://discuss.elastic.co/t/how-to-use-logstash-keystore-values-in-ruby-script/208329 "2019-11-18T23:39:26Z")
Hi, When using an external ruby script parameters can be used with path =\> "script.rb" script\_params =\> { "url\_host\_port" =\> "http://localhost:9200" "url\_username" =\> "{es\_user}" "url\_pass… --- ## [Exec output plugin crashes Logstash](https://discuss.elastic.co/t/exec-output-plugin-crashes-logstash/208359)
**Author:** [@Todd\_Sayers](https://discuss.elastic.co/u/Todd_Sayers)\ **Replies:** 1\ **Last updated:** [November 18, 2019, 11:38pm UTC](https://discuss.elastic.co/t/exec-output-plugin-crashes-logstash/208359 "2019-11-18T23:38:19Z")
I've been using Logstash for over two years, now. I'd now like to have Logstash execute a shell script when certain patterns are seen. To this end, I installed the logstash-output-exec plugin. However, this appears to cr… --- ## [Exclude the line of the log when a non desired value is found](https://discuss.elastic.co/t/exclude-the-line-of-the-log-when-a-non-desired-value-is-found/208365)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 1\ **Last updated:** [November 18, 2019, 11:32pm UTC](https://discuss.elastic.co/t/exclude-the-line-of-the-log-when-a-non-desired-value-is-found/208365 "2019-11-18T23:32:42Z")
Hi, is there a way to exclude a log line if certain value is found? if the percent field receive a number, filter the log. ex: Kerberos 12 if the percent field receive other type dont process the line ex: Deimos n/a… --- ## [Pipeline to update the nested documents](https://discuss.elastic.co/t/pipeline-to-update-the-nested-documents/208375)
**Author:** [@TimurAralov](https://discuss.elastic.co/u/TimurAralov)\ **Replies:** 0\ **Last updated:** [November 18, 2019, 6:54pm UTC](https://discuss.elastic.co/t/pipeline-to-update-the-nested-documents/208375 "2019-11-18T18:54:54Z")
Hi everyone. I'm currently syncing the RDB and Elasticsearch via JDBC plugin databases. Index looks like (names are replaced with readable AAA, BBB and CCC) : PUT indexname { ... "mappings" : { "properties" : { … --- ## [Convert String to Date with ruby](https://discuss.elastic.co/t/convert-string-to-date-with-ruby/208345)
**Author:** [@ja123](https://discuss.elastic.co/u/ja123)\ **Replies:** 0\ **Last updated:** [November 18, 2019, 3:51pm UTC](https://discuss.elastic.co/t/convert-string-to-date-with-ruby/208345 "2019-11-18T15:51:59Z")
I am trying to parse string to date with ruby. With the code event.set("\[parsed\]\[Journey\]\[#{x}\]\[Abfahrtsdatum\]",Date.parse(event.get("\[parsed\]\[Journey\]\[#{x}\]\[Abfahrtsdatum\]"))); I get the following error: Ruby excepti… --- ## [Can I write timeout\_code in aggregate filter in a file and put the path in aggregate filter as parameter?](https://discuss.elastic.co/t/can-i-write-timeout-code-in-aggregate-filter-in-a-file-and-put-the-path-in-aggregate-filter-as-parameter/208354)
**Author:** [@Avinaba\_Ghosh\_Hajra](https://discuss.elastic.co/u/Avinaba_Ghosh_Hajra)\ **Replies:** 0\ **Last updated:** [November 18, 2019, 5:03pm UTC](https://discuss.elastic.co/t/can-i-write-timeout-code-in-aggregate-filter-in-a-file-and-put-the-path-in-aggregate-filter-as-parameter/208354 "2019-11-18T17:03:04Z")
aggregate { task\_id =\> "%{user\_id}" code =\> "map\['clicks'\] ||= 0; map\['clicks'\] += 1;" push\_map\_as\_event\_on\_timeout =\> true timeout\_task\_id\_field =\> "user\_id" timeout =\> 600 # 10 minutes timeout timeout\_tags =\> \['… --- ## [Logstash multiline source validation](https://discuss.elastic.co/t/logstash-multiline-source-validation/208344)
**Author:** [@mkalinowski](https://discuss.elastic.co/u/mkalinowski)\ **Replies:** 2\ **Last updated:** [November 18, 2019, 4:44pm UTC](https://discuss.elastic.co/t/logstash-multiline-source-validation/208344 "2019-11-18T16:44:03Z")
Hello, I am in the process of building ELK infrastructure for my systems. I have several filebeats and so far one logstash. I have several Java servers and apache servers. As we all know, we have something like stack… --- ## [Translate filter is not working with yaml](https://discuss.elastic.co/t/translate-filter-is-not-working-with-yaml/207766)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [November 18, 2019, 3:31pm UTC](https://discuss.elastic.co/t/translate-filter-is-not-working-with-yaml/207766 "2019-11-18T15:31:58Z")
what am I missing here? ### Tranlate and define new field translate { field =\> "\[site\_code\]" dictionary\_path =\> "/tmp/network\_site\_name.yaml" destination =\> "\[site\_name\]" } ### Tranlate and defi… --- ## [Logstash 7.x on raspberry pi 4](https://discuss.elastic.co/t/logstash-7-x-on-raspberry-pi-4/205348)
**Author:** [@wiznil](https://discuss.elastic.co/u/wiznil)\ **Replies:** 3\ **Last updated:** [November 18, 2019, 2:49pm UTC](https://discuss.elastic.co/t/logstash-7-x-on-raspberry-pi-4/205348 "2019-11-18T14:49:35Z")
Hi all, never had any success in getting logstash 7.x installed on raspberry pi 4, is anyone able to get it running? Below are the steps that I have taken: root@wks:/# uname -a Linux home-wks 4.19.57-v7l+ #1244 SMP T… --- ## [How to track down a document based on a Logstash error](https://discuss.elastic.co/t/how-to-track-down-a-document-based-on-a-logstash-error/208139)
**Author:** [@tm8747a](https://discuss.elastic.co/u/tm8747a)\ **Replies:** 3\ **Last updated:** [November 18, 2019, 2:33pm UTC](https://discuss.elastic.co/t/how-to-track-down-a-document-based-on-a-logstash-error/208139 "2019-11-18T14:33:00Z")
I'm getting an error from Logstash that looks like this: Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"np-audit-7.4.0-2019.11.14", :\_type=\>"\_doc", :\_routing=\>nil}, #\ **Author:** [@HansPeterSloot](https://discuss.elastic.co/u/HansPeterSloot)\ **Replies:** 1\ **Last updated:** [November 18, 2019, 2:31pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-in-separate-script-throws-undefined-local-variable-or-method-new-event-block-for-logstash-filt-ers-executioncontext/208324 "2019-11-18T14:31:31Z") Hi, I am migrating my inline ruby script to an external ruby script file. First I create an new event with : service\_trace\_event = LogStash::Event.new After filling the fields like : service\_trace\_event.set("id",re… --- ## [Conditional with a lot of or statements, is there a way to optimize the conditional?](https://discuss.elastic.co/t/conditional-with-a-lot-of-or-statements-is-there-a-way-to-optimize-the-conditional/208266)
**Author:** [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\ **Replies:** 1\ **Last updated:** [November 18, 2019, 2:24pm UTC](https://discuss.elastic.co/t/conditional-with-a-lot-of-or-statements-is-there-a-way-to-optimize-the-conditional/208266 "2019-11-18T14:24:23Z")
Hi, We have a conditional in our pipeline where we check the value of a field against a list of 45+ values. We separate each value like this: if "AAA-5-AAA\_AUTH\_ADMIN\_USER" in \[ciscotag\] OR "APF-4-INVALID\_ENC\_POL\_SET"… --- ## [Unwanted truncating of start of multiline XML message from filebeat](https://discuss.elastic.co/t/unwanted-truncating-of-start-of-multiline-xml-message-from-filebeat/208114)
**Author:** [@Bhozar](https://discuss.elastic.co/u/Bhozar)\ **Replies:** 1\ **Last updated:** [November 18, 2019, 2:09pm UTC](https://discuss.elastic.co/t/unwanted-truncating-of-start-of-multiline-xml-message-from-filebeat/208114 "2019-11-18T14:09:01Z")
I'm having an issue with an XML log file being ingested by Filebeat and some of the messages being passed onto Logstash with the opening tag truncated. There's no pattern to the failures that I can spot and works 95% of … --- ## [How to wait for other dependent event to receive while using script](https://discuss.elastic.co/t/how-to-wait-for-other-dependent-event-to-receive-while-using-script/208318)
**Author:** [@Md\_Iburahimsha\_S](https://discuss.elastic.co/u/Md_Iburahimsha_S)\ **Replies:** 0\ **Last updated:** [November 18, 2019, 1:54pm UTC](https://discuss.elastic.co/t/how-to-wait-for-other-dependent-event-to-receive-while-using-script/208318 "2019-11-18T13:54:03Z")
Currently i am using the below configuration to update the appdata value to other events 000C. Condition like 000A or 000B document ID matched with 000C event Document ID if\[eventid\]=="000A" or \[eventid\]=="000B" { el… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=443) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=445)